Distributed self-organizing body-building intelligent system and method

Abstract

The invention discloses a distributed self-organizing intelligent system and method, and the system comprises an intelligent module which is used for executing a production task, and comprises an automatic chip mounter, a suction nozzle replacement device, and a material handling device; the central control module consists of a server cluster and is used for running a scheduling algorithm and a data analysis task; the distributed cognitive network module comprises a communication unit, an edge server and an environment sensor, and supports edge calculation and real-time cooperation among intelligent agents; the man-machine cooperation module comprises a force feedback control unit, a man-machine interface and a safety protection device and is used for realizing safe man-machine interaction; wherein the system dynamically generates a global state through a distributed cognitive network module, realizes task self-organization distribution based on a swarm intelligence algorithm, and optimizes a production process in combination with edge calculation and a real-time feedback mechanism; the strain capacity of the system is enhanced, and the efficient and stable production process is ensured.

Description

Technical Field

[0001] The present invention relates to the field of industrial automation technology, and in particular to a distributed self-organizing embodied intelligent system and method. Background Art

[0002] Although existing smart factories have made some progress in improving production efficiency, they still have significant problems of lack of flexibility and adaptability when facing rapidly changing small-batch customized orders. Traditional PCBA patch production lines usually rely on fixed layouts and predetermined operating procedures, which are difficult to flexibly adjust to meet the needs of different types of PCB boards or components, resulting in inefficiency in processing small-batch, multi-variety orders, and increased line change time and costs. In addition, the information flow in existing systems is usually one-way, lacking effective two-way communication and real-time feedback mechanisms, making it impossible for devices to share status information in a timely manner, affecting the coordination and response speed of overall production. The static task allocation method also limits the system's self-healing and fault-tolerant capabilities, and is prone to production interruptions when encountering emergencies (such as equipment failures and material shortages). Summary of the invention

[0003] Purpose of the invention: The purpose of the present invention is to provide a distributed self-organizing embodied intelligent system and method to solve the problems of insufficient flexibility and slow response speed in the prior art.

[0004] Technical solution: A distributed self-organizing embodied intelligent system described in the present invention includes: an intelligent module, which includes multiple intelligent body devices with perception, computing and communication capabilities, and is used to perform production tasks, including automatic placement machines, nozzle replacement devices and material handling equipment; a central control module, which is composed of a server cluster and is used to run scheduling algorithms and data analysis tasks; a distributed cognitive network module, which includes a communication unit, an edge server and an environmental sensor, and supports edge computing and real-time collaboration between intelligent bodies. The communication unit realizes low-latency connection between devices through 5G and industrial Internet of Things protocols; a human-computer collaboration module, which includes a force feedback control unit, a human-computer interface and a safety protection device, and is used to realize safe human-computer interaction; wherein, the system dynamically generates a global state through a distributed cognitive network module, realizes self-organizing task allocation based on a swarm intelligence algorithm, and optimizes the production process by combining edge computing with a real-time feedback mechanism.

[0005] Furthermore, the communication unit includes a 5G base station and an industrial switch. The edge server has a built-in GPU or AI accelerator for local data preprocessing and lightweight multimodal large model analysis. The environmental sensors include temperature and humidity sensors, gas sensors, and pressure sensors for real-time monitoring of the workshop environment.

[0006] Furthermore, the force feedback control unit imitates human movements through machine vision and force feedback signals, and the safety protection devices include gratings and safety carpets to ensure the safety of human-machine collaboration.

[0007] Furthermore, the material handling equipment is equipped with a mobile platform, a robotic arm and obstacle avoidance sensors, which can dynamically adjust the path according to the flow of people in the workshop and the location of materials.

[0008] The distributed self-organizing embodied intelligence method described in the present invention is implemented based on a distributed self-organizing embodied intelligence system and includes the following steps: (1) Build a distributed cognitive network module to connect all intelligent agents through 5G and industrial Internet of Things protocols, register the capabilities, performance parameters and location information of each intelligent agent, and generate a global state; (2) Based on real-time data and historical records, a swarm intelligence algorithm is used to dynamically allocate tasks, with the objective function of minimizing total cost or maximizing efficiency, and the value function of the intelligent agent is updated through reinforcement learning; (3) The intelligent agent perceives the environmental state through environmental sensors, generates perception vectors based on the cross-modal dynamic attention mechanism, and dynamically adjusts the device operation parameters; (4) When human intervention is required, the force feedback control unit extracts human motion characteristics and optimizes the collaborative strategy of material handling equipment; (5) Use edge servers to perform lightweight multimodal large model analysis, quickly generate decisions based on local data, and perform real-time optimization through reinforcement learning; (6) Provide real-time production monitoring through the human-machine interface, integrate visualization tools to feedback key performance indicators, and support immediate response to operating instructions.

[0009] Furthermore, in step (2), let T = {t 1 ,t 2 ,...,t m} is the set of tasks to be completed, S is the global state obtained from the initialization phase, Q is the value function of the agent; the objective function F formula is as follows: ; Among them, w j It is the task j The priority weight, f(t j ,a i (j)) is the task t j Assigned to agent a i(j) The cost function, S is the global state obtained in the initialization phase, Q(s i ,a i ) is agent a i In status iis the expected reward for taking an action, reflecting the learning outcome of the swarm intelligence application; α is a balancing factor used to adjust the importance of immediate task allocation and long-term learning; m and n represent the number of tasks and states, respectively.

[0010] Furthermore, in step (3), the cross-modal dynamic attention mechanism generates a perception vector by fusing sensor data with the internal state of the agent. The formula is as follows: ; Among them, E represents the state of the environment, S i is an intelligent agent i The internal state of is the perception function, which converts the environment information and the agent state into the perception vector V i ; g(E,S i ) is the estimated perception vector V i Given an environment E and an agent state S i The probability distribution under is the probability distribution function: .

[0011] Furthermore, in step (4), the robot collaboration strategy is optimized by the following formula: r According to the human-machine collaboration state H and the perception vector V r Choose the best action a r ∗ : ; in It means to find a strategy to maximize the objective function; is the collaboration evaluation function, which measures the quality of human-machine collaboration. r ) is the cost function of the robot action, λ is the factor that adjusts the balance between collaboration quality and cost; for: ; Where P match is the action matching degree, i.e., the human action similarity calculated by machine vision, F feedback is the strength of the force feedback signal, w p and w f is the corresponding weight.

[0012] Furthermore, in step (5), the lightweight multimodal large model is compressed through quantization and pruning techniques and deployed on the edge server to generate local decision recommendations in real time.

[0013] Furthermore, in step (6), the human-machine interface integrates a dynamic dashboard to support production strategy adjustments based on real-time data, and displays equipment utilization, task completion rate, and quality inspection results through visual charts.

[0014] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: The present invention realizes a highly dynamic automated production process through the established distributed cognitive network, self-organizing task planning and real-time optimization supported by edge computing. The intelligent agent self-organizes and allocates tasks based on the global state and real-time data, and can quickly respond to changing needs without relying on the central control system. Through the cross-modal dynamic attention mechanism, multiple sensors can work together to ensure accurate information transmission and make optimal decisions. In addition, the lightweight edge multimodal large model performs local data analysis on the edge node, enabling the intelligent agent to adapt to environmental changes in real time, thereby improving overall production efficiency and flexibility. This dynamic automation not only reduces the need for human intervention, but also enhances the system's resilience and ensures an efficient and stable production process. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a system structure diagram of the present invention; Figure 2 The figure is a block diagram of the method principle of the present invention. DETAILED DESCRIPTION

[0016] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0017] like Figure 1 As shown, an embodiment of the present invention provides a distributed self-organizing embodied intelligent system including an intelligent module, a central control module, a distributed cognitive network module and a human-machine collaboration module. The intelligent module includes multiple types of intelligent business equipment, including an automatic placement machine, a nozzle replacement device, and material handling equipment. The central control module includes a server cluster. The distributed cognitive network module includes a communication unit, an edge server and an environmental sensor. The human-machine collaboration module includes a force feedback control unit, a human-machine interface and a safety protection device. The following is an embodiment of the embodied intelligent self-organizing production line system and method based on a distributed cognitive network of the present invention, which is used to describe the specific implementation process.

[0018] For example, in the patch industry production process, when there is a new production demand, a new production plan needs to be formulated based on the current production status.

[0019] Intelligent modules are used to complete production operations, including automatic placement machines, nozzle replacement devices, and material handling equipment. The automatic placement machines can be used to automatically place components at high speed and high precision, and the Panasonic placement machine NPM series GH model is preferred; the nozzle replacement device can be used to quickly replace nozzles of different specifications of the placement machine, and is installed in the automatic placement machine to quickly adapt to various component types in different production tasks; the material handling equipment has accessories such as mobile platforms, robotic arms and sensor kits, which are used to grab, transport and place materials, and have the capabilities of obstacle avoidance and environmental perception.

[0020] The central control module is used for centralized control, including a server cluster, which is a high-performance computing server cluster used to run complex scheduling algorithms and data analysis tasks.

[0021] The distributed cognitive network module is used for distributed perception, computing and communication, including communication units, edge servers and environmental sensors; the communication unit provides high-speed, low-latency network connections between devices to ensure the reliability and stability of data transmission, including 5G base stations and industrial switches; the edge server is a miniaturized, high-performance computing device deployed close to the data source to reduce the data transmission distance and speed up processing. Built-in GPU or AI accelerator supports local data preprocessing and preliminary analysis. Environmental sensors include temperature and humidity sensors, gas sensors and pressure sensors, etc., which monitor the environmental conditions in the production workshop to ensure compliance with production process requirements and protect the health and safety of workers.

[0022] The human-machine collaboration module is used to provide smooth and efficient human-machine collaboration, including a force feedback control unit, a human-machine interface and safety protection devices. The force feedback unit enables the robot to sense and respond to the strength of the human engineer's movements to imitate human movement patterns; the human-machine interface integrates a dynamic dashboard to support production strategy adjustments based on real-time data, and displays equipment utilization, task completion rate and quality inspection results through visual charts; safety protection devices include physical protection measures such as gratings and safety carpets to protect the safety of engineers.

[0023] The embodiment of the present invention also provides a distributed self-organizing embodied intelligence method, which is implemented based on a distributed self-organizing embodied intelligence system. Figure 2 As shown, the following steps are included: Step 1: Distributed cognitive network initialization. First, the network is constructed to establish a distributed cognitive network module covering all intelligent modules (including automatic placement machines, nozzle replacement devices, material handling equipment, etc. in the embodiment of the present invention), and an efficient and low-latency communication protocol (including 5G and proprietary industrial Internet of Things IoT protocol switches in the embodiment of the present invention) is used to ensure stable connection between nodes. Next, intelligent agent registration is performed. Each intelligent agent a iRegister its capabilities and services in the network C i , including but not limited to processing task types, performance parameters, and current positions. Generate a global state 𝑆, which contains information about all agents and their attributes: ; Among them, C i Represents agent a i The ability of P i Indicates its performance parameters, L i represents its position, and A is the set of agents.

[0024] Step 2: Self-organizing task planning. First, dynamic tasks (the tasks in the embodiment of the present invention include new orders, material shortages, equipment failures, etc.) are allocated. Based on the distributed cognitive network that has been built and initialized, the central control module is no longer the only decision-making center. Instead, it participates in the decision-making process with each intelligent agent and negotiates the best production plan based on real-time information and historical data. Intelligent agents can communicate directly with each other and respond quickly to changing needs. The specific process is as follows: Set the task set: Let T = {t 1 ,t 2 ,...,t m} is a set of tasks to be completed, each task t j The corresponding priority, required resources, and estimated completion time are included. Determine the objective function: It is defined as minimizing the total cost of all tasks or maximizing efficiency, and taking into account the learning results of the agent: ; Among them, w j It is the task j The priority weight, f(t j ,a i (j)) is the task t j Assigned to agent a i(j) The cost function, S is the global state obtained in the initialization phase, Q(s i ,a i ) is agent a i In status i is the expected reward for taking an action, reflecting the learning outcome of the swarm intelligence application; α is a balancing factor used to adjust the importance of immediate task allocation and long-term learning; m and n represent the number of tasks and states, respectively.

[0025] Secondly, the swarm intelligence algorithm is used to continuously optimize dynamic task allocation. The agents can collectively learn and adapt to changes in the environment to optimize overall production efficiency. The agents adapt to environmental changes by iteratively updating their own strategies to optimize long-term rewards. Long-term optimization means that the agents update their own behavior strategies π through swarm intelligence algorithms to maximize long-term rewards: ; where π ∗ is the optimal strategy, γ is the discount factor, R t is the immediate reward obtained at time step t, It means to find a strategy to maximize the objective function. represents the expected value of the sum of the above infinite sequence under the given strategy π.

[0026] After each task is completed, the agent updates its value function Q(s,a) based on the actual results through a feedback mechanism to better guide future task allocation: ; Where β is the learning rate, R is the immediate reward, s′ is the new state, a′ is the possible action to be taken in the new state; γ is the discount factor, and max here means taking the maximum value of the Q value of all possible actions a′ in the new state s′.

[0027] In summary, in the embodiment of the present invention, the process of self-organizing task planning is that in each new task allocation process, the system first uses the current global state S and the value function Q of the agent to calculate the objective function F, thereby determining the best task allocation solution. As the agents perform tasks and collect feedback, they will continuously update their own value functions Q. These updates reflect the learning outcomes and adaptive changes of the agents, providing more accurate data support for subsequent task allocation. Through continuous task allocation and feedback learning, the system gradually optimizes the agent's behavior strategy π, improving overall production efficiency and flexibility.

[0028] Step 3: Situational perception and response. The intelligent agent not only relies on preset programs to work, but also can perceive the surrounding environment through environmental sensors and make the best response that suits the current situation. In an embodiment of the present invention, the automatic placement machine can fine-tune the nozzle angle according to the actual component position; the material handling equipment can adjust the route according to the flow of people and vehicles in the workshop.

[0029] First, establish the perception model, agent a i Use its sensors to obtain the environment state E and generate a perception vector V i : ; Among them, E represents the state of the environment (including object position, temperature, humidity, etc.), S i is an intelligent agent i The internal state of (including current location, task progress, etc.), is the perception function, which converts the environment information and the agent state into the perception vector V i . g(E,Si ) is the estimated perception vector V i Given an environment E and an agent state S i The probability distribution under is the probability distribution function: .

[0030] Step 4: Human-machine collaboration optimization. When human intervention is required, the material handling equipment can not only safely interact with human engineers, but also imitate human motion patterns through machine vision and force feedback technology to provide a more natural collaboration experience. In the embodiments of the present invention, it can be used for task scenarios including initial positioning teaching of chip placement, coordination of material handling and assembly lines, and interactive correction in quality inspection; the details are as follows: First, a human-computer interaction model is established to define a human-computer collaboration state H, which includes the position L of the human operator. h , Posture P h , and robot a r Status S r : ; Secondly, establish the feature V of force feedback and machine vision r The robot uses force feedback Ff and machine vision V v Obtain the action characteristics of human operators and imitate them to optimize their own behavior: ; in is a convolutional neural network. In the embodiment of the present invention, the ResNet-50 algorithm is used. f and W h are the weight matrices of force feedback and collaborative state, b k is the bias term.

[0031] Its collaborative strategy is robot a r According to the human-machine collaboration state H and the perception vector V r Choose the best action a r ∗ , to optimize the collaborative experience, robot a r According to the human-machine collaboration state H and the perception vector V r Choose the best action a r ∗ : ; in It means to find a strategy to maximize the objective function; is the collaboration evaluation function, which measures the quality of human-machine collaboration. r) is the cost function of the robot action, and λ is a factor that adjusts the balance between collaboration quality and cost. for: ; Where P match is the action matching degree (human action similarity calculated by machine vision), F feedback is the strength of the force feedback signal, w p and w f is the corresponding weight.

[0032] Finally, with continuous learning and adaptation optimization, through human-machine collaboration, the robot continuously learns human behavior patterns and integrates them into its own behavior strategy, thereby improving collaboration efficiency. The updated value function Q is: ; Where η is the learning rate, R h is the immediate reward brought by human-machine collaboration, γ is the discount factor, H′ is the new collaborative state, a′ is the possible action to be taken in the new state, and max here means taking the maximum value of the Q values ​​of all possible actions in the new state.

[0033] Step 5: Real-time optimization supported by edge computing. In order to improve response speed and reduce dependence on the central control module, the agent performs preliminary data analysis and decision making on the edge computing node close to the data source. This not only reduces the burden on the central control module, but also enables the agent to adapt to changes in the local environment more quickly. First, a lightweight edge multimodal large model is established. Agent a i Collect local data D on edge nodes i , and use a lightweight edge multimodal large model M edge Conduct preliminary analysis: ; Where D i is an intelligent agent i The collected local data (including sensor readings, operation records, etc.), S i is the state of the agent, A i It is a result or decision suggestion based on edge analysis. edge It is a lightweight customized 1.5B multimodal large model through quantization and pruning.

[0034] Its response strategy is based on edge analysis results A i , agent a i Use reinforcement learning algorithms to perform real-time optimization and select the optimal action a i ∗ , to maximize the immediate reward R and the expected reward Q for long-term learning: ; Among them, f(t j ,a i ) is the task execution cost function obtained from dynamic task allocation, α is the balance factor, Q(s i ,a i ) is agent a i In status i After each task is completed, the agent will update its value function Q according to the actual results through feedback updates to better guide future task allocation: ; Where β is the learning rate, R is the immediate reward, and γ is the discount factor representing the importance of the current decision to the future reward. s′ is the new state, a′ is the possible action to be taken in the new state; max here means taking the maximum value of the Q value of all possible actions a′ under the new state s′.

[0035] Step 6: Real-time feedback mechanism In order to provide an intuitive and efficient user experience, a graphical user interface is used as the core platform for human-machine interaction to achieve real-time feedback and control. Operators can monitor the status and task progress of each agent through this interface and receive instant notifications and warnings from the system. The graphical user interface integrates visualization tools such as charts and dashboards to clearly display key performance indicators and production data. More importantly, users can use the graphical user interface to directly send instructions to the system or adjust parameters, all of which can be responded to quickly and reflected in the actual production process. In this way, not only the convenience and transparency of operation are improved, but also the close cooperation between people and machines is promoted, ensuring smoother and more efficient production activities.

Claims

1. A distributed self-organizing embodied intelligent system, characterized in that: include: The intelligent module contains multiple intelligent devices with perception, computing and communication capabilities, which are used to perform production tasks, including automatic placement machines, nozzle replacement devices and material handling equipment; the central control module is composed of a server cluster and is used to run scheduling algorithms and data analysis tasks; the distributed cognitive network module contains communication units, edge servers and environmental sensors, which support edge computing and real-time collaboration between intelligent agents. The communication unit realizes low-latency connection between devices through 5G and industrial Internet of Things protocols; the human-machine collaboration module includes a force feedback control unit, a human-machine interface and a safety protection device, which is used to achieve safe human-machine interaction; among them, the system dynamically generates a global state through a distributed cognitive network module, realizes self-organizing task allocation based on a swarm intelligence algorithm, and optimizes the production process by combining edge computing with a real-time feedback mechanism.

2. A distributed self-organizing embodied intelligent system according to claim 1, characterized in that: The communication unit includes a 5G base station and an industrial switch. The edge server has a built-in GPU or AI accelerator for local data preprocessing and lightweight multimodal large model analysis. The environmental sensors include temperature and humidity sensors, gas sensors, and pressure sensors for real-time monitoring of the workshop environment.

3. A distributed self-organizing embodied intelligent system according to claim 1, characterized in that: The force feedback control unit imitates human movements through machine vision and force feedback signals. The safety protection devices include light barriers and safety carpets to ensure the safety of human-machine collaboration.

4. A distributed self-organizing embodied intelligent system according to claim 1, characterized in that: The material handling equipment is equipped with a mobile platform, a robotic arm and obstacle avoidance sensors, and can dynamically adjust the path according to the flow of people in the workshop and the location of materials.

5. A distributed self-organizing embodied intelligence method, implemented based on a distributed self-organizing embodied intelligence system, characterized in that: The following steps are involved: (1) Build a distributed cognitive network module to connect all intelligent agents through 5G and industrial Internet of Things protocols, register the capabilities, performance parameters and location information of each intelligent agent, and generate a global state; (2) Based on real-time data and historical records, a swarm intelligence algorithm is used to dynamically allocate tasks, with the objective function of minimizing total cost or maximizing efficiency, and the value function of the agent is updated through reinforcement learning; (3) The intelligent agent perceives the environmental state through environmental sensors, generates perception vectors based on the cross-modal dynamic attention mechanism, and dynamically adjusts the device operation parameters; (4) When human intervention is required, the force feedback control unit extracts human motion characteristics and optimizes the collaborative strategy of material handling equipment; (5) Use edge servers to perform lightweight multimodal large model analysis, combine local data to quickly generate decisions, and perform real-time optimization through reinforcement learning; (6) Provide real-time production monitoring through the human-machine interface, integrate visualization tools to feedback key performance indicators, and support immediate response to operating instructions.

6. A distributed self-organizing embodied intelligence method according to claim 5, characterized in that: In step (2), let T = {t1, t2, ..., t m } is the set of tasks to be completed, S is the global state obtained from the initialization phase, Q is the value function of the agent; the objective function F formula is as follows: ； Among them, w j It is the task j The priority weight, f(t j ,a i (j)) is the task t j Assigned to agent a i(j) The cost function, S is the global state obtained in the initialization phase, Q(s i ,a i ) is agent a i In status i is the expected reward for taking an action, reflecting the learning outcome of the swarm intelligence application; α is a balancing factor used to adjust the importance of immediate task allocation and long-term learning; m and n represent the number of tasks and states, respectively.

7. A distributed self-organizing embodied intelligence method according to claim 5, characterized in that: In step (3), the cross-modal dynamic attention mechanism generates a perception vector by fusing sensor data with the internal state of the agent. The formula is as follows: ； Among them, E represents the state of the environment, S i is an intelligent agent i The internal state of is the perception function, which converts the environment information and the agent state into the perception vector V i ; g(E,S i ) is the estimated perception vector V i Given an environment E and an agent state S i The probability distribution under is the probability distribution function: 。 8. A distributed self-organizing embodied intelligence method according to claim 5, characterized in that: In step (4), the robot collaboration strategy is optimized by the following formula: r According to the human-machine collaboration state H and the perception vector V r Choose the best action a r ∗ : ； in, It means to find a strategy to maximize the objective function; is the collaboration evaluation function, which measures the quality of human-machine collaboration. r ) is the cost function of the robot action, λ is the factor that adjusts the balance between collaboration quality and cost; for: ； Where P match is the action matching degree, i.e., the human action similarity calculated by machine vision, F feedback is the strength of the force feedback signal, w p and w f is the corresponding weight.

9. A distributed self-organizing embodied intelligence method according to claim 5, characterized in that: In step (5), the lightweight multimodal large model is compressed through quantization and pruning techniques and deployed on the edge server to generate local decision recommendations in real time.

10. A distributed self-organizing embodied intelligence method according to claim 5, characterized in that: In step (6), the human-machine interface integrates a dynamic dashboard to support production strategy adjustments based on real-time data, and displays equipment utilization, task completion rate and quality inspection results through visual charts.

Images