An industrial robot collaborative system based on artificial intelligence

Abstract

The present invention discloses an industrial robot collaborative system based on artificial intelligence, and relates to the field of industrial automation control technology. The components of the system include: an environment perception module, a distributed decision trigger module, a local decision generation module, a collaborative strategy optimization module and an execution and feedback module. The present invention abandons the drawbacks of traditional centralized control through a distributed collaborative decision-making mechanism, and gives each industrial robot the ability to make autonomous decisions. Each robot can make decisions quickly and collaborate with other robots based on real-time environmental information and its own status, thereby shortening the decision-making time and improving production efficiency. At the same time, the collaborative optimization module based on reinforcement learning enables the robot to continuously learn and optimize the collaborative strategy during the operation process, so that the system can flexibly respond to various complex production tasks and dynamically changing environments, and significantly improve the overall efficiency of industrial production.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial automation control, and in particular to an industrial robot collaborative system based on artificial intelligence. Background Art

[0002] As today's manufacturing industry continues to move towards intelligence and automation, the demand for complexity and efficiency in industrial production is increasing day by day. With the accelerated pace of product updates and increasingly sophisticated production processes, the market has put forward higher requirements for the flexibility, rapid response capabilities and stability of product quality of industrial production. Industrial robots are key equipment for realizing automated production. The efficiency and quality of collaborative operations directly affect the production efficiency and market competitiveness of enterprises. In many production scenarios, from the complex assembly process of automobile manufacturing to the high-precision processing links in the production of electronic products, there is an urgent need for industrial robots to be able to work closely together to adapt to diverse production tasks and dynamically changing production environments.

[0003] However, the technology currently used in the collaborative work of industrial robots has significant shortcomings. In terms of decision-making mechanism, the traditional centralized control mode relies on the central controller to uniformly coordinate tasks. Although this method is effective in simple scenarios, when faced with complex production tasks, the central controller is very likely to have decision-making delays due to excessive data processing. Once the central controller fails, the entire system will be paralyzed. At the same time, in terms of collaborative optimization, manually adjusting robot parameters and strategies not only consumes a lot of manpower and material resources, but also due to the limitations of human factors, it is difficult to fully consider various complex production situations, resulting in the collaborative work of robots unable to achieve the optimal state in terms of efficiency, accuracy and energy consumption.

[0004] To sum up, the existing industrial robot collaborative technology has exposed the problems of inflexible decision-making mechanism and insufficient collaborative optimization capabilities when facing the complex and changeable needs of modern industrial production. These problems have seriously restricted the further development of industrial production. An innovative industrial robot collaborative system is urgently needed to improve its decision-making efficiency and collaborative optimization level in complex environments, so as to meet the manufacturing industry's urgent needs for efficient, intelligent and flexible production. Summary of the invention

[0005] The purpose of the present invention is to make up for the shortcomings of the prior art and to provide an industrial robot collaborative system based on artificial intelligence. It can abandon the drawbacks of traditional centralized control through a distributed collaborative decision-making mechanism, and give each industrial robot the ability to make autonomous decisions. Each robot can make decisions quickly and collaborate with other robots based on real-time environmental information and its own status, thereby shortening the decision-making time and improving production efficiency. At the same time, the collaborative optimization module based on reinforcement learning enables the robots to continuously learn and optimize collaborative strategies during the operation process, so that the system can flexibly respond to various complex production tasks and dynamically changing environments, significantly improving the overall efficiency of industrial production.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: an industrial robot collaborative system based on artificial intelligence, the components of which include: an environment perception module, a distributed decision triggering module, a local decision generation module, a collaborative strategy optimization module and an execution and feedback module;

[0007] The environmental perception module installs a visual camera for each industrial robot to identify materials and environmental identification, a laser radar to detect obstacles and spatial distances, and a force sensor to sense the force during operation. The sensor data is collected through a sensor driver and the collected data is preprocessed, and the preprocessing includes format conversion and data integration;

[0008] The distributed decision trigger module sets the decision trigger conditions and monitors the pre-processed data in real time through the decision trigger program. The decision trigger conditions include material shortage, task timeout and production order change. When the pre-processed data meets the trigger conditions, a decision trigger signal is sent to the local decision generation module of each robot.

[0009] The local decision generation module assumes the local decision generation function for the intelligent decision module of each industrial robot, and generates a local decision plan according to the current task state of the robot and the received trigger signal. The local decision generation function uses a decision tree algorithm to determine whether to accept a new task and the preliminary task execution order;

[0010] The collaborative strategy optimization module builds a high-speed communication network between the robots, adopts a deep reinforcement learning algorithm, takes the local decision-making plan of each robot as input, and uses the task completion index as a reward signal to perform training and optimization to generate a global optimal collaborative strategy. The specific generation process of the global optimal collaborative strategy is as follows:

[0011] Local decision plan collection: Each industrial robot sends the local decision plan generated by its own local decision generation module to the collaborative strategy optimization module through the communication interface. The local decision plan includes the preliminary decision made by each robot based on its own state and the received trigger signal;

[0012] State space construction: The collaborative strategy optimization module integrates the local decision-making plans of all industrial robots and the current production environment information to construct the state space required by the deep reinforcement learning algorithm. The elements in the state space include the position, task status, remaining power, load status of each industrial robot, as well as the overall progress of the production task and the information of material distribution. The state space S = {s 1 ,s 2 ,…,s n}, where s i Represents a specific state element in the state space;

[0013] Action space definition: Define the set of actions taken by robots when working together, that is, the action space A = {a 1 ,a 2 ,…,a m}, the action space includes the adjustment of the task allocation of each robot, the re-planning of the task execution order, and the change of the operating parameters, that is, the action a in the action space A j Indicates that the industrial robot R k Task T l Reassigned to industrial robots R m ;

[0014] Reward function setting: Set the reward function R(S,A) according to the task completion index to measure the effect of taking action a j After the state s i The reward value r obtained by transferring to a new state, the reward function Among them, α, β, γ are the weight coefficients of the reward function, t is the task completion time, p is the task accuracy, and e is the energy consumption;

[0015] Iterative optimization of deep reinforcement learning: Iterative training and optimization are performed using deep reinforcement learning algorithms, so that the industrial robot updates its own strategy network parameters according to the feedback reward value after each action, and gradually learns the optimal collaborative strategy;

[0016] Global optimal collaborative strategy generation: After multiple deep reinforcement learning iterative optimizations, when the algorithm converges, the obtained strategy network This is the generated global optimal collaborative strategy, which determines the optimal actions that each industrial robot should take under different conditions, thereby achieving efficient collaborative operation between robots;

[0017] The execution and feedback module writes task execution programs for the robot and controls the robot's movements according to the collaborative strategy. During the robot's task execution, the execution status data is collected in real time through the built-in status monitoring program and fed back to the environment perception module and the collaborative strategy optimization module through the communication network.

[0018] Furthermore, the distributed decision trigger module continuously monitors the data preprocessed by the environment perception module through the decision trigger program, compares the monitored data with the trigger condition, and when the data reaches and exceeds the threshold of the trigger condition, determines that the trigger condition is met, thereby starting the sending operation of the decision trigger signal to notify the local decision generation module to make corresponding decision adjustments, wherein:

[0019] For the material shortage trigger condition, the current inventory data is compared with the inventory threshold to determine whether the trigger threshold is reached, that is, For time t wcsj The current inventory quantity of the material at the time, S min is the inventory threshold of the material. , it is determined that the material shortage trigger condition is met;

[0020] For the task timeout trigger condition, the difference between the current time and the task start time is compared with the standard operation time, that is, t kssj is the task start time, t dqsj is the current time, T bzsj is the standard operation time of the task. For a simple task in a single stage, when t dqsj -t kssj ≥T bzsj When , the task is judged to have timed out and the task timeout trigger condition is met; for complex tasks with multiple stages, is the start time of the i-th task phase, is the end time of the i-th task stage, is the standard operation time of the i-th task stage. When , it is determined that the task of this stage has timed out and the task timeout triggering condition is met;

[0021] For the production order change trigger condition, by comparing the order information received twice, when the order information changes, that is, O old For old order information, new For new order information, when O is satisfied new ≠O old , it is determined that the production order change triggering condition is met.

[0022] Furthermore, the local decision generation module assumes the local decision generation function of the intelligent decision module in the industrial robot. The intelligent decision module is an independent processing engine embedded in each industrial robot, which has the functions of data processing and analysis, logical judgment and decision output, and is composed of a knowledge base, a learning unit and a communication interface, wherein:

[0023] The knowledge base stores knowledge and rules related to robot tasks, performance, and production processes. The knowledge includes both the set production process specifications and deterministic rules of robot operation restrictions. The knowledge base will be continuously updated and improved as the robot operates and learns;

[0024] The learning unit optimizes the decision by learning the historical decision data and the real-time production data, and can automatically adjust the decision parameters;

[0025] The communication interface is responsible for data interaction with other modules in the system. On the one hand, it receives real-time environmental data from the environmental perception module and the trigger signal of the distributed decision trigger module, and on the other hand, it passes the generated local decision plan to the collaborative strategy optimization module.

[0026] Furthermore, the working principle of the local decision generation module is:

[0027] Data reception and integration: The intelligent decision module receives the decision trigger signal from the distributed decision trigger module through the communication interface, and clarifies the task type for decision processing, which includes material shortage, task timeout and production order change. At the same time, it obtains real-time data about the robot's own status from the environmental perception module, including the robot's current position, task execution progress, remaining power, load status and material location, and integrates the trigger signal with the real-time data to form a complete decision data set;

[0028] Preliminary decision-making based on knowledge base matching: Based on the received decision-making task type, the knowledge base is searched for matching rules and knowledge. Based on the matched rules, the integrated decision data set is preliminarily analyzed and judged to generate a preliminary decision direction.

[0029] Decision optimization: extract key features from the current decision data set, combine historical decision data, and use decision tree algorithms to optimize preliminary decisions;

[0030] Decision generation and output: The preliminary decision of the knowledge base matching and the results of decision optimization are integrated to generate the final local decision plan, which includes the specific decision content of whether to accept the new task, adjust the task execution order, and change the operation mode;

[0031] Decision coordination and transmission: The intelligent decision-making module sends the local decision plan to the collaborative strategy optimization module through the communication interface, and coordinates and optimizes the decisions of other robots to jointly achieve efficient operation of the entire system.

[0032] Furthermore, the decision tree algorithm is an artificial intelligence algorithm based on a tree structure for decision making, which utilizes the decision data set D = {x 1 ,x 2 ,x3 ,…,x n}, where x i Represent the task characteristics in the decision data set, and optimize the initial decision to a decision that conforms to historical experience and current conditions by continuously dividing the decision data set and adjusting the division and decision rules according to the optimal decision results of historical data;

[0033] Each node of the decision tree represents a test of the task feature. j , whose test function is f j (D) = {True, False}, the dataset is divided into two subsets according to the value of the task feature, namely D j,True and D j,False , for node N j , when f j (D) = True, data enters D j,True , otherwise enter D j,False ;

[0034] Each leaf node L k Represents a decision result, and its decision result function is expressed as g k (D) = {y 1 ,y 2 ,y 3}, where y 1 Indicates whether to accept the new task, y 2 Indicates whether to adjust the task execution order, y 3 Indicates whether to change the operation mode;

[0035] Based on the historical decision data, the decision tree is trained. For each historical sample D s ={x s1 ,x s2 ,x s3 ,…,x sn} and its corresponding optimal decision result y s ={y s1 ,y s2 ,y s3}, optimize the decision tree by minimizing the error function, the error function Among them, m represents the number of samples of historical decision data, g k (D s ) is the decision tree for historical samples D s The decision result given, y s is the actual optimal decision result of the sample. By minimizing the error function, the node test function f of the decision tree is continuously adjusted. j And the decision result function g of the leaf node k , in order to optimize the initial decision.

[0036] Furthermore, the optimization process of the decision tree algorithm for the preliminary decision is:

[0037] Select a task feature from the decision data set D as the test function f of the root node 1 , divide the data set into for each subset D 1,True and D 1,False , continue to select task features as the test function of the next layer of nodes, and so on recursively until all samples in the subset belong to the same type of decision results, that is, they all have the same y s and the number of samples in the subsets is done recursively;

[0038] At each leaf node, count the number of various decision results of the samples in the subset, and select the decision result with the largest number of occurrences as the g of the leaf node. k value and use it as the final decision result of the node.

[0039] Furthermore, the collaborative strategy optimization module uses the deep reinforcement learning algorithm to train the current state s i Select an action a from the action space A j Execute and observe that the system state transfers to the new state s i+1 , and obtain the corresponding reward value r, by continuously adjusting the policy network parameters, so that the long-term cumulative reward is maximized, that is The long-term cumulative reward is maximized according to the policy network π θ (a|s), which represents the probability distribution of selecting action a according to parameter θ in state s, and finding the optimal parameter θ * , so that the expected cumulative reward is maximized, where T is the total time of task execution, γ zk is a discount factor with a value range of [0, 1], which is used to measure the importance of future rewards. The closer γ is to 1, the more attention is paid to future rewards; the closer γ is to 0, the more attention is paid to current rewards. t and a t Respectively represent the state and action taken at time t;

[0040] To update the policy network parameters θ, the gradient ascent method is used, and the policy network parameters are updated as Among them, θ k+1 and θ k are the policy network parameters before and after the update, respectively. η is the learning rate, which controls the step size of each parameter update. A learning rate that is too large may cause the algorithm to be unstable, while a learning rate that is too small will slow down the convergence speed. is the log probability gradient of the policy network, and r is the reward value obtained after executing action a.

[0041] Furthermore, the specific steps of optimizing the collaborative strategy optimization module using the deep reinforcement learning algorithm are as follows:

[0042] Initialization: Initialize the policy network π θ The parameter θ of (a|s) sets the discount factor γ and the learning rate η. At the same time, the state space S, the action space A and the reward function R(S,A) are initialized.

[0043] Strategy selection and execution: At each time t, according to the current state s t , through the policy network π θ (a|s) from the action space A with probability π θ (a t |s t )Select action a t ;

[0044] State transfer and reward acquisition: Execute action a t After that, the system status changes from s t Transfer to new state s t+1 At the same time, the reward value r is calculated according to the reward function R(S,A). According to the time, accuracy and energy consumption indicators of task completion under the new state, the reward value r is calculated by substituting it into the reward function t ;

[0045] Parameter update: Using the policy gradient theorem, according to the reward value r obtained by executing the action t and the log probability gradient of the policy network Update the policy network parameters θ;

[0046] Iterative optimization: Repeat the steps of strategy selection and execution, state transfer and reward acquisition, and parameter update for multiple iterations until the algorithm converges, and finally obtain the optimal strategy network that can maximize the long-term cumulative reward That is the global optimal collaborative strategy.

[0047] Compared with the prior art, this industrial robot collaborative system based on artificial intelligence has the following beneficial effects:

[0048] 1. The present invention abandons the drawbacks of traditional centralized control through a distributed collaborative decision-making mechanism, and gives each industrial robot the ability to make independent decisions. Each robot can make decisions quickly and collaborate with other robots based on real-time environmental information and its own status, shortening the decision-making time and improving production efficiency. At the same time, the collaborative optimization module based on reinforcement learning enables the robots to continuously learn and optimize collaborative strategies during the operation process, so that the system can flexibly respond to various complex production tasks and dynamically changing environments, significantly improving the overall efficiency of industrial production.

[0049] 2. The industrial robot collaborative system of the present invention adopts a decision trigger and local decision generation mechanism. Through the set decision trigger conditions, the system can monitor the production status in real time and automatically trigger the decision program when the conditions are met. The local decision generation module intelligently generates local decision plans based on the current task status of the robot and the received trigger signal by utilizing the decision tree algorithm. This mechanism enables the system to flexibly adjust the decision strategy according to the real-time production situation, optimize the task execution sequence and resource allocation, and improve production efficiency and task execution effects. In addition, through the collaborative strategy optimization module, the deep reinforcement learning algorithm is used for training and optimization. The system can continuously learn and adapt to complex and changeable production environments, realize the global optimal collaborative strategy, and further enhance the overall performance and competitiveness of the production line.

[0050] Other advantages, objectives and features of the present invention will be set forth in part in the following description and, in part, will be apparent to those skilled in the art based on an examination of the following or may be taught from the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 It is an operation flow chart of an industrial robot collaborative system based on artificial intelligence;

[0053] Figure 2 This is a block diagram of the composition of an industrial robot collaborative system based on artificial intelligence. DETAILED DESCRIPTION

[0054] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation mode, structure, characteristics and effects of the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments.

[0055] Embodiment 1

[0056] This embodiment focuses on the scenario of an automobile parts manufacturing factory, and deeply explains the practical application of an industrial robot collaborative system based on artificial intelligence. The system covers an environmental perception module, a distributed decision trigger module, a local decision generation module, a collaborative strategy optimization module and an execution and feedback module. The modules work closely together to collect data through environmental perception. The distributed decision trigger module determines decision requirements, the local decision generation module formulates preliminary plans, the collaborative strategy optimization module generates the global optimal strategy, and the execution and feedback module ensures task execution and feedback optimization, improves production efficiency, optimizes resource allocation, and enhances system adaptability, providing technical support for industrial intelligent production.

[0057] In the specific implementation, the environment perception module is used in the automotive parts manufacturing factory. Each industrial robot is equipped with a visual camera, a laser radar and a force sensor. The visual camera is responsible for identifying various automotive parts on the production line, such as engine cylinders, transmission gears and environmental signs, such as signs and safety warning signs in the material storage area. The laser radar monitors the spatial environment around the robot at all times, accurately detects the position and distance of obstacles, and prevents the robot from colliding during movement. The force sensor is installed at the end of the robot's operation. During the operation of grasping and assembly, it accurately senses the force to avoid damage to parts or affect assembly accuracy due to improper force. The raw data formats generated by these sensors are diverse and complex, and cannot be directly used effectively by the system. Therefore, through the written sensor driver, these raw data are collected and preprocessed. The preprocessing stage includes format conversion, converting the specific format data collected by different sensors, such as image data of visual cameras and distance data of laser radars, into a standard data format; data integration is performed at the same time, and the data from different sensors are merged according to time series and logical relationships to form a complete and orderly data set, providing a reliable data basis for subsequent decision analysis.

[0058] The distributed decision trigger module continuously monitors the data preprocessed by the environmental perception module. Once it finds that the data meets the preset decision trigger conditions, it will send a signal to the local decision generation module to start the decision process. The decision trigger conditions include material shortage, task timeout and production order change. In the process of automobile parts manufacturing, for the material shortage trigger condition, the system will set an inventory threshold S for each part based on the production plan and historical data. min The robot obtains the current inventory quantity of pistons through sensors and material management system. when When t is reached, the material shortage trigger condition is determined to be met. This means that insufficient inventory of key parts will cause the production line to stop working, so it is necessary to trigger the decision in time to adjust the production task or replenish materials. For the task timeout trigger condition, taking the engine cylinder processing task as an example, let the task start time be tkssj , the current time is t dqsj , the standard operation time of this task is T bzsj , if it is a simple task in a single stage, when t dqsj -t kssj ≥T bzsj When , the task is judged to be timed out and the task timeout trigger condition is met. If the engine cylinder processing task is divided into multiple stages, for the i-th task stage, set its start time to End time is Standard operating time is when When the task timeout is reached, it is determined that the task in this stage has timed out. This is because the task timeout will affect the entire production progress, and the robot's task arrangement needs to be adjusted in time to ensure the continuity of production. The production order change is also an important trigger condition. The production orders of automobile manufacturers are often adjusted according to market demand. The system will obtain order information in real time and set the old order information as O. old , the new order information is O new , when O new ≠O old That is, when key information such as product model, quantity, delivery time, etc. in the order changes, it is determined that the production order change trigger conditions are met. For example, if the model of the car required to be produced by the order changes, the type and quantity of parts required may change. At this time, the system needs to re-plan the robot's task.

[0059] Each industrial robot is embedded with an intelligent decision-making module, which consists of a knowledge base, a learning unit and a communication interface, and is responsible for generating local decisions. The knowledge base stores a large amount of knowledge and rules related to automobile parts manufacturing tasks, robot performance, and production processes. These knowledge include both pre-set production process specifications and experience knowledge accumulated in long-term production practice. Moreover, the knowledge base will be continuously updated and improved as the robot runs and learns. For example, when the robot finds a new assembly problem during the assembly process and finds an effective solution, the method will be recorded in the knowledge base; the learning unit optimizes decisions by learning historical decision data and real-time production data, analyzes the effects of different decisions under different production conditions, and automatically adjusts decision parameters. For example, by analyzing the impact of different decisions on production efficiency under material shortage conditions in the past, the learning unit can adjust the decision weights in similar situations to make the decision more reasonable; the communication interface is responsible for data interaction with other modules in the system. On the one hand, it receives real-time environmental data from the environmental perception module, such as the current position of the robot, surrounding obstacle information, etc., as well as the trigger signal of the distributed decision trigger module; on the other hand, it passes the generated local decision plan to the collaborative strategy optimization module. When the intelligent decision module receives the signal from the distributed decision trigger module, it enters the data reception and integration stage. For example, when it receives the trigger signal of material shortage, it obtains the real-time data about the robot's own status from the environmental perception module, including the robot's current position, task execution progress, remaining power, load status, and material location. Then it integrates the trigger signal with these real-time data to form a complete decision data set D = {x 1 ,x 2 ,x 3 ,…,x n}, where x i Represents the task characteristics in the decision data set. Next, knowledge base matching and preliminary decision-making are carried out. According to the received decision task type (such as material shortage), the knowledge base is searched for matching rules and knowledge. For example, there is a rule in the knowledge base that "when materials are short and the robot is close to the material storage point, it is preferred to go to replenish materials". According to this rule, the integrated decision data set is preliminarily analyzed and judged to generate a preliminary decision direction, such as determining whether the robot needs to suspend the current task to replenish materials, and then the decision tree algorithm is used to optimize the preliminary decision. The decision tree algorithm makes decisions based on a tree structure. By using the decision data set D, the decision data set is continuously divided, and the division and decision rules are adjusted according to the optimal decision results of historical data. The preliminary decision is optimized to a decision that conforms to historical experience and current conditions. Each node of the decision tree represents a test of the task characteristics. For node N j , whose test function is f j(D) = {True, False}. For example, for node N 1 , the test function can be to determine whether the remaining power of the robot is greater than a certain threshold. If f 1 (D) = True, data enters D 1,True , otherwise enter D 1,False ; Each leaf node L k Represents a decision result, and its decision result function is expressed as g k (D) = {y 1 ,y 2 ,y 3}, where y 1 Indicates whether to accept the new task, y 2 Indicates whether to adjust the task execution order, y 3 Indicates whether to change the operation mode. The decision tree is trained based on historical decision data. For each historical sample D s ={x s1 ,x s2 ,x s3 ,…,x sn} and its corresponding optimal decision result y s ={y s1 ,y s2 ,y s3}, by minimizing the error function To optimize the decision tree, where m represents the number of samples of historical decision data, g k (D s ) is the decision tree for historical samples D s The decision result given, y s is the actual optimal decision result of the sample, by continuously adjusting the node test function f of the decision tree j And the decision result function g of the leaf node k , to optimize the preliminary decision, the optimization process is: select a task feature from the decision data set D as the test function f of the root node 1 , divide the data set into D 1,True and D 1,False , for each self, continue to select task features as the test function of the next layer of nodes, and so on recursively until the samples in the subset belong to the same type of decision results, that is, they all have the same y s , or the number of samples in the subset reaches the predetermined recursive termination condition, at each leaf node, the number of various decision results of the samples in the subset is counted, and the decision result with the largest number of occurrences is selected as the g of the leaf node kvalue, and use it as the final decision result of the node. Finally, the preliminary decision matched by the comprehensive knowledge base and the result after decision optimization are generated to generate the final local decision plan, which includes specific decision contents such as whether to accept new tasks, adjust the execution order of tasks, change the operation mode, etc. For example, the decision plan may be that the robot suspends the current assembly task, goes to the material storage point to replenish the shortage of parts, and adjusts the execution order of subsequent assembly tasks after returning. The intelligent decision module sends the local decision plan to the collaborative strategy optimization module through the communication interface, and coordinates the optimization with the decisions of other robots.

[0060] The collaborative strategy optimization module builds a high-speed communication network between the robots in the automotive parts manufacturing factory to ensure that the local decision-making plan of each robot can be quickly and accurately transmitted to the collaborative strategy optimization module. The module first collects local decision-making plans. Each industrial robot sends the local decision-making plan generated by its own local decision-making generation module through the communication interface. These plans include the preliminary decisions made by each robot based on its own state and the received trigger signal. Then, the state space is constructed. The collaborative strategy optimization module integrates the local decision-making plans of all industrial robots and the current production environment information to construct the state space S required by the deep reinforcement learning algorithm. 1 ,s 2 ,…,s n}, the elements in the state space include the position, task status, remaining power, load status of each industrial robot, as well as the overall progress of the production task, material distribution and other information. For example, s 1 Can represent the current position coordinates of a robot, s 2 Indicates the progress percentage of the task being executed, s 3 Indicates the remaining quantity of a key material on the production line, etc., and then defines the action space A = {a 1 ,a 2 ,…,a m}, the action space covers the adjustment of task allocation to each robot, the re-planning of task execution order, the change of operation parameters, etc. For example, action a j It can be expressed as industrial robot R k Task T l Reassigned to industrial robots R m , or adjust the operating speed, strength and other parameters of a robot, and set the reward function R(S,A) according to the task completion index to measure the effect of taking action a j After the state s i The reward value r obtained by transferring to a new state, the reward function Among them, α, β, and γ are the weight coefficients of the reward function, which are used to adjust the relative importance of each factor in the reward function. t is the task completion time, p is the task accuracy, and e is the energy consumption. In the manufacturing of automotive parts, if the robot can complete the assembly task with high accuracy in a short time and with low energy consumption, it will get a higher reward value; on the contrary, if the task is completed in timeout, the accuracy is not up to standard, or the energy consumption is too high, the reward value will be low or even a penalty value. The deep reinforcement learning algorithm is used for iterative optimization. In each iteration, based on the current state s i Select an action a from the action space A j Execute and observe that the system state transfers to the new state s i+1 , and obtain the corresponding reward value r, by continuously adjusting the policy network parameters θ, so that the long-term cumulative reward is maximized, that is Here, π θ (a|s) represents the probability distribution of selecting action a according to parameter θ in state s, T is the total time of task execution, γ is the discount factor, and its value range is [0, 1], which is used to measure the importance of future rewards. The closer γ is to 1, the more attention is paid to future rewards; the closer γ is to 0, the more attention is paid to current rewards. t and a t They represent the state and action taken at time t respectively. To update the policy network parameters θ, the gradient ascent method is used. The policy network parameter update formula is: Among them, θ k+1 and θ k are the policy network parameters before and after the update, respectively. η is the learning rate, which controls the step size of each parameter update. A learning rate that is too large may cause the algorithm to be unstable, while a learning rate that is too small will slow down the convergence speed. is the logarithmic probability gradient of the policy network, r is the reward value obtained after executing action a, and the specific optimization steps are:

[0061] 1. Initialization: Initialize the policy network π θ The parameter θ of (a|s) sets the discount factor γ and the learning rate η. At the same time, the state space S, the action space A and the reward function R(S,A) are initialized.

[0062] 2. Strategy selection and execution: At each time t, according to the current state s t , through the policy network π θ (a|s) from the action space A with probability π θ (a t |s t )Select action a t ;

[0063] 3. State transfer and reward acquisition: Execute action a t After that, the system status changes from s tTransfer to new state s t+1 At the same time, the reward value r is calculated according to the reward function R(S,A). According to the time, accuracy and energy consumption indicators of task completion under the new state, the reward value r is calculated by substituting it into the reward function t ;

[0064] 4. Parameter update: Using the policy gradient theorem, according to the reward value r obtained by executing the action t and the log probability gradient of the policy network Update the policy network parameters θ;

[0065] 5. Iterative optimization: Repeat steps 2-4 above for multiple iterations until the algorithm converges and finally obtains the optimal strategy network that maximizes the long-term cumulative reward. That is, the global optimal coordination strategy. This strategy determines the optimal actions that each industrial robot should take under different conditions to achieve efficient collaborative operation between robots. For example, in the case of material shortage, the global optimal coordination strategy may be to arrange the robot that is closest to the material storage point and has sufficient power to replenish the materials, while adjusting the task sequence of other robots to ensure the overall operating efficiency of the production line.

[0066] The execution and feedback module writes a task execution program for the robot according to the global optimal collaborative strategy generated by the collaborative strategy optimization module. In the process of manufacturing automobile parts, the robot controls its own actions according to this program. For example, when assembling the engine, the robot accurately grabs the parts according to the strategy and performs the assembly operation in the prescribed order and force. In the process of the robot executing the task, the built-in status monitoring program collects the execution status data in real time. These data include whether the actual action of the robot conforms to the predetermined strategy, the progress of task completion, the motion parameters of each joint, the operating status of the equipment, etc. These execution status data are fed back to the environment perception module and the collaborative strategy optimization module through the communication network. The data fed back to the environment perception module will be used to update the environmental information so that the system can more accurately perceive the changes in the production environment. For example, if the robot finds that the position of the parts deviates from the preset position during the assembly process, after this information is fed back to the environment perception module, the environment perception module can adjust its cognition of the production environment in time and pass the new environmental information to other modules; the data fed back to the collaborative strategy optimization module is used to evaluate the execution effect of the current strategy. If it is found that the actual execution does not meet the expectations, the collaborative strategy optimization module can optimize the strategy again based on these feedback data, regenerate a collaborative strategy that is more in line with the actual situation, and continuously improve the collaborative operation performance of the system.

[0067] In this embodiment, the environmental perception module of the artificial intelligence-based industrial robot collaborative system accurately obtains production environment information through a variety of sensors and performs preprocessing, providing a reliable data basis for subsequent decision-making. The distributed decision trigger module promptly detects abnormal situations in the production process according to preset conditions, such as material shortages, task timeouts, and production order changes, and starts the decision-making process. The local decision generation module uses the knowledge base, learning unit, and decision tree algorithm of the intelligent decision-making module to generate reasonable local decision-making solutions. The collaborative strategy optimization module uses a deep reinforcement learning algorithm to integrate the local decisions of each robot to generate a global optimal collaborative strategy, thereby achieving efficient collaboration between robots. The execution and feedback module ensures the accurate execution of the strategy, and continuously optimizes the strategy through a feedback mechanism, so that the system can adapt to various changes in the production process.

[0068] In summary, through the close collaboration of various modules, the system effectively improves the production efficiency of automobile parts manufacturing, reduces production stagnation caused by material shortages, task timeouts and other problems, optimizes resource allocation, reduces production costs, and at the same time improves product quality and production flexibility. The system of the present invention provides a practical solution for the intelligent upgrading of industrial production in terms of decision-making efficiency, collaborative optimization capabilities and adaptability to complex production environments.

[0069] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. An industrial robot collaborative system based on artificial intelligence, characterized in that: The components of the system include: environmental perception module, distributed decision triggering module, local decision generation module, collaborative strategy optimization module and execution and feedback module; The environmental perception module installs a visual camera for each industrial robot to identify materials and environmental identification, a laser radar to detect obstacles and spatial distances, and a force sensor to sense the force during operation. The sensor data is collected through a sensor driver and the collected data is preprocessed, and the preprocessing includes format conversion and data integration; The distributed decision trigger module sets the decision trigger conditions and monitors the pre-processed data in real time through the decision trigger program. The decision trigger conditions include material shortage, task timeout and production order change. When the pre-processed data meets the trigger conditions, a decision trigger signal is sent to the local decision generation module of each robot. The local decision generation module assumes the local decision generation function for the intelligent decision module of each industrial robot, and generates a local decision plan according to the current task state of the robot and the received trigger signal. The local decision generation function uses a decision tree algorithm to determine whether to accept a new task and the preliminary task execution order; The collaborative strategy optimization module builds a high-speed communication network between the robots, adopts a deep reinforcement learning algorithm, takes the local decision-making plan of each robot as input, and uses the task completion index as a reward signal to perform training and optimization to generate a global optimal collaborative strategy. The specific generation process of the global optimal collaborative strategy is as follows: Local decision plan collection: Each industrial robot sends the local decision plan generated by its own local decision generation module to the collaborative strategy optimization module through the communication interface. The local decision plan includes the preliminary decision made by each robot based on its own state and the received trigger signal; State space construction: The collaborative strategy optimization module integrates the local decision-making plans of all industrial robots and the current production environment information to construct the state space required by the deep reinforcement learning algorithm. The elements in the state space include the position, task status, remaining power, load status of each industrial robot, as well as the overall progress of the production task and the information of material distribution. The state space S = {s1, s2, …, s n }, where s i Represents a specific state element in the state space; Action space definition: Define the set of actions taken by robots when working together, that is, the action space A = {a1, a2, …, a m }, the action space includes the adjustment of the task allocation of each robot, the re-planning of the task execution order, and the change of the operating parameters, that is, the action a in the action space A j Indicates that the industrial robot R k Task T l Reassigned to industrial robots R m ; Reward function setting: Set the reward function R(S,A) according to the task completion index to measure the effect of taking action a j After the state s i The reward value r obtained by transferring to a new state, the reward function Among them, α, β, γ are the weight coefficients of the reward function, t is the task completion time, p is the task accuracy, and e is the energy consumption; Iterative optimization of deep reinforcement learning: Iterative training and optimization are performed using deep reinforcement learning algorithms, so that the industrial robot updates its own strategy network parameters according to the feedback reward value after each action, and gradually learns the optimal collaborative strategy; Global optimal collaborative strategy generation: After multiple deep reinforcement learning iterative optimizations, when the algorithm converges, the obtained strategy network This is the generated global optimal collaborative strategy, which determines the optimal actions that each industrial robot should take under different states, thereby achieving efficient collaborative operation between robots; The execution and feedback module writes task execution programs for the robot and controls the robot's movements according to the collaborative strategy. During the robot's task execution, the execution status data is collected in real time through the built-in status monitoring program and fed back to the environment perception module and the collaborative strategy optimization module through the communication network.

2. The artificial intelligence-based industrial robot collaborative system according to claim 1, characterized in that: The distributed decision trigger module continuously monitors the data preprocessed by the environment perception module through the decision trigger program, compares the monitored data with the trigger condition, and determines that the trigger condition is met when the data reaches and exceeds the threshold of the trigger condition, thereby starting the sending operation of the decision trigger signal to notify the local decision generation module to make corresponding decision adjustments, wherein: For the material shortage trigger condition, the current inventory data is compared with the inventory threshold to determine whether the trigger threshold is reached, that is, For in time The current inventory quantity of the material at the time, S min is the inventory threshold of the material. , it is determined that the material shortage trigger condition is met; For the task timeout trigger condition, the difference between the current time and the task start time is compared with the standard operation time, that is, t kssj is the task start time, t dqsj is the current time, T bzsj is the standard operation time of the task. For a simple task in a single stage, when t dqsj -t kssj ≥T bzsj When , the task is judged to have timed out and the task timeout trigger condition is met; for complex tasks with multiple stages, is the start time of the i-th task phase, is the end time of the i-th task stage, is the standard operation time of the i-th task stage. When , it is determined that the task of this stage has timed out and the task timeout triggering condition is met; For the production order change trigger condition, by comparing the order information received twice, when the order information changes, that is, O old For old order information, new For new order information, when O is satisfied new ≠O old , it is determined that the production order change triggering condition is met.

3. The artificial intelligence-based industrial robot collaborative system according to claim 1, characterized in that: The local decision generation module assumes the local decision generation function of the intelligent decision module in the industrial robot. The intelligent decision module is an independent processing engine embedded in each industrial robot, which has the functions of data processing and analysis, logical judgment and decision output, and is composed of a knowledge base, a learning unit and a communication interface, wherein: The knowledge base stores knowledge and rules related to robot tasks, performance, and production processes. The knowledge includes both the set production process specifications and deterministic rules of robot operation restrictions. The knowledge base will be continuously updated and improved as the robot operates and learns; The learning unit optimizes the decision by learning the historical decision data and the real-time production data, and can automatically adjust the decision parameters; The communication interface is responsible for data interaction with other modules in the system. On the one hand, it receives real-time environmental data from the environmental perception module and the trigger signal of the distributed decision trigger module, and on the other hand, it passes the generated local decision plan to the collaborative strategy optimization module.

4. The artificial intelligence-based industrial robot collaborative system according to claim 1, characterized in that: The working principle of the local decision generation module is: Data reception and integration: The intelligent decision module receives the decision trigger signal from the distributed decision trigger module through the communication interface, and clarifies the task type for decision processing, which includes material shortage, task timeout and production order change. At the same time, it obtains real-time data about the robot's own status from the environmental perception module, including the robot's current position, task execution progress, remaining power, load status and material location, and integrates the trigger signal with the real-time data to form a complete decision data set; Preliminary decision-making based on knowledge base matching: Based on the received decision-making task type, the knowledge base is searched for matching rules and knowledge. Based on the matched rules, the integrated decision data set is preliminarily analyzed and judged to generate a preliminary decision direction. Decision optimization: extract key features from the current decision data set, combine historical decision data, and use decision tree algorithms to optimize preliminary decisions; Decision generation and output: The preliminary decision of the knowledge base matching and the results of decision optimization are integrated to generate the final local decision plan, which includes the specific decision content of whether to accept the new task, adjust the task execution order, and change the operation mode; Decision coordination and transmission: The intelligent decision-making module sends the local decision plan to the collaborative strategy optimization module through the communication interface, and coordinates and optimizes the decisions of other robots to jointly achieve efficient operation of the entire system.

5. The artificial intelligence-based industrial robot collaborative system according to claim 4, characterized in that: The decision tree algorithm is an artificial intelligence algorithm based on a tree structure for decision making, which uses the decision data set D = {x1, x2, x3, ..., x n }, where x i Represent the task characteristics in the decision data set, and optimize the initial decision to a decision that conforms to historical experience and current conditions by continuously dividing the decision data set and adjusting the division and decision rules according to the optimal decision results of historical data; Each node of the decision tree represents a test of the task feature. j , whose test function is f j (D) = {True, False}, the dataset is divided into two subsets according to the value of the task feature, namely D j,True and D j,False , for node N j , when f j (D) = True, data enters D j,True , otherwise enter D j,False ; Each leaf node L k Represents a decision result, and its decision result function is expressed as g k (D) = {y1, y2, y3}, where y1 indicates whether to accept a new task, y2 indicates whether to adjust the order of task execution, and y3 indicates whether to change the operation mode; Based on the historical decision data, the decision tree is trained. For each historical sample D s ={x s1 ,x s2 ,x s3 ,…,x sn } and its corresponding optimal decision result y s ={y s1 ,y s2 ,y s3 }, optimize the decision tree by minimizing the error function, the error function Among them, m represents the number of samples of historical decision data, g k (D s ) is the decision tree for historical samples D s The decision result given, y s is the actual optimal decision result of the sample. By minimizing the error function, the node test function f of the decision tree is continuously adjusted. j And the decision result function g of the leaf node k , in order to optimize the initial decision.

6. The artificial intelligence-based industrial robot collaborative system according to claim 5, characterized in that: The optimization process of the decision tree algorithm for preliminary decision is: Select a task feature from the decision data set D as the test function f1 of the root node, and divide the data set into 1,True and D 1,False , continue to select task features as the test function of the next layer of nodes, and so on recursively until all samples in the subset belong to the same type of decision results, that is, they all have the same y s and the number of samples in the subsets is done recursively; At each leaf node, count the number of various decision results of the samples in the subset, and select the decision result with the largest number of occurrences as the g of the leaf node. k value and use it as the final decision result of the node.

7. The artificial intelligence-based industrial robot collaborative system according to claim 1, characterized in that: The collaborative strategy optimization module uses the deep reinforcement learning algorithm to train the current state s i Select an action a from the action space A j Execute and observe that the system state transfers to the new state s i+1 , and obtain the corresponding reward value r, by continuously adjusting the policy network parameters, so that the long-term cumulative reward is maximized, that is The long-term cumulative reward is maximized according to the policy network π θ (a|s), which represents the probability distribution of selecting action a according to parameter θ in state s, and finding the optimal parameter θ * , so that the expected cumulative reward is maximized, where T is the total time of task execution, γ zk is the discount factor, s t and a t Respectively represent the state and action taken at time t; To update the policy network parameters θ, the gradient ascent method is used, and the policy network parameters are updated as Among them, θ k+1 and θ k are the policy network parameters before and after the update, η is the learning rate, is the log probability gradient of the policy network, and r is the reward value obtained after executing action a.

8. The artificial intelligence-based industrial robot collaborative system according to claim 7, characterized in that: The specific steps of optimizing the collaborative strategy optimization module using the deep reinforcement learning algorithm are as follows: Initialization: Initialize the policy network π θ The parameter θ of (a|s) sets the discount factor γ and the learning rate η. At the same time, the state space S, the action space A and the reward function R(S,A) are initialized. Strategy selection and execution: At each time t, according to the current state s t , through the policy network π θ (a|s) from the action space A with probability π θ (a t |s t )Select action a t ; State transfer and reward acquisition: Execute action a t After that, the system status changes from s t Transfer to new state s t+1 At the same time, the reward value r is calculated according to the reward function R(S,A). According to the time, accuracy and energy consumption indicators of task completion under the new state, the reward value r is calculated by substituting it into the reward function t ; Parameter update: Using the policy gradient theorem, according to the reward value r obtained by executing the action t and the log probability gradient of the policy network Update the policy network parameters θ; Iterative optimization: Repeat the steps of strategy selection and execution, state transfer and reward acquisition, and parameter update for multiple iterations until the algorithm converges, and finally obtain the optimal strategy network that can maximize the long-term cumulative reward That is the global optimal collaborative strategy.

Images