Task allocation and optimal scheduling method and system for multi-robot collaborative industrial assembly production line

Abstract

The invention provides a task allocation and optimal scheduling method and system for a multi-robot collaborative industrial assembly line, and relates to the technical field of robots, comprising the steps of constructing a task allocation strategy network through deep reinforcement learning, encoding workpiece types and attribute information into a state vector input network, a multi-objective optimization function including total assembly time minimization, load balance degree maximization and energy consumption minimization is designed, a near-end strategy optimization algorithm is adopted to train network parameters, a robot task mapping matrix is output as a distribution scheme, and the network parameters are dynamically optimized and updated in an online learning mode. The task allocation efficiency of the multi-robot collaborative assembly production line can be remarkably improved, the dynamic optimization scheduling of the production line is realized, the system energy consumption is reduced, and the assembly efficiency is improved.

Description

Technical Field

[0001] The present invention relates to robot technology, and in particular to a method and system for task allocation and optimal scheduling of a multi-robot collaborative industrial assembly production line. Background Art

[0002] With the rapid development of intelligent manufacturing, multi-robot collaborative assembly production lines have been widely used in industrial production. In the process of multi-robot collaborative assembly, reasonable allocation of tasks and optimal scheduling of robot workloads are of great significance to improving production efficiency. At present, the task allocation of multi-robot collaborative assembly production lines mainly adopts heuristic algorithms and mathematical programming methods, which require manual design of rules and constraints and have certain limitations.

[0003] The existing technology has the following shortcomings: First, traditional task allocation methods are difficult to adapt to the dynamically changing production environment and cannot quickly adjust the allocation strategy according to the real-time working conditions; second, existing methods often only consider a single indicator such as total assembly time when designing optimization objectives, and fail to take into account robot load balancing and energy consumption optimization at the same time; third, there is a lack of effective online learning mechanism, and it is impossible to dynamically optimize and continuously improve the task allocation strategy according to the actual execution effect.

[0004] The technical problem to be solved by the present invention is: how to construct a task allocation method for a multi-robot collaborative assembly production line based on deep reinforcement learning, which can adaptively learn the optimal allocation strategy, realize multi-objective collaborative optimization, and continuously optimize the task allocation strategy through online learning to improve the overall performance of the system. Summary of the invention

[0005] The embodiments of the present invention provide a method and system for task allocation and optimal scheduling of a multi-robot collaborative industrial assembly production line, which can solve the problems in the prior art.

[0006] According to a first aspect of the embodiments of the present invention,

[0007] Provides task allocation and optimization scheduling methods for multi-robot collaborative industrial assembly production lines, including:

[0008] A task allocation strategy network is constructed using a deep reinforcement learning algorithm, the input layer of the task allocation strategy network is set as a state encoding layer, the hidden layer is set as a multi-layer convolutional neural network and a fully connected layer, and the output layer is set as an action probability distribution layer; the workpiece information in the multi-robot collaborative assembly production line is preprocessed, the workpiece type is encoded by one-hot encoding to generate a type feature vector, the workpiece processing priority and time limit constraint are encoded by numerical encoding to generate an attribute feature vector, and the type feature vector and the attribute feature vector are combined to generate a state vector as the input of the task allocation strategy network;

[0009] A multi-objective optimization function is designed to train the task allocation strategy network, wherein the multi-objective optimization function includes the total assembly time minimization objective, the robot load balance maximization objective and the energy consumption minimization objective, and the weighted summation method is used to combine the three optimization objectives into a unified training objective; a robot task mapping matrix is ​​constructed as the output of the task allocation strategy network, wherein the elements of the robot task mapping matrix represent the probability values ​​of the robot performing the corresponding tasks, and the probability of an infeasible allocation scheme is set to zero according to the robot's working capacity constraints; the task allocation strategy network is trained using a proximal strategy optimization algorithm, and state-action pairs and reward values ​​are obtained by interacting with a simulation environment, and the parameters of the task allocation strategy network are updated;

[0010] The workpiece information in the multi-robot collaborative assembly production line is encoded as a state vector and input into the trained task allocation strategy network to output an optimal task allocation plan; the optimal task allocation plan is sent to the multi-robot collaborative assembly production line for execution; the task completion time, robot load distribution and energy consumption data are collected, the actual optimization target value is calculated, and the parameters of the task allocation strategy network are dynamically optimized and updated using an online learning method.

[0011] Preprocessing workpiece information in a multi-robot collaborative assembly production line, generating a type feature vector by one-hot encoding the workpiece type, generating an attribute feature vector by numerically encoding the workpiece processing priority and time limit constraint, and combining the type feature vector and the attribute feature vector to generate a state vector as the input of the task allocation strategy network includes:

[0012] Performing one-hot encoding preprocessing on the workpiece type information, constructing a binary type feature vector according to the total number of workpiece types in the production line, marking the corresponding type position of the workpiece as 1, and marking other type positions as 0, to obtain the type feature vector of the workpiece;

[0013] Performing numerical coding preprocessing on the workpiece processing priority information and the workpiece processing time limit information, mapping the workpiece processing priority information to a priority value in the interval of 0 to 1 according to the priority level of the workpiece in the production plan, calculating the remaining time according to the processing deadline of the workpiece and performing maximum and minimum value normalization processing to obtain a time limit value, and combining the priority value and the time limit value to generate an attribute feature vector;

[0014] The type feature vector and the attribute feature vector are combined and spliced ​​in a preset dimension order to generate a workpiece state vector, the mean and standard deviation of each dimension of the workpiece state vector are calculated, and a standardized state vector is obtained by performing a standardization process;

[0015] The standardized state vector is input into a task allocation strategy network, which is constructed based on a deep reinforcement learning algorithm, and the standardized state vector is used as an input of the task allocation strategy network.

[0016] A multi-objective optimization function is designed to train the task allocation strategy network. The multi-objective optimization function includes the goal of minimizing the total assembly time, maximizing the robot load balance, and minimizing the energy consumption. The weighted summation method is used to combine the three optimization goals into a unified training goal, including:

[0017] Constructing a multi-objective optimization function, wherein the multi-objective optimization function includes a total assembly time minimization objective, a robot load balance maximization objective, and an energy consumption minimization objective, and the multi-objective optimization function is used to train a task allocation strategy network;

[0018] Acquire real-time working condition data of a multi-robot collaborative assembly system, the real-time working condition data including workpiece processing parameters, robot motion parameters and tool operation parameters, calculate processing time based on the workpiece processing parameters, calculate motion time based on the robot motion parameters, calculate switching time based on the tool operation parameters, accumulate the processing time, motion time and switching time to obtain a total assembly time, and use the total assembly time as an optimization variable for the total assembly time minimization objective;

[0019] Calculate the task load distribution of each robot based on the real-time working condition data, determine the task workload according to the workpiece processing parameters, determine the task type according to the robot motion parameters, determine the execution time according to the tool operation parameters, take the product of the task workload, the task type and the execution time as the load value, calculate the discrete degree of the load value of each robot, and use the discrete degree as the optimization variable of the robot load balance maximization objective;

[0020] A system energy consumption model is established based on the real-time working condition data, joint drive energy consumption is calculated according to the robot motion parameters, tool execution energy consumption is calculated according to the tool operation parameters, and the joint drive energy consumption and the tool execution energy consumption are summed to obtain the total system energy consumption, and the total system energy consumption is used as the optimization variable of the energy consumption minimization objective;

[0021] The total assembly time, the discrete degree and the total energy consumption of the system are normalized to obtain standardized optimization variables, the weight coefficients of each optimization target are set according to production requirements, and the standardized optimization variables are combined into a unified training target using a weighted summation method.

[0022] A robot task mapping matrix is ​​constructed as the output of the task allocation strategy network, wherein the elements of the robot task mapping matrix represent the probability values ​​of the robot executing the corresponding tasks, and the probability of the infeasible allocation scheme is set to zero according to the robot working capacity constraint; the task allocation strategy network is trained using a proximal strategy optimization algorithm, and the state-action pairs and reward values ​​are obtained by interacting with the simulation environment, and the parameters of the task allocation strategy network are updated, including:

[0023] Constructing a robot task mapping matrix as an output of the task allocation strategy network, wherein the matrix elements in the robot task mapping matrix represent the probability values ​​of the robot performing the corresponding task;

[0024] Acquire robot working capacity constraints, wherein the working capacity constraints include robot working range parameters, robot load parameters, and robot end-effector configuration parameters, and identify infeasible task allocation solutions based on the working capacity constraints;

[0025] Generate a constraint processing matrix according to the infeasible task allocation scheme, wherein the constraint processing matrix has the same dimension as the robot task mapping matrix, and set the probability value corresponding to the infeasible task allocation scheme to zero through the constraint processing matrix;

[0026] Constructing a simulation interaction environment, wherein the simulation interaction environment is used to simulate a multi-robot collaborative task allocation scenario, and performing task allocation actions in the simulation interaction environment based on the probability distribution of the robot task mapping matrix;

[0027] The task allocation strategy network is trained by using a proximal strategy optimization algorithm, and state-action pairs and corresponding reward values ​​are obtained by interactive sampling with the simulation interaction environment;

[0028] A policy gradient is constructed based on the state-action pair and the reward value, a truncated objective function is used to limit the policy update step, an entropy regularization term is introduced to balance policy exploration and utilization; and parameters of the task allocation policy network are iteratively updated.

[0029] Encoding the sequence of workpieces to be processed in the multi-robot collaborative assembly production line as a state vector and inputting it into the trained task allocation strategy network to output an optimal task allocation plan; sending the optimal task allocation plan to the multi-robot collaborative assembly production line for execution includes:

[0030] The state vectors corresponding to all the workpieces to be processed in the sequence of workpieces to be processed are combined into a state matrix, and the state matrix is ​​padded to a fixed dimension using a filling mark; the state matrix is ​​input into a trained task allocation strategy network, and the task allocation strategy network outputs a workpiece-robot allocation probability matrix;

[0031] A temperature parameter is introduced to adjust the probability distribution of the allocation probability matrix, and an optimal task allocation scheme is generated based on the adjusted probability distribution using a greedy strategy, wherein the optimal task allocation scheme determines an allocation relationship between each of the workpieces to be processed and the robots in the multi-robot collaborative assembly production line;

[0032] According to the optimal task allocation scheme, each processing task of the workpiece to be processed is decomposed into a basic operation instruction sequence, and an execution time window is generated for each basic operation instruction based on process constraints and resource constraints;

[0033] According to the execution time window, a collision-free motion trajectory is planned for each robot, and the collision-free motion trajectory is converted into a corresponding robot control instruction; the robot control instruction is sent to the corresponding robot in the multi-robot collaborative assembly production line for execution;

[0034] monitoring the execution status of the robot control instructions in real time, and when it is detected that the execution deviation exceeds a preset threshold, updating the time constraint feature in the workpiece state vector, and re-inputting the updated state matrix into the task allocation strategy network to generate a new optimal task allocation plan;

[0035] A smooth transition process is performed on the newly generated optimal task allocation plan and the original task allocation plan, and the smoothed task allocation plan is converted into a new robot control instruction and sent to the multi-robot collaborative assembly production line for execution.

[0036] Collecting task completion time, robot load distribution and energy consumption data, calculating the actual optimization target value, and dynamically optimizing and updating the parameters of the task allocation strategy network using online learning includes:

[0037] Collecting task execution data of each robot in a multi-robot collaborative assembly production line, wherein the task execution data includes a process start timestamp, a process end timestamp, a task queue length, a resource occupancy rate, a joint motor current value, and a joint motor voltage value;

[0038] Calculate the actual task completion time based on the process start timestamp and the process end timestamp, calculate the robot load distribution based on the task queue length and the resource occupancy rate, and calculate the robot energy consumption based on the joint motor current value and the joint motor voltage value;

[0039] The actual task completion time, the robot load distribution and the robot energy consumption are used as time dimension indicators, load dimension indicators and energy consumption dimension indicators respectively, and the three dimension indicators are combined into a comprehensive optimization target value by using an adaptive weighting method;

[0040] Constructing an experience replay buffer pool, storing workpiece status information, task allocation decision and the comprehensive optimization target value as training samples in the experience replay buffer pool, and updating the experience replay buffer pool in a first-in-first-out manner when the number of samples in the experience replay buffer pool exceeds a preset threshold;

[0041] A training batch is randomly sampled from the experience replay buffer pool, the workpiece state information is input into the task allocation strategy network to obtain a predicted task allocation plan, and the strategy loss between the predicted task allocation plan and the actually executed task allocation decision is calculated; based on the strategy loss and the comprehensive optimization target value, the parameters of the task allocation strategy network are updated using the stochastic gradient descent method, and a momentum factor is introduced for soft updating.

[0042] According to a second aspect of the embodiments of the present invention,

[0043] Provides multi-robot collaborative industrial assembly production line task allocation and optimization scheduling system, including:

[0044] The first unit is used to construct a task allocation strategy network using a deep reinforcement learning algorithm, set the input layer of the task allocation strategy network to a state encoding layer, set the hidden layer to a multi-layer convolutional neural network and a fully connected layer, and set the output layer to an action probability distribution layer; pre-process the workpiece information in the multi-robot collaborative assembly production line, generate a type feature vector by using one-hot encoding for the workpiece type, generate an attribute feature vector by using numerical encoding for the workpiece processing priority and time limit constraint, and combine the type feature vector and the attribute feature vector to generate a state vector as the input of the task allocation strategy network;

[0045] The second unit is used to design a multi-objective optimization function to train the task allocation strategy network, wherein the multi-objective optimization function includes a total assembly time minimization objective, a robot load balancing maximization objective, and an energy consumption minimization objective, and a weighted summation method is used to combine the three optimization objectives into a unified training objective; a robot task mapping matrix is ​​constructed as the output of the task allocation strategy network, wherein the elements of the robot task mapping matrix represent the probability values ​​of the robot performing the corresponding tasks, and the probability of an infeasible allocation scheme is set to zero according to the robot's working capacity constraints; the task allocation strategy network is trained using a proximal strategy optimization algorithm, and state-action pairs and reward values ​​are obtained by interacting with a simulation environment, and the parameters of the task allocation strategy network are updated;

[0046] The third unit is used to encode the workpiece information in the multi-robot collaborative assembly production line into a state vector and input it into the trained task allocation strategy network to output an optimal task allocation plan; send the optimal task allocation plan to the multi-robot collaborative assembly production line for execution; collect task completion time, robot load distribution and energy consumption data, calculate the actual optimization target value, and dynamically optimize and update the parameters of the task allocation strategy network using online learning.

[0047] According to a third aspect of the embodiments of the present invention,

[0048] An electronic device is provided, comprising:

[0049] processor;

[0050] a memory for storing processor-executable instructions;

[0051] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0052] A fourth aspect of the embodiments of the present invention is:

[0053] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0054] The beneficial effects of this application are as follows:

[0055] The present invention adopts deep reinforcement learning to construct a task allocation strategy network, processes workpiece information through one-hot encoding and numerical encoding, effectively integrates workpiece type characteristics and attribute characteristics into a state vector as the input of the task allocation strategy network, can adaptively learn the optimal allocation strategy, avoids the limitations of manually designed rules, and improves the intelligence level and environmental adaptability of task allocation.

[0056] The present invention designs a multi-objective optimization function that includes minimizing the total assembly time, maximizing the load balance and minimizing the energy consumption. It uses the weighted summation method to unify multiple optimization objectives into a training objective, and outputs the task allocation plan through the robot task mapping matrix. It realizes the coordinated optimization of multiple performance indicators such as assembly efficiency, load balance and energy consumption, and significantly improves the comprehensive operation efficiency of the production line.

[0057] The present invention adopts an online learning method, collects task completion time, robot load and energy consumption data in real time, calculates the actual optimization target value and dynamically updates the parameters of the task allocation strategy network, thereby realizing continuous optimization and self-improvement of the task allocation strategy, ensuring the optimal performance of the system during actual operation, and improving the dynamic adaptability and robustness of the production line. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 A flowchart of a method for task allocation and optimal scheduling of a multi-robot collaborative industrial assembly production line according to an embodiment of the present invention;

[0059] Figure 2 It is a structural schematic diagram of a multi-robot collaborative industrial assembly production line task allocation and optimization scheduling system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0062] Figure 1 FIG. 1 is a flow chart of a method for task allocation and optimal scheduling of a multi-robot collaborative industrial assembly production line according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0063] S11. Use a deep reinforcement learning algorithm to construct a task allocation strategy network, set the input layer of the task allocation strategy network to a state encoding layer, set the hidden layer to a multi-layer convolutional neural network and a fully connected layer, and set the output layer to an action probability distribution layer; pre-process the workpiece information in the multi-robot collaborative assembly production line, generate a type feature vector by using one-hot encoding for the workpiece type, generate an attribute feature vector by using numerical encoding for the workpiece processing priority and time limit constraint, and combine the type feature vector and the attribute feature vector to generate a state vector as the input of the task allocation strategy network;

[0064] S12. Design a multi-objective optimization function to train the task allocation strategy network, wherein the multi-objective optimization function includes the total assembly time minimization objective, the robot load balance maximization objective and the energy consumption minimization objective, and the weighted summation method is used to combine the three optimization objectives into a unified training objective; construct a robot task mapping matrix as the output of the task allocation strategy network, wherein the elements of the robot task mapping matrix represent the probability values ​​of the robot performing the corresponding tasks, and the probability of an infeasible allocation scheme is set to zero according to the robot's working capacity constraints; adopt a proximal strategy optimization algorithm to train the task allocation strategy network, obtain state-action pairs and reward values ​​by interacting with the simulation environment, and update the parameters of the task allocation strategy network;

[0065] S13. Encode the workpiece information in the multi-robot collaborative assembly production line as a state vector and input it into the trained task allocation strategy network to output the optimal task allocation plan; send the optimal task allocation plan to the multi-robot collaborative assembly production line for execution; collect task completion time, robot load distribution and energy consumption data, calculate the actual optimization target value, and use online learning to dynamically optimize and update the parameters of the task allocation strategy network.

[0066] In an optional implementation, preprocessing workpiece information in a multi-robot collaborative assembly production line, generating a type feature vector by one-hot encoding the workpiece type, generating an attribute feature vector by numerically encoding the workpiece processing priority and time limit constraint, and combining the type feature vector and the attribute feature vector to generate a state vector as the input of the task allocation strategy network includes:

[0067] Performing one-hot encoding preprocessing on the workpiece type information, constructing a binary type feature vector according to the total number of workpiece types in the production line, marking the corresponding type position of the workpiece as 1, and marking other type positions as 0, to obtain the type feature vector of the workpiece;

[0068] Performing numerical coding preprocessing on the workpiece processing priority information and the workpiece processing time limit information, mapping the workpiece processing priority information to a priority value in the interval of 0 to 1 according to the priority level of the workpiece in the production plan, calculating the remaining time according to the processing deadline of the workpiece and performing maximum and minimum value normalization processing to obtain a time limit value, and combining the priority value and the time limit value to generate an attribute feature vector;

[0069] The type feature vector and the attribute feature vector are combined and spliced ​​in a preset dimension order to generate a workpiece state vector, the mean and standard deviation of each dimension of the workpiece state vector are calculated, and a standardized state vector is obtained by performing a standardization process;

[0070] The standardized state vector is input into a task allocation strategy network, which is constructed based on a deep reinforcement learning algorithm, and the standardized state vector is used as an input of the task allocation strategy network.

[0071] In order to optimize task allocation in a multi-robot collaborative assembly production line, a task allocation method based on workpiece information preprocessing and deep reinforcement learning is proposed. This method preprocesses the workpiece information and generates a state vector as the input of the reinforcement learning network, so that the network can learn the optimal allocation strategy more effectively.

[0072] First, obtain the information of each workpiece in the production line. This information includes the basic attributes and constraints of the workpiece. The basic attributes include the workpiece type and processing priority. The constraints include the workpiece processing time limit. For example, suppose there are three types of workpieces in the production line: A, B, and C. The information of a workpiece is: type A, high priority, and time limit is 48 hours.

[0073] Next, the workpiece type information is one-hot encoded. A binary type feature vector is constructed based on the total number of workpiece types in the production line. The positions corresponding to the workpiece type are marked as 1, and other types are marked as 0. Taking the above example, the type feature vector of the workpiece is [1,0,0].

[0074] Then, the processing priority and time limit information of the workpiece are numerically encoded. The workpiece processing priority information is mapped to a priority value in the range of 0 to 1. For example, the "high" priority is mapped to 0.8, the "medium" priority is mapped to 0.5, and the "low" priority is mapped to 0.2. The remaining time of the workpiece processing deadline is calculated and the maximum and minimum values ​​are normalized to obtain the time limit value. Assuming that the current time is 0, the time limit of the workpiece is 48 hours, and the longest time limit of all workpieces is 72 hours and the shortest time limit is 24 hours, then the time limit value of the workpiece is (48-24) / (72-24)=0.5. The priority value and the time limit value are combined to generate an attribute feature vector. Taking the above example, the attribute feature vector of the workpiece is [0.8,0.5].

[0075] Combine the type feature vector and the attribute feature vector in the preset dimension order to generate the workpiece state vector. Taking the above example, combine the type feature vector [1,0,0] and the attribute feature vector [0.8,0.5] to obtain the workpiece state vector [1,0,0,0.8,0.5]. Calculate the mean and standard deviation of each dimension of all workpiece state vectors. Assuming that the first dimension mean of all workpiece state vectors is 0.5 and the standard deviation is 0.2, then the value of the first dimension of the workpiece state vector is (1-0.5) / 0.2=2.5, and so on. Standardize each dimension to obtain a standardized state vector.

[0076] Finally, the normalized state vector is fed into the task allocation policy network. This network is built based on a deep reinforcement learning algorithm, takes the normalized state vector as input, and outputs a task allocation plan for each robot.

[0077] The beneficial effects of this method can be summarized in the following three aspects:

[0078] Improve task allocation efficiency: By preprocessing the artifact information, the artifact type, priority, deadline and other information are converted into a structured state vector, which provides more effective input for the reinforcement learning network, thereby improving the network learning efficiency and task allocation efficiency.

[0079] Optimized resource utilization: This method is able to allocate tasks according to constraints such as workpiece priority and time limit, thereby optimizing the utilization of production resources and reducing the idle time and waiting time of robots.

[0080] Enhance system robustness: The policy network based on deep reinforcement learning can adapt to complex production environments and dynamically changing workpiece information, thereby enhancing the robustness and adaptability of the system.

[0081] In an optional implementation, a multi-objective optimization function is designed to train the task allocation strategy network, wherein the multi-objective optimization function includes a total assembly time minimization objective, a robot load balance maximization objective, and an energy consumption minimization objective, and a weighted summation method is used to combine the three optimization objectives into a unified training objective, including:

[0082] Constructing a multi-objective optimization function, wherein the multi-objective optimization function includes a total assembly time minimization objective, a robot load balance maximization objective, and an energy consumption minimization objective, and the multi-objective optimization function is used to train a task allocation strategy network;

[0083] Acquire real-time working condition data of a multi-robot collaborative assembly system, the real-time working condition data including workpiece processing parameters, robot motion parameters and tool operation parameters, calculate processing time based on the workpiece processing parameters, calculate motion time based on the robot motion parameters, calculate switching time based on the tool operation parameters, accumulate the processing time, motion time and switching time to obtain a total assembly time, and use the total assembly time as an optimization variable for the total assembly time minimization objective;

[0084] Calculate the task load distribution of each robot based on the real-time working condition data, determine the task workload according to the workpiece processing parameters, determine the task type according to the robot motion parameters, determine the execution time according to the tool operation parameters, take the product of the task workload, the task type and the execution time as the load value, calculate the discrete degree of the load value of each robot, and use the discrete degree as the optimization variable of the robot load balance maximization objective;

[0085] A system energy consumption model is established based on the real-time working condition data, joint drive energy consumption is calculated according to the robot motion parameters, tool execution energy consumption is calculated according to the tool operation parameters, and the joint drive energy consumption and the tool execution energy consumption are summed to obtain the total system energy consumption, and the total system energy consumption is used as the optimization variable of the energy consumption minimization objective;

[0086] The total assembly time, the discrete degree and the total energy consumption of the system are normalized to obtain standardized optimization variables, the weight coefficients of each optimization target are set according to production requirements, and the standardized optimization variables are combined into a unified training target using a weighted summation method.

[0087] The task allocation strategy network training method for multi-robot collaborative assembly system aims to optimize the three objectives of total assembly time, robot load balance and energy consumption.

[0088] First, a multi-objective optimization function is constructed. This function contains three sub-objectives: minimizing the total assembly time, maximizing the robot load balance, and minimizing energy consumption. These three sub-objectives together constitute a multi-objective optimization function, which is used to train the task allocation strategy network so that the network can learn the optimal task allocation strategy.

[0089] Next, the real-time working condition data of the multi-robot collaborative assembly system is obtained. These data include workpiece processing parameters (such as workpiece size, material, processing technology, etc.), robot motion parameters (such as robot position, speed, acceleration, etc.) and tool operation parameters (such as tool type, operation force, operation time, etc.).

[0090] Then, the total assembly time is calculated based on the real-time working condition data. The processing time of each workpiece is calculated according to the workpiece processing parameters, the movement time of the robot from one station to another is calculated according to the robot motion parameters, and the switching time of the robot switching different tools is calculated according to the tool operation parameters. The processing time of all workpieces, the movement time of all robots, and the switching time of all tools are accumulated to obtain the total assembly time. For example, suppose there are two workpieces, the processing time is 10 seconds and 15 seconds respectively; there are two robots, the movement time is 5 seconds and 7 seconds respectively; and the tool switching time is 2 seconds. Then the total assembly time is 10+15+5+7+2=39 seconds.

[0091] Subsequently, the robot load balance is calculated based on the real-time working condition data. First, the workload of each task is determined according to the workpiece processing parameters, for example, the cutting amount required to process the workpiece is used as the workload. Then, the task type is determined according to the robot motion parameters. For example, moving the robot to a specified position and performing a specific operation is defined as a task type. The execution time of each task is then determined according to the tool operation parameters. Multiply the task workload, task type, and execution time to obtain the load value of each robot. For example, assuming that the task workload of robot 1 is 10, the task type is 1, and the execution time is 5 seconds, the load value is 10*1*5=50. Finally, the degree of dispersion of all robot load values ​​is calculated. For example, the standard deviation can be used to measure the degree of dispersion. The smaller the degree of dispersion, the higher the degree of load balance of the robot.

[0092] Next, a system energy consumption model is established based on real-time working condition data. The joint drive energy consumption of each robot is calculated according to the robot motion parameters, for example, the energy consumption is calculated according to the rotation angle and speed of the robot joint. The execution energy consumption of each tool is calculated according to the tool operation parameters, for example, the energy consumption is calculated according to the power and usage time of the tool. The joint drive energy consumption of all robots and the execution energy consumption of all tools are added together to obtain the total energy consumption of the system. For example, assuming that the joint drive energy consumption of two robots is 100 watt-hours and 120 watt-hours respectively, and the tool execution energy consumption is 50 watt-hours, the total energy consumption of the system is 100+120+50=270 watt-hours.

[0093] Finally, the total assembly time, degree of discreteness, and total system energy consumption are normalized. For example, the minimum-maximum normalization method can be used to scale these values ​​to between 0 and 1. Then, the weight coefficient of each optimization objective is set according to production requirements. For example, the weights can be set to 0.5, 0.3, and 0.2 respectively according to the importance of total assembly time, load balancing, and energy consumption. Finally, the weighted summation method is used to combine the standardized optimization variables into a unified training objective for training the task allocation strategy network. For example, assuming that the normalized total assembly time is 0.8, the degree of discreteness is 0.2, and the total system energy consumption is 0.5, the unified training objective is 0.5*0.8+0.3*(1-0.2)+0.2*0.5=0.74.

[0094] Beneficial effects:

[0095] 1. Shorten the total assembly time: By optimizing the task allocation strategy, the total time for workpiece processing, robot movement and tool switching can be effectively reduced, thereby improving production efficiency.

[0096] 2. Improve robot load balance: By evenly distributing tasks, it is possible to avoid situations where some robots are overworked while others are idle, thereby improving equipment utilization and extending equipment life.

[0097] 3. Reduce system energy consumption: By optimizing the robot motion trajectory and tool operation parameters, the energy consumption of joint drive and tool execution can be reduced, thereby reducing production costs.

[0098] In an optional implementation, a robot task mapping matrix is ​​constructed as the output of the task allocation strategy network, the elements of the robot task mapping matrix represent the probability values ​​of the robot executing the corresponding tasks, and the probability of an infeasible allocation scheme is set to zero according to the robot's working capacity constraints; the task allocation strategy network is trained using a proximal strategy optimization algorithm, and state-action pairs and reward values ​​are obtained by interacting with a simulation environment, and the parameters of the task allocation strategy network are updated, including:

[0099] Constructing a robot task mapping matrix as an output of the task allocation strategy network, wherein the matrix elements in the robot task mapping matrix represent the probability values ​​of the robot performing the corresponding task;

[0100] Acquire robot working capacity constraints, wherein the working capacity constraints include robot working range parameters, robot load parameters, and robot end-effector configuration parameters, and identify infeasible task allocation solutions based on the working capacity constraints;

[0101] Generate a constraint processing matrix according to the infeasible task allocation scheme, wherein the constraint processing matrix has the same dimension as the robot task mapping matrix, and set the probability value corresponding to the infeasible task allocation scheme to zero through the constraint processing matrix;

[0102] Constructing a simulation interaction environment, wherein the simulation interaction environment is used to simulate a multi-robot collaborative task allocation scenario, and performing task allocation actions in the simulation interaction environment based on the probability distribution of the robot task mapping matrix;

[0103] The task allocation strategy network is trained by using a proximal strategy optimization algorithm, and state-action pairs and corresponding reward values ​​are obtained by interactive sampling with the simulation interaction environment;

[0104] A policy gradient is constructed based on the state-action pair and the reward value, a truncated objective function is used to limit the policy update step, an entropy regularization term is introduced to balance policy exploration and utilization; and parameters of the task allocation policy network are iteratively updated.

[0105] In order to achieve efficient allocation of tasks for multi-robot collaboration, a task allocation method based on reinforcement learning is proposed. This method constructs a task allocation strategy network, whose output is the robot task mapping matrix, and the matrix elements represent the probability of the robot performing the corresponding task. By training the network in a simulation environment, it can learn the optimal task allocation strategy.

[0106] First, a task allocation strategy network is constructed. The input of the network is the current environment state, such as the position and load status of each robot, and the attributes and position of each task. The output of the network is a robot task mapping matrix. Assuming there are 3 robots and 4 tasks, the output matrix dimension is 3x4. Each element in the matrix represents the probability of the corresponding robot performing the corresponding task. For example, the value of the matrix element (1,2) is 0.8, which means that the probability of robot 1 performing task 2 is 0.8.

[0107] Next, obtain the robot's working capacity constraints. These constraints include the robot's working range parameters, robot load parameters, and robot end-effector configuration parameters. For example, the working range of Robot 1 is a circular area with a radius of 5 meters, its maximum load is 10 kilograms, and it is equipped with a gripper-type end-effector. Task 1 is outside the working range of Robot 1, Task 2 requires carrying a 15-kilogram weight, and Task 3 requires a suction cup-type end-effector. Based on these constraints, it can be identified that Robot 1 cannot perform Task 1 and Task 2. Robot 1 is also limited in performing Task 3, and it is necessary to confirm whether its gripper can effectively grasp the object corresponding to Task 3.

[0108] Then, a constraint processing matrix is ​​generated based on the infeasible task allocation scheme. This matrix has the same dimension as the robot task mapping matrix. For infeasible task allocation schemes, the elements at the corresponding positions in the constraint processing matrix are set to 0, and the other positions are set to 1. For example, since robot 1 cannot perform tasks 1 and 2, the elements at positions (1,1) and (1,2) are set to 0 in the 3x4 constraint processing matrix. There are restrictions on robot 1 performing task 3, and it is necessary to determine whether it is feasible based on the actual situation. If it is determined to be infeasible, the element at position (1,3) is also set to 0.

[0109] Next, a simulation interaction environment is constructed. This environment simulates the multi-robot collaborative task allocation scenario, including virtual robots, tasks, and work areas. Based on the probability distribution of the robot task mapping matrix, the task allocation action is performed in the simulation environment. For example, according to the robot task mapping matrix, robot 1 performs task 2 with a probability of 0.8 and performs task 4 with a probability of 0.2. In the simulation environment, random sampling is performed based on these probabilities to determine which task robot 1 will eventually perform. If the sampling result is to perform task 2, but because task 2 exceeds the load capacity of robot 1, the task allocation fails.

[0110] After that, the proximal policy optimization algorithm is used to train the task allocation policy network. By interactive sampling with the simulation environment, the state-action pairs and corresponding reward values ​​are obtained. For example, if the robot successfully completes the task, it will receive a positive reward; if the task allocation fails or the task execution time is too long, it will receive a negative reward.

[0111] Finally, the policy gradient is constructed based on the state-action pair and the reward value, a truncated objective function is used to limit the policy update step size, and an entropy regularization term is introduced to balance policy exploration and utilization, and the parameters of the task allocation policy network are iteratively updated. For example, in the initial state, the probability of robot 1 performing each task is evenly distributed. After multiple rounds of training, the network learns that robot 1 has a higher probability of performing task 4 because it is more suitable for performing task 4.

[0112] For example, in a specific scenario, there are 2 robots and 3 tasks. Robot 1 has a maximum load of 5kg and a working range radius of 3m; Robot 2 has a maximum load of 10kg and a working range radius of 5m. Task 1 weighs 2kg and is located at (1,1); Task 2 weighs 8kg and is located at (4,4); Task 3 weighs 3kg and is located at (2,3). After training, the robot task mapping matrix output by the policy network may be as follows:

[0113] [[0.9,0.05,0.05],[0.05,0.05,0.9]].

[0114] This shows that robot 1 prefers to perform tasks 1 and 3, while robot 2 prefers to perform task 3.

[0115] Beneficial Effects

[0116] 1. Improve task allocation efficiency: The policy network obtained through reinforcement learning training can quickly generate efficient task allocation solutions, avoiding the complex calculations and manual intervention in traditional methods, thereby shortening the task completion time.

[0117] 2. Enhance system robustness: This method can adapt to dynamically changing environments, such as robot failures or changes in task requirements. Through interactive learning with the simulation environment, the policy network can automatically adjust the task allocation strategy to ensure the smooth completion of the task.

[0118] 3. Optimize resource utilization: This method can allocate tasks according to the robot's capacity constraints, avoiding resource waste, such as assigning tasks that exceed the robot's load capacity to the robot. This allows the robots to better play their respective advantages, thereby improving overall efficiency.

[0119] In an optional implementation, encoding the sequence of workpieces to be processed in the multi-robot collaborative assembly production line as a state vector and inputting it into the trained task allocation strategy network to output an optimal task allocation plan; sending the optimal task allocation plan to the multi-robot collaborative assembly production line for execution includes:

[0120] The state vectors corresponding to all the workpieces to be processed in the sequence of workpieces to be processed are combined into a state matrix, and the state matrix is ​​padded to a fixed dimension using a filling mark; the state matrix is ​​input into a trained task allocation strategy network, and the task allocation strategy network outputs a workpiece-robot allocation probability matrix;

[0121] A temperature parameter is introduced to adjust the probability distribution of the allocation probability matrix, and an optimal task allocation scheme is generated based on the adjusted probability distribution using a greedy strategy, wherein the optimal task allocation scheme determines an allocation relationship between each of the workpieces to be processed and the robots in the multi-robot collaborative assembly production line;

[0122] According to the optimal task allocation scheme, each processing task of the workpiece to be processed is decomposed into a basic operation instruction sequence, and an execution time window is generated for each basic operation instruction based on process constraints and resource constraints;

[0123] According to the execution time window, a collision-free motion trajectory is planned for each robot, and the collision-free motion trajectory is converted into a corresponding robot control instruction; the robot control instruction is sent to the corresponding robot in the multi-robot collaborative assembly production line for execution;

[0124] monitoring the execution status of the robot control instructions in real time, and when it is detected that the execution deviation exceeds a preset threshold, updating the time constraint feature in the workpiece state vector, and re-inputting the updated state matrix into the task allocation strategy network to generate a new optimal task allocation plan;

[0125] A smooth transition process is performed on the newly generated optimal task allocation plan and the original task allocation plan, and the smoothed task allocation plan is converted into a new robot control instruction and sent to the multi-robot collaborative assembly production line for execution.

[0126] The task allocation method for multi-robot collaborative assembly production line optimizes the task allocation of multiple robots in the production line and improves production efficiency by encoding the workpiece status and combining it with a deep learning task allocation strategy network.

[0127] First, the sequence information of the workpieces to be processed in the multi-robot collaborative assembly production line is obtained. For example, there are currently 5 workpieces to be processed in the production line, which are recorded as workpiece A, workpiece B, workpiece C, workpiece D and workpiece E respectively.

[0128] Next, the workpiece state encoding module is constructed, which includes a feature extraction unit and a position encoding unit. The feature extraction unit is responsible for collecting the physical features, process requirement features, and time constraint features of each workpiece to be processed. Taking workpiece A as an example, its physical features include geometric dimensions (length 10cm, width 5cm, height 2cm), weight (1kg), and material (metal); process requirement features include processing accuracy (0.1mm), surface quality requirements (Ra0.8), number of processes (3) and process constraints (process 1 must be completed before process 2); time constraint features include start processing time (8:00 on November 1, 2023), completion time (12:00 on November 1, 2023) and delivery time (8:00 on November 2, 2023). These features are spliced ​​into a feature vector. Similar feature extraction is performed on all workpieces to form their own feature vectors.

[0129] The position encoding unit uses a sine function to encode the position information of the workpiece in the sequence. For example, workpiece A is the first in the sequence, workpiece B is the second, and so on. Through position encoding, a position encoding vector corresponding to each workpiece is generated.

[0130] The feature vector is combined with the position encoding vector to form a workpiece state vector, which is used to characterize the complete state information of the workpiece to be processed. For example, the state vector of workpiece A is composed of its feature vector and position encoding vector. The state vectors of all workpieces form a state matrix. If the number of workpieces to be processed is less than the preset value (for example, 10), the state matrix is ​​padded with fill marks to fix its dimension to 10xN, where N is the dimension of the state vector.

[0131] Input the state matrix to the pre-trained task allocation strategy network. The network structure can be a multi-layer fully connected neural network or other suitable deep learning network structure. The input of the network is the state matrix, and the output is the workpiece-robot allocation probability matrix. For example, the network outputs a 5x3 matrix, which represents the probability of 5 workpieces being assigned to 3 robots.

[0132] In order to avoid falling into the local optimal solution, the temperature parameter is introduced to adjust the probability distribution of the allocation probability matrix. The higher the temperature parameter, the more uniform the probability distribution; the lower the temperature parameter, the more concentrated the probability distribution. Based on the adjusted probability distribution, the greedy strategy is used to generate the optimal task allocation solution. For example, the allocation relationship with the highest probability is determined as the final allocation result, thereby determining which robot each workpiece is assigned to.

[0133] According to the optimal task allocation scheme, the processing task of each workpiece is decomposed into a sequence of basic operation instructions. For example, the processing task of workpiece A is decomposed into a sequence of instructions such as "grab", "move to station 1", "processing operation 1", "move to station 2", "processing operation 2", "move to station 3", "processing operation 3", and "place". And based on the process constraints and resource constraints, an execution time window is generated for each basic operation instruction. For example, the execution time window of "processing operation 1" is 9:00-10:00.

[0134] According to the execution time window, a collision-free motion trajectory is planned for each robot. For example, robot 1 needs to move to the initial position of workpiece A to grab it, then move to station 1 for processing, then move to station 2 for processing, and so on. The collision-free motion trajectory is converted into the corresponding robot control instructions.

[0135] The robot control instructions are sent to the corresponding robots in the multi-robot collaborative assembly production line for execution.

[0136] The execution status of the robot control instructions is monitored in real time. When the execution deviation is detected to exceed the preset threshold, such as the delay in the processing time of a workpiece, the time constraint feature in the workpiece state vector is updated, and the updated state matrix is ​​re-input into the task allocation strategy network to generate a new optimal task allocation plan.

[0137] The newly generated optimal task allocation plan and the original task allocation plan are smoothly transitioned to avoid drastic changes in robot movements. The smoothed task allocation plan is converted into a new robot control instruction and sent to the production line for execution.

[0138] Beneficial effects:

[0139] 1. Improve production efficiency: By optimizing the task allocation scheme, the robot's idle time and waiting time are reduced, thereby improving the overall efficiency of the production line.

[0140] 2. Improve product quality: Accurate task allocation and time control ensure the processing accuracy and quality of each workpiece, thereby improving the overall quality of the product.

[0141] 3. Enhance system flexibility: The task allocation plan can be adjusted according to the real-time status, so that the system can adapt to changes in the production environment, enhancing the flexibility and robustness of the system.

[0142] In an optional implementation, collecting task completion time, robot load distribution and energy consumption data, calculating the actual optimization target value, and dynamically optimizing and updating the parameters of the task allocation strategy network using online learning includes:

[0143] Collecting task execution data of each robot in a multi-robot collaborative assembly production line, wherein the task execution data includes a process start timestamp, a process end timestamp, a task queue length, a resource occupancy rate, a joint motor current value, and a joint motor voltage value;

[0144] Calculate the actual task completion time based on the process start timestamp and the process end timestamp, calculate the robot load distribution based on the task queue length and the resource occupancy rate, and calculate the robot energy consumption based on the joint motor current value and the joint motor voltage value;

[0145] The actual task completion time, the robot load distribution and the robot energy consumption are used as time dimension indicators, load dimension indicators and energy consumption dimension indicators respectively, and the three dimension indicators are combined into a comprehensive optimization target value by using an adaptive weighting method;

[0146] Constructing an experience replay buffer pool, storing workpiece status information, task allocation decision and the comprehensive optimization target value as training samples in the experience replay buffer pool, and updating the experience replay buffer pool in a first-in-first-out manner when the number of samples in the experience replay buffer pool exceeds a preset threshold;

[0147] A training batch is randomly sampled from the experience replay buffer pool, the workpiece state information is input into the task allocation strategy network to obtain a predicted task allocation plan, and the strategy loss between the predicted task allocation plan and the actually executed task allocation decision is calculated; based on the strategy loss and the comprehensive optimization target value, the parameters of the task allocation strategy network are updated using the stochastic gradient descent method, and a momentum factor is introduced for soft updating.

[0148] The online optimization method of task allocation strategy for multi-robot collaborative assembly production line can dynamically adjust the task allocation strategy according to actual production conditions, improve production efficiency and reduce energy consumption.

[0149] First, a multi-robot collaborative assembly production line experimental platform was built, which includes several robots with independent control systems, conveyor belts for transferring workpieces, and sensors for monitoring the production process. For example, the platform includes three six-degree-of-freedom robotic arms, the conveyor belt speed is adjustable, and the sensors can collect data such as robot joint position, current, and voltage.

[0150] Next, the task execution data of each robot on the production line is collected. Specifically, the start timestamp and end timestamp of each process are recorded. For example, the timestamp when robot 1 starts to execute process A is 10:00:00, and the end timestamp is 10:00:15. At the same time, the length of the task queue is recorded. For example, the current length of the task queue of robot 2 is 3, which means that there are 3 tasks waiting to be executed. In addition, the resource occupancy rate is recorded. For example, the end effector of robot 3 is currently busy and the resource occupancy rate is 100%. Finally, the current value and voltage value of each robot joint motor are recorded. For example, the current value of the first joint motor of robot 1 is 2A and the voltage value is 24V.

[0151] Then, the actual task completion time, robot load distribution, and robot energy consumption are calculated based on the collected data. The actual task completion time can be obtained by subtracting the process start timestamp from the process end timestamp. For example, the task completion time of robot 1 performing process A is 15 seconds. The robot load distribution can be measured by the task queue length and resource occupancy rate. For example, the task queue length of robot 2 is 3 and the resource occupancy rate is 50%, indicating a moderate load. The robot energy consumption can be calculated by the joint motor current and voltage values. For example, the power of the first joint motor of robot 1 is 48W.

[0152] Afterwards, the calculated actual task completion time, robot load distribution, and robot energy consumption are used as time dimension indicators, load dimension indicators, and energy consumption dimension indicators, respectively, and the three dimension indicators are combined into a comprehensive optimization target value using an adaptive weighting method. For example, if the time dimension weight is set to 0.5, the load dimension weight is set to 0.3, and the energy consumption dimension weight is set to 0.2, then the comprehensive optimization target value is 0.5*task completion time+0.3*load distribution+0.2*energy consumption. Assuming that the task completion time is 15 seconds, the load distribution is 0.75, and the energy consumption is 50W, the comprehensive optimization target value is 0.5*15+0.3*0.75+0.2*50=18.225.

[0153] Construct an experience replay buffer pool, and store the workpiece state information, task allocation decision and comprehensive optimization target value into training samples in the buffer pool. For example, the workpiece state information may include the workpiece position, size, etc., the task allocation decision may be expressed as which workpiece is assigned to which robot, and the comprehensive optimization target value is 18.225. When the number of samples in the experience replay buffer pool exceeds the preset threshold, such as 1000 samples, the buffer pool is updated in a first-in-first-out manner.

[0154] Randomly sample training batches from the experience replay buffer pool, for example, sample 64 samples. Input the workpiece state information into the task allocation policy network to obtain the predicted task allocation plan, for example, assign workpiece 1 to robot 2. Calculate the policy loss between the predicted task allocation plan and the actual task allocation decision. For example, if the actual task allocation decision is to assign workpiece 1 to robot 1, the policy loss is large. Based on the policy loss and the comprehensive optimization target value, the stochastic gradient descent method is used to update the parameters of the task allocation policy network, and a momentum factor is introduced for soft update. For example, the learning rate is set to 0.001 and the momentum factor is set to 0.9.

[0155] Beneficial effects:

[0156] 1. Improve production efficiency: By dynamically optimizing the task allocation strategy, the task completion time can be reduced, thereby improving the overall production efficiency of the production line.

[0157] 2. Balancing robot loads: This method can make the loads of various robots more balanced, avoiding the situation where some robots are overly busy while others are idle, thereby improving equipment utilization and extending equipment life.

[0158] 3. Reduce energy consumption: By optimizing the task allocation strategy, the energy consumption of the robot can be reduced, thereby saving energy and reducing production costs.

[0159] Figure 2 FIG. 1 is a schematic diagram of a multi-robot collaborative industrial assembly production line task allocation and optimization scheduling system according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0160] The first unit is used to construct a task allocation strategy network using a deep reinforcement learning algorithm, set the input layer of the task allocation strategy network to a state encoding layer, set the hidden layer to a multi-layer convolutional neural network and a fully connected layer, and set the output layer to an action probability distribution layer; pre-process the workpiece information in the multi-robot collaborative assembly production line, generate a type feature vector by using one-hot encoding for the workpiece type, generate an attribute feature vector by using numerical encoding for the workpiece processing priority and time limit constraint, and combine the type feature vector and the attribute feature vector to generate a state vector as the input of the task allocation strategy network;

[0161] The second unit is used to design a multi-objective optimization function to train the task allocation strategy network, wherein the multi-objective optimization function includes a total assembly time minimization objective, a robot load balancing maximization objective, and an energy consumption minimization objective, and a weighted summation method is used to combine the three optimization objectives into a unified training objective; a robot task mapping matrix is ​​constructed as the output of the task allocation strategy network, wherein the elements of the robot task mapping matrix represent the probability values ​​of the robot performing the corresponding tasks, and the probability of an infeasible allocation scheme is set to zero according to the robot's working capacity constraints; the task allocation strategy network is trained using a proximal strategy optimization algorithm, and state-action pairs and reward values ​​are obtained by interacting with a simulation environment, and the parameters of the task allocation strategy network are updated;

[0162] The third unit is used to encode the workpiece information in the multi-robot collaborative assembly production line into a state vector and input it into the trained task allocation strategy network to output an optimal task allocation plan; send the optimal task allocation plan to the multi-robot collaborative assembly production line for execution; collect task completion time, robot load distribution and energy consumption data, calculate the actual optimization target value, and dynamically optimize and update the parameters of the task allocation strategy network using online learning.

[0163] According to a third aspect of the embodiments of the present invention,

[0164] An electronic device is provided, comprising:

[0165] processor;

[0166] a memory for storing processor-executable instructions;

[0167] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0168] A fourth aspect of the embodiments of the present invention is:

[0169] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0170] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0171] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for task allocation and optimization scheduling of a multi-robot collaborative industrial assembly production line, characterized in that: include: A task allocation strategy network is constructed using a deep reinforcement learning algorithm, the input layer of the task allocation strategy network is set as a state encoding layer, the hidden layer is set as a multi-layer convolutional neural network and a fully connected layer, and the output layer is set as an action probability distribution layer; the workpiece information in the multi-robot collaborative assembly production line is preprocessed, the workpiece type is encoded by one-hot encoding to generate a type feature vector, the workpiece processing priority and time limit constraint are encoded by numerical encoding to generate an attribute feature vector, and the type feature vector and the attribute feature vector are combined to generate a state vector as the input of the task allocation strategy network; A multi-objective optimization function is designed to train the task allocation strategy network, wherein the multi-objective optimization function includes the total assembly time minimization objective, the robot load balance maximization objective and the energy consumption minimization objective, and the weighted summation method is used to combine the three optimization objectives into a unified training objective; a robot task mapping matrix is ​​constructed as the output of the task allocation strategy network, wherein the elements of the robot task mapping matrix represent the probability values ​​of the robot performing the corresponding tasks, and the probability of an infeasible allocation scheme is set to zero according to the robot's working capacity constraints; the task allocation strategy network is trained using a proximal strategy optimization algorithm, and state-action pairs and reward values ​​are obtained by interacting with a simulation environment, and the parameters of the task allocation strategy network are updated; Encoding the workpiece information in the multi-robot collaborative assembly production line as a state vector and inputting it into the trained task allocation strategy network to output an optimal task allocation plan; sending the optimal task allocation plan to the multi-robot collaborative assembly production line for execution; The task completion time, robot load distribution and energy consumption data are collected, the actual optimization target value is calculated, and the parameters of the task allocation strategy network are dynamically optimized and updated using an online learning method.

2. The method according to claim 1, characterized in that Preprocessing workpiece information in a multi-robot collaborative assembly production line, generating a type feature vector by one-hot encoding the workpiece type, generating an attribute feature vector by numerically encoding the workpiece processing priority and time limit constraint, and combining the type feature vector and the attribute feature vector to generate a state vector as the input of the task allocation strategy network includes: Performing one-hot encoding preprocessing on the workpiece type information, constructing a binary type feature vector according to the total number of workpiece types in the production line, marking the corresponding type position of the workpiece as 1, and marking other type positions as 0, to obtain the type feature vector of the workpiece; Performing numerical coding preprocessing on the workpiece processing priority information and the workpiece processing time limit information, mapping the workpiece processing priority information to a priority value in the interval of 0 to 1 according to the priority level of the workpiece in the production plan, calculating the remaining time according to the processing deadline of the workpiece and performing maximum and minimum value normalization processing to obtain a time limit value, and combining the priority value and the time limit value to generate an attribute feature vector; The type feature vector and the attribute feature vector are combined and spliced ​​in a preset dimension order to generate a workpiece state vector, the mean and standard deviation of each dimension of the workpiece state vector are calculated, and a standardized state vector is obtained by performing a standardization process; The standardized state vector is input into a task allocation strategy network, which is constructed based on a deep reinforcement learning algorithm, and the standardized state vector is used as an input of the task allocation strategy network.

3. The method according to claim 1, characterized in that A multi-objective optimization function is designed to train the task allocation strategy network. The multi-objective optimization function includes the goal of minimizing the total assembly time, maximizing the robot load balance, and minimizing the energy consumption. The weighted summation method is used to combine the three optimization goals into a unified training goal, including: Constructing a multi-objective optimization function, wherein the multi-objective optimization function includes a total assembly time minimization objective, a robot load balance maximization objective, and an energy consumption minimization objective, and the multi-objective optimization function is used to train a task allocation strategy network; Acquire real-time working condition data of a multi-robot collaborative assembly system, the real-time working condition data including workpiece processing parameters, robot motion parameters and tool operation parameters, calculate processing time based on the workpiece processing parameters, calculate motion time based on the robot motion parameters, calculate switching time based on the tool operation parameters, accumulate the processing time, motion time and switching time to obtain a total assembly time, and use the total assembly time as an optimization variable for the total assembly time minimization objective; Calculate the task load distribution of each robot based on the real-time working condition data, determine the task workload according to the workpiece processing parameters, determine the task type according to the robot motion parameters, determine the execution time according to the tool operation parameters, take the product of the task workload, the task type and the execution time as the load value, calculate the discrete degree of the load value of each robot, and use the discrete degree as the optimization variable of the robot load balance maximization objective; A system energy consumption model is established based on the real-time working condition data, joint drive energy consumption is calculated according to the robot motion parameters, tool execution energy consumption is calculated according to the tool operation parameters, and the joint drive energy consumption and the tool execution energy consumption are summed to obtain the total system energy consumption, and the total system energy consumption is used as the optimization variable of the energy consumption minimization objective; The total assembly time, the discrete degree and the total energy consumption of the system are normalized to obtain standardized optimization variables, the weight coefficients of each optimization target are set according to production requirements, and the standardized optimization variables are combined into a unified training target using a weighted summation method.

4. The method according to claim 1, characterized in that: A robot task mapping matrix is ​​constructed as the output of the task allocation strategy network, wherein the elements of the robot task mapping matrix represent the probability values ​​of the robot executing the corresponding tasks, and the probability of the infeasible allocation scheme is set to zero according to the robot working capacity constraint; the task allocation strategy network is trained using a proximal strategy optimization algorithm, and the state-action pairs and reward values ​​are obtained by interacting with the simulation environment, and the parameters of the task allocation strategy network are updated, including: Constructing a robot task mapping matrix as an output of the task allocation strategy network, wherein the matrix elements in the robot task mapping matrix represent the probability values ​​of the robot performing the corresponding task; Acquire robot working capacity constraints, wherein the working capacity constraints include robot working range parameters, robot load parameters, and robot end-effector configuration parameters, and identify infeasible task allocation solutions based on the working capacity constraints; Generate a constraint processing matrix according to the infeasible task allocation scheme, wherein the constraint processing matrix has the same dimension as the robot task mapping matrix, and set the probability value corresponding to the infeasible task allocation scheme to zero through the constraint processing matrix; Constructing a simulation interaction environment, wherein the simulation interaction environment is used to simulate a multi-robot collaborative task allocation scenario, and performing task allocation actions in the simulation interaction environment based on the probability distribution of the robot task mapping matrix; The task allocation strategy network is trained by using a proximal strategy optimization algorithm, and state-action pairs and corresponding reward values ​​are obtained by interactive sampling with the simulation interaction environment; A policy gradient is constructed based on the state-action pair and the reward value, a truncated objective function is used to limit the policy update step, an entropy regularization term is introduced to balance policy exploration and utilization; and parameters of the task allocation policy network are iteratively updated.

5. The method according to claim 1, characterized in that: Encoding the workpiece information in the multi-robot collaborative assembly production line into a state vector and inputting it into the trained task allocation strategy network to output an optimal task allocation solution; Sending the optimal task allocation plan to the multi-robot collaborative assembly production line for execution includes: The state vectors corresponding to all the workpieces to be processed in the sequence of workpieces to be processed are combined into a state matrix, and the state matrix is ​​padded to a fixed dimension using a filling mark; the state matrix is ​​input into a trained task allocation strategy network, and the task allocation strategy network outputs a workpiece-robot allocation probability matrix; A temperature parameter is introduced to adjust the probability distribution of the allocation probability matrix, and an optimal task allocation scheme is generated based on the adjusted probability distribution using a greedy strategy, wherein the optimal task allocation scheme determines the allocation relationship between each of the workpieces to be processed and the robots in the multi-robot collaborative assembly production line; According to the optimal task allocation scheme, each processing task of the workpiece to be processed is decomposed into a basic operation instruction sequence, and an execution time window is generated for each basic operation instruction based on process constraints and resource constraints; According to the execution time window, a collision-free motion trajectory is planned for each robot, and the collision-free motion trajectory is converted into a corresponding robot control instruction; the robot control instruction is sent to the corresponding robot in the multi-robot collaborative assembly production line for execution; monitoring the execution status of the robot control instructions in real time, and when it is detected that the execution deviation exceeds a preset threshold, updating the time constraint feature in the workpiece state vector, and re-inputting the updated state matrix into the task allocation strategy network to generate a new optimal task allocation plan; A smooth transition process is performed on the newly generated optimal task allocation plan and the original task allocation plan, and the smoothed task allocation plan is converted into a new robot control instruction and sent to the multi-robot collaborative assembly production line for execution.

6. The method according to claim 1, characterized in that Collecting task completion time, robot load distribution and energy consumption data, calculating the actual optimization target value, and dynamically optimizing and updating the parameters of the task allocation strategy network using online learning includes: Collecting task execution data of each robot in a multi-robot collaborative assembly production line, wherein the task execution data includes a process start timestamp, a process end timestamp, a task queue length, a resource occupancy rate, a joint motor current value, and a joint motor voltage value; Calculate the actual task completion time based on the process start timestamp and the process end timestamp, calculate the robot load distribution based on the task queue length and the resource occupancy rate, and calculate the robot energy consumption based on the joint motor current value and the joint motor voltage value; The actual task completion time, the robot load distribution and the robot energy consumption are used as time dimension indicators, load dimension indicators and energy consumption dimension indicators respectively, and the three dimension indicators are combined into a comprehensive optimization target value by using an adaptive weighting method; Constructing an experience replay buffer pool, storing workpiece status information, task allocation decision and the comprehensive optimization target value as training samples in the experience replay buffer pool, and updating the experience replay buffer pool in a first-in-first-out manner when the number of samples in the experience replay buffer pool exceeds a preset threshold; A training batch is randomly sampled from the experience replay buffer pool, the workpiece state information is input into the task allocation strategy network to obtain a predicted task allocation plan, and the strategy loss between the predicted task allocation plan and the actually executed task allocation decision is calculated; based on the strategy loss and the comprehensive optimization target value, the parameters of the task allocation strategy network are updated using the stochastic gradient descent method, and a momentum factor is introduced for soft updating.

7. A multi-robot collaborative industrial assembly production line task allocation and optimization scheduling system, used to implement the method described in any one of claims 1 to 6, characterized in that: include: The first unit is used to construct a task allocation strategy network using a deep reinforcement learning algorithm, set the input layer of the task allocation strategy network to a state encoding layer, set the hidden layer to a multi-layer convolutional neural network and a fully connected layer, and set the output layer to an action probability distribution layer; pre-process the workpiece information in the multi-robot collaborative assembly production line, generate a type feature vector by using one-hot encoding for the workpiece type, generate an attribute feature vector by using numerical encoding for the workpiece processing priority and time limit constraint, and combine the type feature vector and the attribute feature vector to generate a state vector as the input of the task allocation strategy network; The second unit is used to design a multi-objective optimization function to train the task allocation strategy network, wherein the multi-objective optimization function includes a total assembly time minimization objective, a robot load balancing maximization objective, and an energy consumption minimization objective, and a weighted summation method is used to combine the three optimization objectives into a unified training objective; a robot task mapping matrix is ​​constructed as the output of the task allocation strategy network, wherein the elements of the robot task mapping matrix represent the probability values ​​of the robot performing the corresponding tasks, and the probability of an infeasible allocation scheme is set to zero according to the robot's working capacity constraints; the task allocation strategy network is trained using a proximal strategy optimization algorithm, and state-action pairs and reward values ​​are obtained by interacting with a simulation environment, and the parameters of the task allocation strategy network are updated; The third unit is used to encode the workpiece information in the multi-robot collaborative assembly production line into a state vector and input it into the trained task allocation strategy network to output an optimal task allocation plan; and send the optimal task allocation plan to the multi-robot collaborative assembly production line for execution; The task completion time, robot load distribution and energy consumption data are collected, the actual optimization target value is calculated, and the parameters of the task allocation strategy network are dynamically optimized and updated using an online learning method.

8. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images