Task allocation and optimization scheduling method and system for industrial assembly production lines with multi-robot collaboration

Abstract

This invention provides a method and system for task allocation and optimized scheduling of multi-robot collaborative industrial assembly lines, relating to the field of robotics. The method includes constructing a task allocation strategy network through deep reinforcement learning, encoding workpiece type and attribute information as state vectors input to the network, designing a multi-objective optimization function encompassing minimizing total assembly time, maximizing load balancing, and minimizing energy consumption, training network parameters using a proximal policy optimization algorithm, outputting a robot task mapping matrix as the allocation scheme, and dynamically optimizing and updating the network parameters through online learning. This invention can significantly improve the task allocation efficiency of multi-robot collaborative assembly lines, achieve dynamic optimized scheduling of the production line, reduce system energy consumption, and improve assembly efficiency.

Description

Technical Field

[0001] This invention relates to robotics technology, and more particularly to a method and system for task allocation and optimized scheduling of multi-robot collaborative industrial assembly lines. Background Technology

[0002] With the rapid development of intelligent manufacturing, multi-robot collaborative assembly lines are widely used in industrial production. In the process of multi-robot collaborative assembly, the rational allocation of tasks and the optimization of robot workload scheduling are crucial for improving production efficiency. Currently, task allocation in multi-robot collaborative assembly lines mainly employs heuristic algorithms and mathematical programming methods. These methods require manual design of rules and constraints, and therefore have certain limitations.

[0003] The existing technology has the following shortcomings: First, traditional task allocation methods are difficult to adapt to dynamically changing production environments and cannot quickly adjust allocation strategies according to real-time working conditions; second, existing methods often only consider a single indicator, such as total assembly time, when designing optimization goals, and fail to simultaneously take into account robot load balancing and energy consumption optimization; third, there is a lack of effective online learning mechanisms, which makes it impossible to dynamically optimize and continuously improve task allocation strategies based on actual execution results.

[0004] The technical problem to be solved by this invention is: how to construct a task allocation method for a multi-robot collaborative assembly production line based on deep reinforcement learning. This method can adaptively learn the optimal allocation strategy, achieve multi-objective collaborative optimization, and continuously optimize the task allocation strategy through online learning to improve the overall performance of the system. Summary of the Invention

[0005] This invention provides a method and system for task allocation and optimized scheduling of multi-robot collaborative industrial assembly production lines, which can solve the problems in the prior art.

[0006] A first aspect of the present invention,

[0007] This provides a method for task allocation and optimized scheduling in industrial assembly lines with multi-robot collaboration, including:

[0008] A task allocation strategy network is constructed using a deep reinforcement learning algorithm. The input layer of the task allocation strategy network is set as a state encoding layer, the hidden layer is set as a multi-layer convolutional neural network and a fully connected layer, and the output layer is set as an action probability distribution layer. The workpiece information in the multi-robot collaborative assembly line is preprocessed. The workpiece type is generated into a type feature vector through one-hot encoding, and the workpiece processing priority and time limit constraints are generated into an attribute feature vector through numerical encoding. The type feature vector and the attribute feature vector are combined to generate a state vector as the input of the task allocation strategy network.

[0009] A multi-objective optimization function is designed to train the task allocation strategy network. This function includes objectives for minimizing total assembly time, maximizing robot load balancing, and minimizing energy consumption. A weighted summation method is used to combine these three objectives into a unified training objective. A robot task mapping matrix is ​​constructed as the output of the task allocation strategy network. The elements of this matrix represent the probability values ​​of the robot executing the corresponding task, and the probabilities of infeasible allocation schemes are set to zero based on robot working capacity constraints. A proximal policy optimization algorithm is used to train the task allocation strategy network. State-action pairs and reward values ​​are obtained through interaction with the simulation environment, and the parameters of the task allocation strategy network are updated accordingly.

[0010] The workpiece information in the multi-robot collaborative assembly line is encoded as a state vector and input into the trained task allocation strategy network to output the optimal task allocation scheme. The optimal task allocation scheme is then sent to the multi-robot collaborative assembly line for execution. Data on task completion time, robot load distribution, and energy consumption are collected, the actual optimization target value is calculated, and the parameters of the task allocation strategy network are dynamically optimized and updated using an online learning method.

[0011] Preprocessing of workpiece information in a multi-robot collaborative assembly line involves generating a type feature vector from the workpiece type using one-hot encoding, generating an attribute feature vector from the workpiece processing priority and time constraints using numerical encoding, and combining the type feature vector and the attribute feature vector to generate a state vector as input to the task allocation strategy network.

[0012] The workpiece type information is preprocessed by one-hot encoding. A binary type feature vector is constructed based on the total number of workpiece types in the production line. The workpiece is marked as 1 at the corresponding type position and 0 at other type positions to obtain the workpiece type feature vector.

[0013] The workpiece processing priority information and the workpiece processing time limit information are preprocessed with numerical encoding. The workpiece processing priority information is mapped to a priority value in the range of 0 to 1 according to the priority level of the workpiece in the production plan. The remaining time is calculated according to the processing deadline of the workpiece and the maximum and minimum value normalization is performed to obtain the time limit value. The priority value and the time limit value are combined to generate an attribute feature vector.

[0014] The type feature vector and the attribute feature vector are combined and concatenated according to a preset dimension order to generate a workpiece state vector. The mean and standard deviation of each dimension of the workpiece state vector are calculated, and a standardized state vector is obtained by standardization.

[0015] The standardized state vector is input into the task allocation policy network, which is constructed based on a deep reinforcement learning algorithm, and the standardized state vector is used as the input to the task allocation policy network.

[0016] A multi-objective optimization function is designed to train the task allocation strategy network. This multi-objective optimization function includes three objectives: minimizing total assembly time, maximizing robot load balancing, and minimizing energy consumption. A weighted summation method is used to combine these three objectives into a unified training objective, which includes:

[0017] A multi-objective optimization function is constructed, which includes the objectives of minimizing total assembly time, maximizing robot load balance, and minimizing energy consumption. The multi-objective optimization function is used to train the task allocation strategy network.

[0018] Real-time operating data of a multi-robot collaborative assembly system is acquired. The real-time operating data includes workpiece processing parameters, robot motion parameters, and tool operation parameters. The processing time is calculated based on the workpiece processing parameters, the motion time is calculated based on the robot motion parameters, and the switching time is calculated based on the tool operation parameters. The processing time, the motion time, and the switching time are accumulated to obtain the total assembly time. The total assembly time is used as the optimization variable for the goal of minimizing the total assembly time.

[0019] The task load distribution of each robot is calculated based on the real-time working condition data. The task workload is determined according to the workpiece processing parameters. The task type is determined according to the robot motion parameters. The execution time is determined according to the tool operation parameters. The product of the task workload, the task type and the execution time is used as the load value. The dispersion of the load value of each robot is calculated. The dispersion is used as the optimization variable for maximizing the robot load balance.

[0020] A system energy consumption model is established based on the real-time operating data. The joint drive energy consumption is calculated based on the robot motion parameters, and the tool execution energy consumption is calculated based on the tool operation parameters. The total system energy consumption is obtained by summing the joint drive energy consumption and the tool execution energy consumption. The total system energy consumption is used as the optimization variable for the energy consumption minimization objective.

[0021] The total assembly time, the degree of dispersion, and the total energy consumption of the system are normalized to obtain standardized optimization variables. Weight coefficients for each optimization objective are set according to production requirements, and the standardized optimization variables are combined into a unified training objective using a weighted summation method.

[0022] A robot task mapping matrix is ​​constructed as the output of the task allocation policy network. The elements of the robot task mapping matrix represent the probability values ​​of the robot performing the corresponding task, and the probabilities of infeasible allocation schemes are set to zero based on robot work capacity constraints. The task allocation policy network is trained using a proximal policy optimization algorithm. By interacting with the simulation environment to obtain state-action pairs and reward values, the parameters of the task allocation policy network are updated, including:

[0023] A robot task mapping matrix is ​​constructed as the output of the task allocation strategy network, and the matrix elements in the robot task mapping matrix represent the probability values ​​of the robot executing the corresponding task;

[0024] Obtain robot working capacity constraints, including robot working range parameters, robot load parameters, and robot end effector configuration parameters; and identify infeasible task allocation schemes based on the working capacity constraints.

[0025] A constraint processing matrix is ​​generated based on the infeasible task allocation scheme. The constraint processing matrix has the same dimension as the robot task mapping matrix. The probability value corresponding to the infeasible task allocation scheme is set to zero through the constraint processing matrix.

[0026] A simulation interactive environment is constructed to simulate a multi-robot collaborative task allocation scenario. The task allocation action is performed in the simulation interactive environment based on the probability distribution of the robot task mapping matrix.

[0027] The task allocation policy network is trained using a proximal policy optimization algorithm, and state-action pairs and corresponding reward values ​​are obtained by interactive sampling with the simulation environment.

[0028] The policy gradient is constructed based on the state-action pair and the reward value. A truncated objective function is used to limit the policy update step size. An entropy regularization term is introduced to balance policy exploration and utilization. The parameters of the task allocation policy network are iteratively updated.

[0029] The sequence of workpieces to be processed in the multi-robot collaborative assembly line is encoded as a state vector and input into the trained task allocation strategy network to output the optimal task allocation scheme; the optimal task allocation scheme is then sent to the multi-robot collaborative assembly line for execution, including:

[0030] The state vectors corresponding to all workpieces in the workpiece sequence are used to form a state matrix, and the state matrix is ​​padded to a fixed dimension using padding markers. The state matrix is ​​then input into the trained task allocation strategy network, which outputs a workpiece-robot allocation probability matrix.

[0031] A temperature parameter is introduced to adjust the probability distribution of the allocation probability matrix. Based on the adjusted probability distribution, a greedy strategy is used to generate an optimal task allocation scheme. The optimal task allocation scheme determines the allocation relationship between each workpiece to be processed and the robots in the multi-robot collaborative assembly line.

[0032] According to the optimal task allocation scheme, the processing task of each workpiece to be processed is decomposed into a sequence of basic operation instructions, and an execution time window is generated for each basic operation instruction based on process constraints and resource constraints.

[0033] Based on the execution time window, a collision-free motion trajectory is planned for each robot, and the collision-free motion trajectory is converted into a corresponding robot control command; the robot control command is then sent to the corresponding robot in the multi-robot collaborative assembly line for execution.

[0034] The execution status of the robot control commands is monitored in real time. When the execution deviation is detected to exceed the preset threshold, the time constraint feature in the workpiece state vector is updated, and the updated state matrix is ​​re-input into the task allocation strategy network to generate a new optimal task allocation scheme.

[0035] The newly generated optimal task allocation scheme is smoothly transitioned to the original task allocation scheme. The smoothed task allocation scheme is then converted into new robot control commands and sent to the multi-robot collaborative assembly production line for execution.

[0036] Collecting task completion time, robot load distribution, and energy consumption data; calculating the actual optimization target value; and dynamically optimizing and updating the parameters of the task allocation strategy network using an online learning approach include:

[0037] Collect task execution data for each robot in a multi-robot collaborative assembly production line. The task execution data includes process start timestamp, process end timestamp, task queue length, resource utilization rate, joint motor current value, and joint motor voltage value.

[0038] The actual task completion time is calculated based on the process start timestamp and the process end timestamp; the robot load distribution is calculated based on the task queue length and the resource occupancy rate; and the robot energy consumption is calculated based on the joint motor current value and the joint motor voltage value.

[0039] The actual task completion time, the robot load distribution, and the robot energy consumption are respectively used as time dimension indicators, load dimension indicators, and energy consumption dimension indicators. An adaptive weighting method is used to combine the three dimension indicators into a comprehensive optimization target value.

[0040] An experience replay buffer is constructed, and the workpiece status information, task allocation decision and the comprehensive optimization target value are combined to form training samples and stored in the experience replay buffer. When the number of samples in the experience replay buffer exceeds a preset threshold, the experience replay buffer is updated in a first-in-first-out manner.

[0041] Training batches are randomly sampled from the experience replay buffer. The workpiece state information is input into the task allocation strategy network to obtain the predicted task allocation scheme. The strategy loss between the predicted task allocation scheme and the actual task allocation decision is calculated. Based on the strategy loss and the comprehensive optimization objective value, the parameters of the task allocation strategy network are updated using the stochastic gradient descent method, and a momentum factor is introduced for soft updates.

[0042] A second aspect of the present invention,

[0043] A task allocation and optimization scheduling system for multi-robot collaborative industrial assembly lines, including:

[0044] The first unit is used to construct a task allocation strategy network using a deep reinforcement learning algorithm. The input layer of the task allocation strategy network is set as a state encoding layer, the hidden layer is set as a multi-layer convolutional neural network and a fully connected layer, and the output layer is set as an action probability distribution layer. The workpiece information in the multi-robot collaborative assembly line is preprocessed. The workpiece type is generated into a type feature vector through one-hot encoding, and the workpiece processing priority and time limit constraints are generated into an attribute feature vector through numerical encoding. The type feature vector and the attribute feature vector are combined to generate a state vector as the input of the task allocation strategy network.

[0045] The second unit is used to design a multi-objective optimization function to train the task allocation strategy network. The multi-objective optimization function includes the objectives of minimizing total assembly time, maximizing robot load balancing, and minimizing energy consumption. The three optimization objectives are combined into a unified training objective using a weighted summation method. A robot task mapping matrix is ​​constructed as the output of the task allocation strategy network. The elements of the robot task mapping matrix represent the probability values ​​of the robot executing the corresponding task, and the probability of infeasible allocation schemes is set to zero according to the robot's working capacity constraints. The task allocation strategy network is trained using a proximal policy optimization algorithm. By interacting with the simulation environment to obtain state-action pairs and reward values, the parameters of the task allocation strategy network are updated.

[0046] The third unit is used to encode the workpiece information in the multi-robot collaborative assembly line into a state vector and input it into the trained task allocation strategy network to output the optimal task allocation scheme; to send the optimal task allocation scheme to the multi-robot collaborative assembly line for execution; to collect task completion time, robot load distribution and energy consumption data, calculate the actual optimization target value, and dynamically optimize and update the parameters of the task allocation strategy network using an online learning method.

[0047] A third aspect of the embodiments of the present invention,

[0048] An electronic device is provided, comprising:

[0049] processor;

[0050] Memory used to store processor-executable instructions;

[0051] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0052] Fourth aspect of the present invention,

[0053] A computer-readable storage medium is provided, having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0054] The beneficial effects of this application are as follows:

[0055] This invention employs deep reinforcement learning to construct a task allocation strategy network. By processing workpiece information through one-hot encoding and numerical encoding, it effectively integrates workpiece type features and attribute features into a state vector, which serves as the input to the task allocation strategy network. This enables the network to adaptively learn the optimal allocation strategy, avoiding the limitations of manually designed rules and improving the intelligence level and environmental adaptability of task allocation.

[0056] This invention designs a multi-objective optimization function that includes minimizing total assembly time, maximizing load balancing, and minimizing energy consumption. It uses a weighted summation method to unify multiple optimization objectives into a training objective and outputs a task allocation scheme through a robot task mapping matrix. This achieves coordinated optimization of multiple performance indicators such as assembly efficiency, load balancing, and energy consumption, significantly improving the overall operating efficiency of the production line.

[0057] This invention employs an online learning approach, which collects task completion time, robot load, and energy consumption data in real time, calculates the actual optimization target value, and dynamically updates the parameters of the task allocation strategy network. This enables continuous optimization and self-improvement of the task allocation strategy, ensuring optimal system performance during actual operation and enhancing the dynamic adaptability and robustness of the production line. Attached Figure Description

[0058] Figure 1 This is a flowchart illustrating the task allocation and optimization scheduling method for a multi-robot collaborative industrial assembly line according to an embodiment of the present invention.

[0059] Figure 2 This is a schematic diagram of the task allocation and optimization scheduling system for a multi-robot collaborative industrial assembly line according to an embodiment of the present invention. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0062] Figure 1 This is a flowchart illustrating the task allocation and optimization scheduling method for a multi-robot collaborative industrial assembly line according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0063] S11. A task allocation strategy network is constructed using a deep reinforcement learning algorithm. The input layer of the task allocation strategy network is set as a state encoding layer, the hidden layer is set as a multi-layer convolutional neural network and a fully connected layer, and the output layer is set as an action probability distribution layer. The workpiece information in the multi-robot collaborative assembly line is preprocessed. The workpiece type is generated into a type feature vector through one-hot encoding, and the workpiece processing priority and time limit constraints are generated into an attribute feature vector through numerical encoding. The type feature vector and the attribute feature vector are combined to generate a state vector as the input of the task allocation strategy network.

[0064] S12. A multi-objective optimization function is designed to train the task allocation strategy network. The multi-objective optimization function includes the objectives of minimizing total assembly time, maximizing robot load balancing, and minimizing energy consumption. The three optimization objectives are combined into a unified training objective using a weighted summation method. A robot task mapping matrix is ​​constructed as the output of the task allocation strategy network. The elements of the robot task mapping matrix represent the probability values ​​of the robot executing the corresponding task, and the probability of infeasible allocation schemes is set to zero according to the robot's working capacity constraints. The task allocation strategy network is trained using a proximal policy optimization algorithm. The parameters of the task allocation strategy network are updated by obtaining state-action pairs and reward values ​​through interaction with the simulation environment.

[0065] S13. Encode the workpiece information in the multi-robot collaborative assembly line into a state vector and input it into the trained task allocation strategy network to output the optimal task allocation scheme; send the optimal task allocation scheme to the multi-robot collaborative assembly line for execution; collect task completion time, robot load distribution and energy consumption data, calculate the actual optimization target value, and dynamically optimize and update the parameters of the task allocation strategy network using an online learning method.

[0066] In one optional implementation, the workpiece information in the multi-robot collaborative assembly line is preprocessed. The workpiece type is generated as a type feature vector through one-hot encoding, and the workpiece processing priority and time constraints are generated as attribute feature vectors through numerical encoding. The type feature vector and the attribute feature vector are combined to generate a state vector, which is then used as input to the task allocation strategy network.

[0067] The workpiece type information is preprocessed by one-hot encoding. A binary type feature vector is constructed based on the total number of workpiece types in the production line. The workpiece is marked as 1 at the corresponding type position and 0 at other type positions to obtain the workpiece type feature vector.

[0068] The workpiece processing priority information and the workpiece processing time limit information are preprocessed with numerical encoding. The workpiece processing priority information is mapped to a priority value in the range of 0 to 1 according to the priority level of the workpiece in the production plan. The remaining time is calculated according to the processing deadline of the workpiece and the maximum and minimum value normalization is performed to obtain the time limit value. The priority value and the time limit value are combined to generate an attribute feature vector.

[0069] The type feature vector and the attribute feature vector are combined and concatenated according to a preset dimension order to generate a workpiece state vector. The mean and standard deviation of each dimension of the workpiece state vector are calculated, and a standardized state vector is obtained by standardization.

[0070] The standardized state vector is input into the task allocation policy network, which is constructed based on a deep reinforcement learning algorithm, and the standardized state vector is used as the input to the task allocation policy network.

[0071] In a multi-robot collaborative assembly line, a task allocation method based on workpiece information preprocessing and deep reinforcement learning is proposed to optimize task allocation. This method preprocesses workpiece information to generate state vectors, which serve as input to the reinforcement learning network, enabling the network to learn the optimal allocation strategy more effectively.

[0072] First, obtain information about each workpiece in the production line. This information includes the workpiece's basic attributes and constraints. Basic attributes include workpiece type and processing priority. Constraints include workpiece processing time limits. For example, suppose the production line has three types of workpieces: A, B, and C. A certain workpiece's information is: Type A, high priority, time limit 48 hours.

[0073] Next, the workpiece type information is one-hot encoded. A binary type feature vector is constructed based on the total number of workpiece types in the production line. The position corresponding to the workpiece type is marked as 1, and other types are marked as 0. Taking the above example, the type feature vector of this workpiece is [1,0,0].

[0074] Then, the processing priority and time limit information of the workpiece are numerically encoded. The processing priority information of the workpiece is mapped to a priority value in the range of 0 to 1. For example, "high" priority is mapped to 0.8, "medium" priority to 0.5, and "low" priority to 0.2. The remaining time for the workpiece's processing deadline is calculated and normalized to obtain the time limit value. Assuming the current time is 0, the time limit for this workpiece is 48 hours, while the longest time limit for all workpieces is 72 hours and the shortest time limit is 24 hours, then the time limit value for this workpiece is (48-24) / (72-24) = 0.5. The priority value and time limit value are combined to generate an attribute feature vector. Taking the above example, the attribute feature vector of this workpiece is [0.8, 0.5].

[0075] The type feature vector and attribute feature vector are combined and concatenated according to a preset dimensional order to generate the workpiece state vector. Taking the example above, combining the type feature vector [1,0,0] and the attribute feature vector [0.8,0.5] yields the workpiece state vector [1,0,0,0.8,0.5]. The mean and standard deviation of each dimension of all workpiece state vectors are calculated. Assuming the mean of the first dimension of all workpiece state vectors is 0.5 and the standard deviation is 0.2, the value of the first dimension of this workpiece state vector is (1-0.5) / 0.2 = 2.5. This process is repeated for each dimension to obtain a standardized state vector.

[0076] Finally, the standardized state vectors are input into the task allocation policy network. This network, built on a deep reinforcement learning algorithm, takes the standardized state vectors as input and outputs a task allocation scheme for each robot.

[0077] The beneficial effects of this method can be summarized in the following three aspects:

[0078] Improving task allocation efficiency: By preprocessing the workpiece information, the workpiece type, priority, and time limit are transformed into a structured state vector, providing a more effective input for the reinforcement learning network, thereby improving the network's learning efficiency and task allocation efficiency.

[0079] Optimized resource utilization: This method can allocate tasks based on constraints such as workpiece priority and time limit, thereby optimizing the utilization of production resources and reducing robot idle time and waiting time.

[0080] Enhancing system robustness: Policy networks based on deep reinforcement learning can adapt to complex production environments and dynamically changing workpiece information, thereby enhancing the robustness and adaptability of the system.

[0081] In one optional implementation, a multi-objective optimization function is designed to train the task allocation strategy network. The multi-objective optimization function includes three objectives: minimizing total assembly time, maximizing robot load balancing, and minimizing energy consumption. A weighted summation method is used to combine the three optimization objectives into a unified training objective, including:

[0082] A multi-objective optimization function is constructed, which includes the objectives of minimizing total assembly time, maximizing robot load balance, and minimizing energy consumption. The multi-objective optimization function is used to train the task allocation strategy network.

[0083] Real-time operating data of a multi-robot collaborative assembly system is acquired. The real-time operating data includes workpiece processing parameters, robot motion parameters, and tool operation parameters. The processing time is calculated based on the workpiece processing parameters, the motion time is calculated based on the robot motion parameters, and the switching time is calculated based on the tool operation parameters. The processing time, the motion time, and the switching time are accumulated to obtain the total assembly time. The total assembly time is used as the optimization variable for the goal of minimizing the total assembly time.

[0084] The task load distribution of each robot is calculated based on the real-time working condition data. The task workload is determined according to the workpiece processing parameters. The task type is determined according to the robot motion parameters. The execution time is determined according to the tool operation parameters. The product of the task workload, the task type and the execution time is used as the load value. The dispersion of the load value of each robot is calculated. The dispersion is used as the optimization variable for maximizing the robot load balance.

[0085] A system energy consumption model is established based on the real-time operating data. The joint drive energy consumption is calculated based on the robot motion parameters, and the tool execution energy consumption is calculated based on the tool operation parameters. The total system energy consumption is obtained by summing the joint drive energy consumption and the tool execution energy consumption. The total system energy consumption is used as the optimization variable for the energy consumption minimization objective.

[0086] The total assembly time, the degree of dispersion, and the total energy consumption of the system are normalized to obtain standardized optimization variables. Weight coefficients for each optimization objective are set according to production requirements, and the standardized optimization variables are combined into a unified training objective using a weighted summation method.

[0087] The proposed task allocation strategy network training method for multi-robot collaborative assembly systems aims to optimize three objectives: total assembly time, robot load balancing, and energy consumption.

[0088] First, a multi-objective optimization function is constructed. This function contains three sub-objectives: minimizing total assembly time, maximizing robot load balancing, and minimizing energy consumption. These three sub-objectives together constitute the multi-objective optimization function, which is used to train the task allocation policy network, enabling the network to learn the optimal task allocation strategy.

[0089] Next, real-time operating data of the multi-robot collaborative assembly system is acquired. This data includes workpiece processing parameters (e.g., workpiece size, material, processing technology), robot motion parameters (e.g., robot position, speed, acceleration), and tool operation parameters (e.g., tool type, operating force, operation time).

[0090] Then, the total assembly time is calculated based on real-time operating data. The processing time for each workpiece is calculated according to its machining parameters; the movement time of the robot from one workstation to another is calculated according to its motion parameters; and the switching time between different tools is calculated according to its tool operation parameters. The processing times for all workpieces, the movement times for all robots, and the switching times for all tools are added together to obtain the total assembly time. For example, assuming there are two workpieces with processing times of 10 seconds and 15 seconds respectively; two robots with movement times of 5 seconds and 7 seconds respectively; and a tool switching time of 2 seconds, then the total assembly time is 10 + 15 + 5 + 7 + 2 = 39 seconds.

[0091] Subsequently, the robot load balance is calculated based on real-time operating data. First, the workload of each task is determined according to the workpiece machining parameters, for example, the cutting amount required to machine the workpiece is used as the workload. Then, the task type is determined according to the robot motion parameters, for example, moving the robot to a designated position and performing a specific operation is defined as a task type. Next, the execution time of each task is determined according to the tool operation parameters. The task workload, task type, and execution time are multiplied together to obtain the load value of each robot. For example, assuming that robot 1's task workload is 10, task type is 1, and execution time is 5 seconds, then the load value is 10 * 1 * 5 = 50. Finally, the dispersion of all robot load values ​​is calculated, for example, using the standard deviation to measure the dispersion. The smaller the dispersion, the higher the robot load balance.

[0092] Next, a system energy consumption model is established based on real-time operating data. The joint drive energy consumption of each robot is calculated based on its motion parameters, such as the rotation angle and speed of the robot joints. The execution energy consumption of each tool is calculated based on its operating parameters, such as the tool's power and usage time. The total system energy consumption is obtained by adding the joint drive energy consumption of all robots and the execution energy consumption of all tools. For example, assuming the joint drive energy consumption of two robots is 100 Wh and 120 Wh respectively, and the tool execution energy consumption is 50 Wh, then the total system energy consumption is 100 + 120 + 50 = 270 Wh.

[0093] Finally, the total assembly time, dispersion, and total system energy consumption are normalized. For example, the min-max normalization method can be used to scale these values ​​to between 0 and 1. Then, the weight coefficients of each optimization objective are set according to production requirements; for example, the weights can be set to 0.5, 0.3, and 0.2 based on the importance of total assembly time, load balancing, and energy consumption, respectively. Finally, a weighted summation method is used to combine the standardized optimization variables into a unified training objective for training the task allocation strategy network. For example, assuming the normalized total assembly time is 0.8, the dispersion is 0.2, and the total system energy consumption is 0.5, then the unified training objective is 0.5*0.8 + 0.3*(1-0.2) + 0.2*0.5 = 0.74.

[0094] Beneficial effects:

[0095] 1. Reduce total assembly time: By optimizing task allocation strategies, the total time for workpiece processing, robot movement, and tool changeover can be effectively reduced, thereby improving production efficiency.

[0096] 2. Improve robot load balancing: By distributing tasks evenly, it is possible to avoid situations where some robots are overworked while others are idle, thereby improving equipment utilization and extending equipment life.

[0097] 3. Reduce system energy consumption: By optimizing robot motion trajectory and tool operation parameters, the energy consumption of joint drive and tool execution can be reduced, thereby reducing production costs.

[0098] In one optional implementation, a robot task mapping matrix is ​​constructed as the output of the task allocation policy network. The elements of the robot task mapping matrix represent the probability values ​​of the robot performing the corresponding task, and the probabilities of infeasible allocation schemes are set to zero based on robot capability constraints. The task allocation policy network is trained using a proximal policy optimization algorithm, and the parameters of the task allocation policy network are updated by interacting with the simulation environment to obtain state-action pairs and reward values, including:

[0099] A robot task mapping matrix is ​​constructed as the output of the task allocation strategy network, and the matrix elements in the robot task mapping matrix represent the probability values ​​of the robot executing the corresponding task;

[0100] Obtain robot working capacity constraints, including robot working range parameters, robot load parameters, and robot end effector configuration parameters; and identify infeasible task allocation schemes based on the working capacity constraints.

[0101] A constraint processing matrix is ​​generated based on the infeasible task allocation scheme. The constraint processing matrix has the same dimension as the robot task mapping matrix. The probability value corresponding to the infeasible task allocation scheme is set to zero through the constraint processing matrix.

[0102] A simulation interactive environment is constructed to simulate a multi-robot collaborative task allocation scenario. The task allocation action is performed in the simulation interactive environment based on the probability distribution of the robot task mapping matrix.

[0103] The task allocation policy network is trained using a proximal policy optimization algorithm, and state-action pairs and corresponding reward values ​​are obtained by interactive sampling with the simulation environment.

[0104] The policy gradient is constructed based on the state-action pair and the reward value. A truncated objective function is used to limit the policy update step size. An entropy regularization term is introduced to balance policy exploration and utilization. The parameters of the task allocation policy network are iteratively updated.

[0105] To achieve efficient task allocation for multi-robot collaborative execution, a task allocation method based on reinforcement learning is proposed. This method constructs a task allocation policy network, whose output is a robot task mapping matrix, where each element represents the probability of a robot executing a corresponding task. The network is trained in a simulation environment to learn the optimal task allocation policy.

[0106] First, a task allocation strategy network is constructed. The network's input is the current environmental state, such as the position and load status of each robot, and the attributes and location of each task. The network's output is a robot-task mapping matrix. Assuming there are 3 robots and 4 tasks, the output matrix will be 3x4. Each element in the matrix represents the probability that the corresponding robot will execute the corresponding task. For example, the value of matrix element (1,2) is 0.8, indicating that the probability of robot 1 executing task 2 is 0.8.

[0107] Next, the constraints on the robot's working capabilities are determined. These constraints include the robot's working range parameters, robot load parameters, and robot end effector configuration parameters. For example, robot 1's working range is a circular area with a radius of 5 meters, its maximum load is 10 kg, and it is equipped with a gripper-type end effector. Task 1 is outside robot 1's working range, Task 2 requires moving a 15 kg object, and Task 3 requires the use of a suction cup-type end effector. Based on these constraints, it can be identified that robot 1 cannot perform Task 1 and Task 2. Robot 1 also faces limitations in performing Task 3; it is necessary to confirm whether its gripper can effectively grasp the object corresponding to Task 3.

[0108] Then, a constraint processing matrix is ​​generated based on infeasible task assignment schemes. This matrix has the same dimensions as the robot task mapping matrix. For infeasible task assignment schemes, the corresponding elements in the constraint processing matrix are set to 0, and the other elements are set to 1. For example, since robot 1 cannot perform tasks 1 and 2, the elements at positions (1,1) and (1,2) in the 3x4 constraint processing matrix are set to 0. There are restrictions on robot 1 performing task 3; its feasibility needs to be determined based on the actual situation. If it is determined to be infeasible, the element at position (1,3) is also set to 0.

[0109] Next, a simulation environment is constructed. This environment simulates a multi-robot collaborative task allocation scenario, including virtual robots, tasks, and work areas. Based on the probability distribution of the robot task mapping matrix, task allocation actions are performed in the simulation environment. For example, according to the robot task mapping matrix, robot 1 executes task 2 with a probability of 0.8 and task 4 with a probability of 0.2. In the simulation environment, random sampling is performed based on these probabilities to determine which task robot 1 ultimately executes. If the sampling result is to execute task 2, but task 2 exceeds robot 1's load capacity, then the task allocation fails.

[0110] Next, a proximal policy optimization algorithm is used to train the task allocation policy network. State-action pairs and their corresponding reward values ​​are obtained through interactive sampling with the simulation environment. For example, if the robot successfully completes the task, it receives a positive reward; if the task allocation fails or the task execution time is too long, it receives a negative reward.

[0111] Finally, a policy gradient is constructed based on state-action pairs and reward values. A truncated objective function is used to limit the policy update step size, and an entropy regularization term is introduced to balance policy exploration and utilization, iteratively updating the parameters of the task assignment policy network. For example, initially, the probability of robot 1 performing each task is evenly distributed. After multiple rounds of training, the network learns that robot 1 has a higher probability of performing task 4 because it is more suitable for performing task 4.

[0112] For example, in a specific scenario, there are 2 robots and 3 tasks. Robot 1 has a maximum payload of 5kg and a working radius of 3m; Robot 2 has a maximum payload of 10kg and a working radius of 5m. Task 1 weighs 2kg and is located at (1,1); Task 2 weighs 8kg and is located at (4,4); Task 3 weighs 3kg and is located at (2,3). After training, the robot task mapping matrix output by the policy network might be as follows:

[0113] [[0.9,0.05,0.05],[0.05,0.05,0.9]].

[0114] This indicates that robot 1 is more inclined to perform tasks 1 and 3, while robot 2 is more inclined to perform task 3.

[0115] Beneficial effects

[0116] 1. Improve task allocation efficiency: The policy network obtained through reinforcement learning training can quickly generate efficient task allocation schemes, avoiding complex calculations and manual intervention in traditional methods, thereby shortening task completion time.

[0117] 2. Enhanced System Robustness: This method can adapt to dynamically changing environments, such as robot malfunctions or changes in task requirements. Through interactive learning with the simulation environment, the policy network can automatically adjust task allocation strategies to ensure successful task completion.

[0118] 3. Optimized resource utilization: This method can allocate tasks based on robot capability constraints, avoiding resource waste. For example, tasks exceeding the robot's load capacity can be assigned to that robot. This allows robots to better leverage their respective strengths, thereby improving overall efficiency.

[0119] In one optional implementation, the sequence of workpieces to be processed in the multi-robot collaborative assembly line is encoded as a state vector and input into the trained task allocation strategy network to output an optimal task allocation scheme; the optimal task allocation scheme is then distributed to the multi-robot collaborative assembly line for execution, including:

[0120] The state vectors corresponding to all workpieces in the workpiece sequence are used to form a state matrix, and the state matrix is ​​padded to a fixed dimension using padding markers. The state matrix is ​​then input into the trained task allocation strategy network, which outputs a workpiece-robot allocation probability matrix.

[0121] A temperature parameter is introduced to adjust the probability distribution of the allocation probability matrix. Based on the adjusted probability distribution, a greedy strategy is used to generate an optimal task allocation scheme. The optimal task allocation scheme determines the allocation relationship between each workpiece to be processed and the robots in the multi-robot collaborative assembly line.

[0122] According to the optimal task allocation scheme, the processing task of each workpiece to be processed is decomposed into a sequence of basic operation instructions, and an execution time window is generated for each basic operation instruction based on process constraints and resource constraints.

[0123] Based on the execution time window, a collision-free motion trajectory is planned for each robot, and the collision-free motion trajectory is converted into a corresponding robot control command; the robot control command is then sent to the corresponding robot in the multi-robot collaborative assembly line for execution.

[0124] The execution status of the robot control commands is monitored in real time. When the execution deviation is detected to exceed the preset threshold, the time constraint feature in the workpiece state vector is updated, and the updated state matrix is ​​re-input into the task allocation strategy network to generate a new optimal task allocation scheme.

[0125] The newly generated optimal task allocation scheme is smoothly transitioned to the original task allocation scheme. The smoothed task allocation scheme is then converted into new robot control commands and sent to the multi-robot collaborative assembly production line for execution.

[0126] A task allocation method for multi-robot collaborative assembly line optimizes task allocation among multiple robots in the production line by encoding the workpiece state and combining it with a deep learning task allocation strategy network, thereby improving production efficiency.

[0127] First, obtain the sequence information of the workpieces to be processed in the multi-robot collaborative assembly line. For example, the production line currently has 5 workpieces to be processed, denoted as workpiece A, workpiece B, workpiece C, workpiece D, and workpiece E.

[0128] Next, a workpiece status coding module is constructed, which includes a feature extraction unit and a position coding unit. The feature extraction unit is responsible for collecting the physical features, process requirement features, and time constraint features of each workpiece to be processed. Taking workpiece A as an example, its physical features include geometric dimensions (length 10cm, width 5cm, height 2cm), weight (1kg), and material (metal); process requirement features include machining accuracy (0.1mm), surface quality requirement (Ra0.8), number of processes (3), and process constraints (process 1 must be completed before process 2); time constraint features include start processing time (November 1, 2023, 8:00 AM), completion time (November 1, 2023, 12:00 PM), and delivery time (November 2, 2023, 8:00 AM). These features are concatenated into a feature vector. Similar feature extraction is performed on all workpieces to form their respective feature vectors.

[0129] The position encoding unit uses a sine function to encode the position information of the workpiece in the sequence. For example, workpiece A is the first in the sequence, workpiece B is the second, and so on. Position encoding generates a position encoding vector for each workpiece.

[0130] The feature vector and the position encoding vector are combined to form the workpiece state vector, which is used to represent the complete state information of the workpiece to be processed. For example, the state vector of workpiece A is formed by concatenating its feature vector and position encoding vector. The state vectors of all workpieces are assembled into a state matrix. If the number of workpieces to be processed is less than a preset value (e.g., 10), padding markers are used to complete the state matrix, fixing its dimension to 10xN, where N is the dimension of the state vector.

[0131] The state matrix is ​​input into a pre-trained task assignment policy network. This network can be a multi-layer fully connected neural network or other suitable deep learning network architecture. The network input is the state matrix, and the output is a job-robot assignment probability matrix. For example, the network outputs a 5x3 matrix representing the probability of assigning 5 jobs to 3 robots.

[0132] To avoid getting trapped in local optima, a temperature parameter is introduced to adjust the probability distribution of the assignment probability matrix. A higher temperature parameter results in a more uniform probability distribution, while a lower temperature parameter leads to a more concentrated probability distribution. Based on the adjusted probability distribution, a greedy strategy is used to generate the optimal task assignment scheme. For example, the assignment relationship with the highest probability is determined as the final assignment result, thus determining which robot each workpiece is assigned to.

[0133] Based on the optimal task allocation scheme, the processing task of each workpiece is decomposed into a sequence of basic operation instructions. For example, the processing task of workpiece A is decomposed into the instruction sequence of "grab", "move to station 1", "processing operation 1", "move to station 2", "processing operation 2", "move to station 3", "processing operation 3", and "place". Based on process constraints and resource constraints, an execution time window is generated for each basic operation instruction. For example, the execution time window for "processing operation 1" is 9:00-10:00.

[0134] Based on the execution time window, plan a collision-free motion trajectory for each robot. For example, robot 1 needs to first move to the initial position of workpiece A for gripping, then move to station 1 for processing, then move to station 2 for processing, and so on. Convert the collision-free motion trajectory into corresponding robot control instructions.

[0135] The robot control commands are sent to the corresponding robots in the multi-robot collaborative assembly line for execution.

[0136] The system monitors the execution status of robot control commands in real time. When an execution deviation is detected to exceed a preset threshold, such as a processing time delay for a workpiece, the time constraint features in the workpiece state vector are updated. The updated state matrix is ​​then re-inputted into the task allocation strategy network to generate a new optimal task allocation scheme.

[0137] The newly generated optimal task allocation scheme is smoothly transitioned to the original task allocation scheme to avoid drastic changes in robot movements. The smoothed task allocation scheme is then converted into new robot control commands and issued to the production line for execution.

[0138] Beneficial effects:

[0139] 1. Improve production efficiency: By optimizing the task allocation scheme, the idle time and waiting time of robots are reduced, thereby improving the overall efficiency of the production line.

[0140] 2. Improve product quality: Precise task allocation and time control ensure the processing accuracy and quality of each workpiece, thereby improving the overall quality of the product.

[0141] 3. Enhanced system flexibility: The system can adjust the task allocation scheme according to the real-time status, enabling it to adapt to changes in the production environment and enhancing its flexibility and robustness.

[0142] In one optional implementation, collecting task completion time, robot load distribution, and energy consumption data, calculating the actual optimization target value, and dynamically optimizing and updating the parameters of the task allocation strategy network using an online learning method includes:

[0143] Collect task execution data for each robot in a multi-robot collaborative assembly production line. The task execution data includes process start timestamp, process end timestamp, task queue length, resource utilization rate, joint motor current value, and joint motor voltage value.

[0144] The actual task completion time is calculated based on the process start timestamp and the process end timestamp; the robot load distribution is calculated based on the task queue length and the resource occupancy rate; and the robot energy consumption is calculated based on the joint motor current value and the joint motor voltage value.

[0145] The actual task completion time, the robot load distribution, and the robot energy consumption are respectively used as time dimension indicators, load dimension indicators, and energy consumption dimension indicators. An adaptive weighting method is used to combine the three dimension indicators into a comprehensive optimization target value.

[0146] An experience replay buffer is constructed, and the workpiece status information, task allocation decision and the comprehensive optimization target value are combined to form training samples and stored in the experience replay buffer. When the number of samples in the experience replay buffer exceeds a preset threshold, the experience replay buffer is updated in a first-in-first-out manner.

[0147] Training batches are randomly sampled from the experience replay buffer. The workpiece state information is input into the task allocation strategy network to obtain the predicted task allocation scheme. The strategy loss between the predicted task allocation scheme and the actual task allocation decision is calculated. Based on the strategy loss and the comprehensive optimization objective value, the parameters of the task allocation strategy network are updated using the stochastic gradient descent method, and a momentum factor is introduced for soft updates.

[0148] An online optimization method for task allocation strategy in multi-robot collaborative assembly production lines can dynamically adjust the task allocation strategy according to actual production conditions, thereby improving production efficiency and reducing energy consumption.

[0149] First, an experimental platform for a multi-robot collaborative assembly line was built. This platform includes several robots with independent control systems, conveyor belts for transporting workpieces, and sensors for monitoring the production process. For example, the platform includes three six-degree-of-freedom robotic arms, the conveyor belt speed is adjustable, and the sensors can collect data such as robot joint positions, current, and voltage.

[0150] Next, task execution data for each robot on the production line is collected. Specifically, the start and end timestamps of each process are recorded. For example, robot 1 starts process A at 10:00:00 and ends at 10:00:15. The task queue length is also recorded; for example, robot 2's current task queue length is 3, indicating 3 tasks are waiting to be executed. Furthermore, resource utilization is recorded; for example, robot 3's end effector is currently busy, with a resource utilization rate of 100%. Finally, the current and voltage values ​​of each robot joint motor are recorded; for example, robot 1's first joint motor has a current of 2A and a voltage of 24V.

[0151] Then, based on the collected data, the actual task completion time, robot load distribution, and robot energy consumption are calculated. The actual task completion time can be obtained by subtracting the start time of the process from the end timestamp. For example, robot 1 completes process A in 15 seconds. Robot load distribution can be measured by task queue length and resource utilization. For example, robot 2 has a task queue length of 3 and a resource utilization of 50%, indicating a moderate load. Robot energy consumption can be calculated using the joint motor current and voltage values. For example, the power of the first joint motor of robot 1 is 48W.

[0152] Next, the calculated actual task completion time, robot load distribution, and robot energy consumption are used as time-dimensional, load-dimensional, and energy-dimensional indicators, respectively. An adaptive weighting method is then used to combine these three indicators into a comprehensive optimization target value. For example, if the weight of the time dimension is set to 0.5, the weight of the load dimension to 0.3, and the weight of the energy consumption dimension to 0.2, then the comprehensive optimization target value is 0.5 * task completion time + 0.3 * load distribution + 0.2 * energy consumption. Assuming the task completion time is 15 seconds, the load distribution is 0.75, and the energy consumption is 50W, then the comprehensive optimization target value is 0.5 * 15 + 0.3 * 0.75 + 0.2 * 50 = 18.225.

[0153] An experience replay buffer is constructed, storing training samples composed of workpiece state information, task allocation decisions, and comprehensive optimization target values. For example, workpiece state information may include workpiece position and size, task allocation decisions may be represented as assigning a workpiece to a specific robot, and the comprehensive optimization target value is 18.225. When the number of samples in the experience replay buffer exceeds a preset threshold, such as 1000 samples, the buffer is updated using a first-in, first-out (FIFO) approach.

[0154] Training batches are randomly sampled from the experience replay buffer, for example, 64 samples. Workpiece state information is input into the task allocation policy network to obtain a predicted task allocation scheme, for example, assigning workpiece 1 to robot 2. The policy loss between the predicted task allocation scheme and the actual task allocation decision is calculated. For example, if the actual task allocation decision is to assign workpiece 1 to robot 1, the policy loss is relatively large. Based on the policy loss and the comprehensive optimization objective value, the parameters of the task allocation policy network are updated using stochastic gradient descent, and a momentum factor is introduced for soft updates. For example, the learning rate is set to 0.001 and the momentum factor is set to 0.9.

[0155] Beneficial effects:

[0156] 1. Improve production efficiency: By dynamically optimizing task allocation strategies, task completion time can be reduced, thereby improving the overall production efficiency of the production line.

[0157] 2. Balancing robot load: This method can make the load of each robot more balanced, avoiding the situation where some robots are overworked while others are idle, thereby improving equipment utilization and extending equipment life.

[0158] 3. Reduce energy consumption: By optimizing task allocation strategies, the energy consumption of robots can be reduced, thereby saving energy and reducing production costs.

[0159] Figure 2 This is a schematic diagram of the structure of a multi-robot collaborative industrial assembly line task allocation and optimization scheduling system according to an embodiment of the present invention, as shown below. Figure 2 As shown, the system includes:

[0160] The first unit is used to construct a task allocation strategy network using a deep reinforcement learning algorithm. The input layer of the task allocation strategy network is set as a state encoding layer, the hidden layer is set as a multi-layer convolutional neural network and a fully connected layer, and the output layer is set as an action probability distribution layer. The workpiece information in the multi-robot collaborative assembly line is preprocessed. The workpiece type is generated into a type feature vector through one-hot encoding, and the workpiece processing priority and time limit constraints are generated into an attribute feature vector through numerical encoding. The type feature vector and the attribute feature vector are combined to generate a state vector as the input of the task allocation strategy network.

[0161] The second unit is used to design a multi-objective optimization function to train the task allocation strategy network. The multi-objective optimization function includes the objectives of minimizing total assembly time, maximizing robot load balancing, and minimizing energy consumption. The three optimization objectives are combined into a unified training objective using a weighted summation method. A robot task mapping matrix is ​​constructed as the output of the task allocation strategy network. The elements of the robot task mapping matrix represent the probability values ​​of the robot executing the corresponding task, and the probability of infeasible allocation schemes is set to zero according to the robot's working capacity constraints. The task allocation strategy network is trained using a proximal policy optimization algorithm. By interacting with the simulation environment to obtain state-action pairs and reward values, the parameters of the task allocation strategy network are updated.

[0162] The third unit is used to encode the workpiece information in the multi-robot collaborative assembly line into a state vector and input it into the trained task allocation strategy network to output the optimal task allocation scheme; to send the optimal task allocation scheme to the multi-robot collaborative assembly line for execution; to collect task completion time, robot load distribution and energy consumption data, calculate the actual optimization target value, and dynamically optimize and update the parameters of the task allocation strategy network using an online learning method.

[0163] A third aspect of the embodiments of the present invention,

[0164] An electronic device is provided, comprising:

[0165] processor;

[0166] Memory used to store processor-executable instructions;

[0167] The processor is configured to invoke instructions stored in the memory to execute the aforementioned method.

[0168] Fourth aspect of the present invention,

[0169] A computer-readable storage medium is provided, having stored thereon computer program instructions that, when executed by a processor, implement the aforementioned method.

[0170] This invention can be a method, apparatus, system, and / or computer program product. The computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention.

[0171] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for task allocation and optimized scheduling of multi-robot collaborative industrial assembly lines, characterized in that, include: A task allocation strategy network is constructed using a deep reinforcement learning algorithm. The input layer of the task allocation strategy network is set as a state encoding layer, the hidden layer is set as a multi-layer convolutional neural network and a fully connected layer, and the output layer is set as an action probability distribution layer. The workpiece information in the multi-robot collaborative assembly line is preprocessed. The workpiece type is generated into a type feature vector through one-hot encoding, and the workpiece processing priority and time limit constraints are generated into an attribute feature vector through numerical encoding. The type feature vector and the attribute feature vector are combined to generate a state vector as the input of the task allocation strategy network. A multi-objective optimization function is designed to train the task allocation strategy network. This function includes objectives for minimizing total assembly time, maximizing robot load balancing, and minimizing energy consumption. A weighted summation method is used to combine these three objectives into a unified training objective. A robot task mapping matrix is ​​constructed as the output of the task allocation strategy network. The elements of this matrix represent the probability values ​​of the robot executing the corresponding task, and the probabilities of infeasible allocation schemes are set to zero based on robot working capacity constraints. A proximal policy optimization algorithm is used to train the task allocation strategy network. State-action pairs and reward values ​​are obtained through interaction with the simulation environment, and the parameters of the task allocation strategy network are updated accordingly. The workpiece information in the multi-robot collaborative assembly line is encoded as a state vector and input into the trained task allocation strategy network to output the optimal task allocation scheme; the optimal task allocation scheme is then sent to the multi-robot collaborative assembly line for execution. Data on task completion time, robot load distribution, and energy consumption are collected. The actual optimization target value is calculated, and the parameters of the task allocation strategy network are dynamically optimized and updated using an online learning method.

2. The method according to claim 1, characterized in that, Preprocessing of workpiece information in a multi-robot collaborative assembly line involves generating a type feature vector from the workpiece type using one-hot encoding, generating an attribute feature vector from the workpiece processing priority and time constraints using numerical encoding, and combining the type feature vector and the attribute feature vector to generate a state vector as input to the task allocation strategy network. The workpiece type information is preprocessed by one-hot encoding. A binary type feature vector is constructed based on the total number of workpiece types in the production line. The workpiece is marked as 1 at the corresponding type position and 0 at other type positions to obtain the workpiece type feature vector. The workpiece processing priority information and workpiece processing time limit information are preprocessed with numerical encoding. The workpiece processing priority information is mapped to a priority value in the range of 0 to 1 according to the priority level of the workpiece in the production plan. The remaining time is calculated according to the workpiece processing deadline and the maximum and minimum values ​​are normalized to obtain the time limit value. The priority value and the time limit value are combined to generate an attribute feature vector. The type feature vector and the attribute feature vector are combined and concatenated according to a preset dimension order to generate a workpiece state vector. The mean and standard deviation of each dimension of the workpiece state vector are calculated, and a standardized state vector is obtained by standardization. The standardized state vector is input into the task allocation policy network, which is constructed based on a deep reinforcement learning algorithm, and the standardized state vector is used as the input to the task allocation policy network.

3. The method according to claim 1, characterized in that, A multi-objective optimization function is designed to train the task allocation strategy network. This multi-objective optimization function includes three objectives: minimizing total assembly time, maximizing robot load balancing, and minimizing energy consumption. A weighted summation method is used to combine these three objectives into a unified training objective, which includes: A multi-objective optimization function is constructed, which includes the objectives of minimizing total assembly time, maximizing robot load balance, and minimizing energy consumption. The multi-objective optimization function is used to train the task allocation strategy network. Real-time operating data of a multi-robot collaborative assembly system is acquired. The real-time operating data includes workpiece processing parameters, robot motion parameters, and tool operation parameters. The processing time is calculated based on the workpiece processing parameters, the motion time is calculated based on the robot motion parameters, and the switching time is calculated based on the tool operation parameters. The processing time, the motion time, and the switching time are accumulated to obtain the total assembly time. The total assembly time is used as the optimization variable for the goal of minimizing the total assembly time. The task load distribution of each robot is calculated based on the real-time working condition data. The task workload is determined according to the workpiece processing parameters. The task type is determined according to the robot motion parameters. The execution time is determined according to the tool operation parameters. The product of the task workload, the task type and the execution time is used as the load value. The dispersion of the load value of each robot is calculated. The dispersion is used as the optimization variable for maximizing the robot load balance. A system energy consumption model is established based on the real-time operating data. The joint drive energy consumption is calculated based on the robot motion parameters, and the tool execution energy consumption is calculated based on the tool operation parameters. The total system energy consumption is obtained by summing the joint drive energy consumption and the tool execution energy consumption. The total system energy consumption is used as the optimization variable for the energy consumption minimization objective. The total assembly time, the degree of dispersion, and the total energy consumption of the system are normalized to obtain standardized optimization variables. Weight coefficients for each optimization objective are set according to production requirements, and the standardized optimization variables are combined into a unified training objective using a weighted summation method.

4. The method according to claim 1, characterized in that, A robot task mapping matrix is ​​constructed as the output of the task allocation policy network. The elements of the robot task mapping matrix represent the probability values ​​of the robot performing the corresponding task, and the probabilities of infeasible allocation schemes are set to zero based on robot work capacity constraints. The task allocation policy network is trained using a proximal policy optimization algorithm. By interacting with the simulation environment to obtain state-action pairs and reward values, the parameters of the task allocation policy network are updated, including: A robot task mapping matrix is ​​constructed as the output of the task allocation strategy network, and the matrix elements in the robot task mapping matrix represent the probability values ​​of the robot executing the corresponding task; Obtain robot working capacity constraints, including robot working range parameters, robot load parameters, and robot end effector configuration parameters; and identify infeasible task allocation schemes based on the working capacity constraints. A constraint processing matrix is ​​generated based on the infeasible task allocation scheme. The constraint processing matrix has the same dimension as the robot task mapping matrix. The probability value corresponding to the infeasible task allocation scheme is set to zero through the constraint processing matrix. A simulation interactive environment is constructed to simulate a multi-robot collaborative task allocation scenario. The task allocation action is performed in the simulation interactive environment based on the probability distribution of the robot task mapping matrix. The task allocation policy network is trained using a proximal policy optimization algorithm, and state-action pairs and corresponding reward values ​​are obtained by interactive sampling with the simulation environment. The policy gradient is constructed based on the state-action pair and the reward value. A truncated objective function is used to limit the policy update step size. An entropy regularization term is introduced to balance policy exploration and utilization. The parameters of the task allocation policy network are iteratively updated.

5. The method according to claim 1, characterized in that, The workpiece information in the multi-robot collaborative assembly production line is encoded as a state vector and input into the trained task allocation strategy network to output the optimal task allocation scheme. Sending the optimal task allocation scheme to the multi-robot collaborative assembly line for execution includes: The state vectors corresponding to all workpieces in the workpiece sequence are used to form a state matrix, and the state matrix is ​​padded to a fixed dimension using padding markers. The state matrix is ​​then input into the trained task allocation strategy network, which outputs a workpiece-robot allocation probability matrix. A temperature parameter is introduced to adjust the probability distribution of the allocation probability matrix. Based on the adjusted probability distribution, a greedy strategy is used to generate an optimal task allocation scheme. The optimal task allocation scheme determines the allocation relationship between each workpiece to be processed and the robots in the multi-robot collaborative assembly line. According to the optimal task allocation scheme, the processing task of each workpiece to be processed is decomposed into a sequence of basic operation instructions, and an execution time window is generated for each basic operation instruction based on process constraints and resource constraints. Based on the execution time window, a collision-free motion trajectory is planned for each robot, and the collision-free motion trajectory is converted into a corresponding robot control command; the robot control command is then sent to the corresponding robot in the multi-robot collaborative assembly line for execution. The execution status of the robot control commands is monitored in real time. When the execution deviation is detected to exceed the preset threshold, the time constraint feature in the workpiece state vector is updated, and the updated state matrix is ​​re-input into the task allocation strategy network to generate a new optimal task allocation scheme. The newly generated optimal task allocation scheme is smoothly transitioned to the original task allocation scheme. The smoothed task allocation scheme is then converted into new robot control commands and sent to the multi-robot collaborative assembly production line for execution.

6. The method according to claim 1, characterized in that, Collecting task completion time, robot load distribution, and energy consumption data; calculating the actual optimization target value; and dynamically optimizing and updating the parameters of the task allocation strategy network using an online learning approach include: Collect task execution data for each robot in a multi-robot collaborative assembly production line. The task execution data includes process start timestamp, process end timestamp, task queue length, resource utilization rate, joint motor current value, and joint motor voltage value. The actual task completion time is calculated based on the process start timestamp and the process end timestamp; the robot load distribution is calculated based on the task queue length and the resource occupancy rate; and the robot energy consumption is calculated based on the joint motor current value and the joint motor voltage value. The actual task completion time, the robot load distribution, and the robot energy consumption are respectively used as time dimension indicators, load dimension indicators, and energy consumption dimension indicators. An adaptive weighting method is used to combine the three dimension indicators into a comprehensive optimization target value. An experience replay buffer is constructed, and the workpiece status information, task allocation decision and the comprehensive optimization target value are combined to form training samples and stored in the experience replay buffer. When the number of samples in the experience replay buffer exceeds a preset threshold, the experience replay buffer is updated in a first-in-first-out manner. Training batches are randomly sampled from the experience replay buffer. The workpiece state information is input into the task allocation strategy network to obtain the predicted task allocation scheme. The strategy loss between the predicted task allocation scheme and the actual task allocation decision is calculated. Based on the strategy loss and the comprehensive optimization objective value, the parameters of the task allocation strategy network are updated using the stochastic gradient descent method, and a momentum factor is introduced for soft updates.

7. A task allocation and optimization scheduling system for a multi-robot collaborative industrial assembly line, used to implement the method of any one of claims 1-6, characterized in that, include: The first unit is used to construct a task allocation strategy network using a deep reinforcement learning algorithm. The input layer of the task allocation strategy network is set as a state encoding layer, the hidden layer is set as a multi-layer convolutional neural network and a fully connected layer, and the output layer is set as an action probability distribution layer. The workpiece information in the multi-robot collaborative assembly line is preprocessed. The workpiece type is generated into a type feature vector through one-hot encoding, and the workpiece processing priority and time limit constraints are generated into an attribute feature vector through numerical encoding. The type feature vector and the attribute feature vector are combined to generate a state vector as the input of the task allocation strategy network. The second unit is used to design a multi-objective optimization function to train the task allocation strategy network. The multi-objective optimization function includes the objectives of minimizing total assembly time, maximizing robot load balancing, and minimizing energy consumption. The three optimization objectives are combined into a unified training objective using a weighted summation method. A robot task mapping matrix is ​​constructed as the output of the task allocation strategy network. The elements of the robot task mapping matrix represent the probability values ​​of the robot executing the corresponding task, and the probability of infeasible allocation schemes is set to zero according to the robot's working capacity constraints. The task allocation strategy network is trained using a proximal policy optimization algorithm. By interacting with the simulation environment to obtain state-action pairs and reward values, the parameters of the task allocation strategy network are updated. The third unit is used to encode the workpiece information in the multi-robot collaborative assembly line into a state vector, input it into the trained task allocation strategy network, output the optimal task allocation scheme, and send the optimal task allocation scheme to the multi-robot collaborative assembly line for execution. Data on task completion time, robot load distribution, and energy consumption are collected. The actual optimization target value is calculated, and the parameters of the task allocation strategy network are dynamically optimized and updated using an online learning method.

8. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 6.

Images