Production line feeding and discharging control system based on multi-agent cooperation

Abstract

The invention discloses a production line feeding and discharging control system based on multi-agent cooperation, and the system comprises a production data collection module which is used for collecting and preprocessing the operation data of all stations and a feeding and discharging robot; the multi-agent cooperative control module is used for inputting a multi-agent cooperative control framework and training a state and action mapping relation by adopting an improved IPPO method; the state updating module is used for collecting and fusing workpiece space pose information and intelligent agent state information to update task state data; the task scheduling module is used for carrying out loading and unloading operation according to the task state data and outputting an updated scheduling result; the beat optimization module is used for generating a beat control instruction by adopting chaos leapfrog optimization; and the cooperative control module is used for generating a comprehensive cooperative control instruction and driving the production line to operate cooperatively. According to the method, multi-agent reinforcement learning and chaos optimization methods are fused, cooperative control over feeding and discharging of the production line is achieved, and the method has the advantages of being high in adaptability, accurate in rhythm and efficient in cooperation.

Description

Technical Field

[0001] This invention relates to the field of industrial automation control technology, and in particular to a production line loading and unloading control system based on multi-agent cooperation. Background Technology

[0002] Existing production line loading and unloading control systems typically employ independent control of a single robot or centralized scheduling, using preset paths and fixed cycle times to achieve workpiece gripping, handling, and placement. However, with the increasing demands for multi-station parallel operations and flexible manufacturing, traditional control methods exhibit slow response speeds and poor coordination when faced with changes in operating conditions, cycle time fluctuations, and uneven loads, making it difficult to achieve efficient collaboration and adaptive adjustment among multiple robots. Furthermore, existing scheduling strategies largely rely on manual settings or heuristic algorithms, lacking online learning and self-optimization capabilities, thus limiting overall production efficiency.

[0003] In recent years, reinforcement learning and swarm intelligence optimization methods have been introduced into the field of industrial production control to improve robot collaboration and scheduling performance. However, existing solutions generally suffer from problems such as scarce training samples, insufficient policy generalization ability, and disconnect between cycle control and task scheduling, failing to form a complete closed-loop optimization mechanism. Especially in multi-agent systems, the lack of adaptive control for fusion modeling of multi-source perception data and cycle matching leads to poor cycle synchronization and high energy consumption in production lines, making it difficult to meet the intelligent collaboration requirements in complex manufacturing scenarios.

[0004] Therefore, how to provide a production line loading and unloading control system based on multi-agent collaboration is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a production line loading and unloading control system based on multi-agent cooperation. This invention integrates multi-agent reinforcement learning and chaotic optimization methods to achieve collaborative control of loading and unloading on the production line, and has the advantages of strong adaptability, precise cycle time and high efficiency of cooperation.

[0006] A production line loading / unloading control system based on multi-agent cooperation according to an embodiment of the present invention includes: The production data acquisition module is used to collect and preprocess the operating data of each workstation and multiple loading and unloading robots on the production line. The multi-agent cooperative control module is used to input the standardized operating dataset into the multi-agent cooperative control architecture, train the mapping relationship between the state and action of each agent using the improved IPPO method, and generate a cooperative strategy model. The state update module is used to collect workpiece spatial pose information through a 3D vision perception system and position sensors, fuse it with the intelligent agent's state information, input it into the cooperative strategy model, and update the task state data. The task scheduling module is used to control the loading and unloading robots to perform loading and unloading operations based on task status data, collect feedback data to calculate task completion rate and cycle time deviation, and update task scheduling results. The cycle time optimization module is used to calculate and adjust the cycle time matching relationship of each workstation based on historical cycle time data and current load status, and generate adaptive cycle time control instructions by using the chaotic frog jump optimization method. The collaborative control module is used to jointly optimize the execution sequence of each workstation and loading / unloading robots based on task scheduling results and cycle control instructions, generate comprehensive collaborative control instructions, and drive the production line to operate adaptively and collaboratively.

[0007] Optionally, modules can be integrated using the following methods: Collect operational data from each workstation and multiple loading / unloading robots on the production line, preprocess the data, and generate a standardized operational dataset. The standardized operational dataset is input into the multi-agent cooperative control architecture, and the improved IPPO method is used to train the mapping relationship between the state and action of each agent, generating a cooperative strategy model that includes task allocation and timing control. The spatial pose information and real-time position data of the workpiece are collected by a 3D vision perception system and position sensor. After being synchronously fused with the state information of each intelligent agent, the data is input into the collaborative strategy model for reasoning and correction, and the task state data is updated. Based on the task status data, control each loading and unloading robot to perform loading and unloading operations, collect execution feedback data to calculate task completion and cycle deviation, input the cycle deviation information into the collaborative strategy model to update parameters and adjust task scheduling, and output the updated task scheduling results. Based on historical cycle time data and current load status, the chaotic frog-jump optimization method is used to dynamically calculate and adjust the cycle time matching relationship of each workstation, and generate adaptive cycle time control instructions. Based on the updated task scheduling results and adaptive cycle control instructions, the execution sequence of each workstation and loading / unloading robots is jointly optimized to generate comprehensive collaborative control instructions and drive the production line to operate adaptively and collaboratively.

[0008] Optionally, the operating data includes robot pose information, conveyor line speed information, fixture status information, hopper position information, and workpiece identification information. The preprocessing includes time synchronization, noise reduction, normalization, removal of abnormal and missing data, and standardization of coordinates and format.

[0009] Optionally, the generation of the collaborative strategy model includes the following specific steps: The standardized operational dataset is divided into a subset of state data, a subset of action data, and a subset of reward data, and then input into a multi-agent collaborative control architecture. The multi-agent collaborative control architecture consists of multiple independent agents, each agent corresponding to a loading / unloading robot or a workstation node. Each agent is a control unit with independent perception, decision-making, and execution functions. Each agent includes a perception module, a policy network, a value network, and an interface module. In a multi-agent cooperative control architecture, an independent policy network and value network are established for each agent. ； An improved IPPO method is used to train the policy network of each agent. In each iteration, the expected return difference is calculated based on the actual interaction data. A loss function is constructed and a pruning constraint is added to limit the update range. The improved IPPO method introduces a dream-like self-training mechanism to expand the training samples on the basis of the original policy optimization framework, introduces a neural plasticity regulation mechanism to adjust the change relationship of neuron connection weights in the policy network, and combines the joint update structure of shared feature layer and local decision layer to perform parameter synchronization and feature fusion. The mathematical expression for the loss function is: ; in, Indicates the policy network in parameters The overall loss function under, Indicates time step The expected operation on the upsampled data, This represents the ratio of the probability of actions taken under the same state using the new and old strategies. Represents the dominance function. This represents the untrunculated policy gradient term. This indicates that the policy update range will be limited to a certain range. Inside, This represents the policy gradient term after pruning constraints. Update the constraint coefficients for the strategy. These are the weighting coefficients for the error term of the value function. Representing the value network in relation to state The cumulative revenue estimate, This represents the actual cumulative revenue calculated based on actual interaction data. The weighting coefficients for the entropy regularization term are... This indicates that the agent is at time step The execution action, Indicates at time step The environmental state of the intelligent agent at all times. Indicates the policy network in parameters Below, in state Choose action The probability distribution, The entropy value represents the strategy; During the training phase, a dream-like self-training mechanism is introduced to generate a potential environment model based on historical interaction data. Virtual policy replay is performed in the potential environment model to generate an expanded sample set and incorporate it into the training dataset. Based on the updated training dataset, a neural plasticity adjustment mechanism is introduced during the parameter update phase. The change range of neuron connection weights in the policy network is adjusted according to the correlation between action and reward. The weight parameters are updated according to the inverse relationship between action reward ratio and error term. The neural plasticity adjustment mechanism includes calculating the initial gradient of connection weights based on the input-output relationship of each neuron in the policy network, normalizing the gradient and generating a plasticity adjustment factor by combining the correlation coefficient between action reward and error term. The plasticity adjustment factor is applied to the weight update formula to correct and iteratively update the weight parameters of each layer of the policy network layer by layer. At the same time, the parameter vector of the corresponding layer of the value network is updated simultaneously to obtain the updated policy network parameter and value network parameter set. Through multiple rounds of iterative training, the policy network parameters and value network parameters of each agent are integrated. Under the framework of the cooperative policy model, the shared feature layer and the local decision layer are jointly updated to generate a cooperative policy model that includes task allocation and timing control.

[0010] Optionally, the update of the task status data includes: collecting workpiece spatial pose data output by the 3D vision perception system and real-time position data collected by the position sensor; performing attitude calculation and coordinate system calibration on the three-dimensional coordinate point cloud of the workpiece to obtain the attitude matrix and displacement vector of the workpiece in the global coordinate system of the production line; performing timestamp matching and synchronization between the attitude matrix and displacement vector and the pose parameters in the status information of each intelligent agent; performing feature encoding and normalization processing on the synchronized data to generate a fused state feature vector; inputting the fused state feature vector into the state update module of the collaborative strategy model; performing inference calculation on the execution state of the current task stage to obtain the predicted action output and state evaluation results of each intelligent agent; calculating the state deviation based on the predicted action output and execution feedback data; weighting and correcting the state deviation and the original state parameters to generate an updated state vector; performing normalization processing and index encoding; and outputting task status data including workstation running status, loading and unloading operation progress parameters, and cycle deviation information.

[0011] Optionally, the output of the updated task scheduling result includes: collecting feedback data from each loading and unloading robot during execution; organizing the feedback data into a time series and labeling it with status tags; constructing an execution feedback set including task completion, cycle deviation, and energy consumption information; comparing the execution feedback set with the predicted action sequence output by the collaborative strategy model; calculating the task execution deviation vector; performing weighted decomposition on the deviation vector and generating a task execution error matrix; inputting the task execution error matrix into the parameter update module of the collaborative strategy model; correcting the parameter vectors of each agent based on the cycle deviation information and task completion information in the error matrix; normalizing and fusing the corrected parameter vectors to form a new task scheduling parameter set; synchronously mapping the task scheduling parameter set to each agent node in the multi-agent collaborative control architecture; and outputting the updated task scheduling result.

[0012] Optionally, the generation of the adaptive beat control command includes the following specific steps: Collect historical beat data and current load status data, perform time series smoothing and normalization on the beat data, and generate beat feature sequences; Initialize the chaotic frog jumping optimization method population, set the population size, maximum number of iterations and chaos factor, and randomly initialize the population individuals according to the beat feature sequence and divide them into several subpopulations; In each iteration, individuals within each subgroup are sorted according to the fitness function, the best and worst individuals are selected, and the beat adjustment parameters of the worst individual are updated based on the fitness difference between the two and the chaotic mapping function. The locally optimized beat adjustment parameters are then output. The fitness function is: ; in, Represents the global fitness function. This indicates the number of workstations participating in the cycle time matching optimization. Indicates the first Workstation cycle time deviation, Indicates the first Workstation load imbalance This is a trade-off factor between cycle time and load. Based on the locally optimized beat adjustment parameters, cross-group chaotic search is performed on individuals in each subgroup. The beat parameters are perturbed by chaotic variables and the global optimal solution is updated to obtain the optimized beat matching relationship. Based on the optimized beat matching relationship, parameter calculations are performed to generate adaptive beat control instructions, which include the beat adjustment coefficient, synchronization delay parameter and task coordination cycle of each workstation.

[0013] Optionally, the generation of the integrated collaborative control command includes: performing data alignment and normalization processing on the updated task scheduling results and adaptive cycle control commands; performing feature fusion on the task scheduling parameters, cycle adjustment coefficients, and synchronization delay parameters of each workstation; constructing a joint optimization input matrix including execution order, cycle matching relationship, and inter-workstation dependency relationship; dynamically calculating and adjusting the execution order of the loading and unloading robots based on the joint optimization input matrix; calculating coordination weights according to the workstation load balance degree and task completion time; generating an integrated control vector by weighted summation of the coordination weights and task scheduling parameters; performing inverse normalization and parameter mapping processing on the integrated control vector; and outputting an integrated collaborative control command including task allocation commands for each workstation, loading and unloading robot execution order commands, and cycle synchronization parameters.

[0014] The beneficial effects of this invention are: This invention constructs a production line loading and unloading control system based on multi-agent collaboration, achieving adaptive coordination and efficient cooperation among multiple workstations and robots in complex manufacturing environments. The system employs an improved IPPO method, introducing a dream-like self-training mechanism and a neural plasticity adjustment mechanism within the multi-agent framework. This enables each agent to autonomously learn and transfer strategies under different working conditions. Virtual replay of the latent environment model expands the training samples, significantly improving the generalization ability and convergence speed of the policy model. The neural plasticity adjustment mechanism adaptively adjusts the network weight update pattern based on the dynamic correlation between actions and rewards, improving the stability and learning accuracy of the policy network during long-term operation. This ensures that the system maintains efficient and stable collaborative control even when facing production cycle fluctuations and random load disturbances.

[0015] During task execution, this invention achieves synchronous fusion of workpiece spatial pose and robot state information through a 3D vision perception system and position sensors. Combined with a dynamic update mechanism for task state data, it realizes unified state modeling and intelligent reasoning for the workpiece, workstation, and robot. The system can make task allocation and cycle time adjustment decisions based on fused state characteristics, and correct scheduling parameters in real time based on actual execution feedback, thus forming a closed-loop optimization system of task-cycle time-scheduling. By embedding cycle time deviation information as a reward signal into the collaborative strategy model, the system can proactively identify cycle time imbalances and automatically adjust the task sequence and cycle time synchronization parameters, effectively improving the overall coordination and real-time performance of the production line.

[0016] Furthermore, this invention introduces a chaotic frog-jump optimization method to dynamically calculate the cycle time matching relationship between workstations. A chaotic sequence generated by a chaotic mapping function is used to globally perturb and converge to correct the cycle time adjustment parameters, obtaining adaptive cycle time control commands. This achieves precise matching and synchronous control of cycle time between multiple workstations. Combined with task scheduling results, the system further generates comprehensive collaborative control commands to jointly optimize the execution sequence of each workstation and the loading / unloading robots, achieving deep integration of task scheduling and cycle time control. This system not only significantly improves the operating efficiency and cycle time stability of the production line but also enhances its adaptability to load fluctuations and changes in operating conditions. It enables intelligent, self-optimizing, and highly robust operation of multi-robot collaborative loading / unloading in complex and variable production environments, comprehensively improving the intelligence level and sustainable operating performance of the production system. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0018] Figure 1 This is a flowchart of a production line loading and unloading control system based on multi-agent cooperation proposed in this invention. Figure 2 This is a schematic diagram of the collaborative strategy model of a production line loading and unloading control system based on multi-agent cooperation proposed in this invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0020] refer to Figure 1 and Figure 2 A production line loading and unloading control system based on multi-agent collaboration includes: The production data acquisition module is used to collect and preprocess the operating data of each workstation and multiple loading and unloading robots on the production line. The multi-agent cooperative control module is used to input the standardized operating dataset into the multi-agent cooperative control architecture, train the mapping relationship between the state and action of each agent using the improved IPPO method, and generate a cooperative strategy model. The state update module is used to collect workpiece spatial pose information through a 3D vision perception system and position sensors, fuse it with the intelligent agent's state information, input it into the cooperative strategy model, and update the task state data. The task scheduling module is used to control the loading and unloading robots to perform loading and unloading operations based on task status data, collect feedback data to calculate task completion rate and cycle time deviation, and update task scheduling results. The cycle time optimization module is used to calculate and adjust the cycle time matching relationship of each workstation based on historical cycle time data and current load status, and generate adaptive cycle time control instructions by using the chaotic frog jump optimization method. The collaborative control module is used to jointly optimize the execution sequence of each workstation and loading / unloading robots based on task scheduling results and cycle control instructions, generate comprehensive collaborative control instructions, and drive the production line to operate adaptively and collaboratively.

[0021] In this embodiment, the modules are interconnected using the following method: Collect operational data from each workstation and multiple loading / unloading robots on the production line, preprocess the data, and generate a standardized operational dataset. The standardized operational dataset is input into the multi-agent cooperative control architecture, and the improved IPPO method is used to train the mapping relationship between the state and action of each agent, generating a cooperative strategy model that includes task allocation and timing control. The spatial pose information and real-time position data of the workpiece are collected by a 3D vision perception system and position sensor. After being synchronously fused with the state information of each intelligent agent, the data is input into the collaborative strategy model for reasoning and correction, and the task state data is updated. Based on the task status data, control each loading and unloading robot to perform loading and unloading operations, collect execution feedback data to calculate task completion and cycle deviation, input the cycle deviation information into the collaborative strategy model to update parameters and adjust task scheduling, and output the updated task scheduling results. Based on historical cycle time data and current load status, the chaotic frog-jump optimization method is used to dynamically calculate and adjust the cycle time matching relationship of each workstation, and generate adaptive cycle time control instructions. Based on the updated task scheduling results and adaptive cycle control instructions, the execution sequence of each workstation and loading / unloading robots is jointly optimized to generate comprehensive collaborative control instructions and drive the production line to operate adaptively and collaboratively.

[0022] In this embodiment, the operating data includes robot pose information, conveyor line speed information, fixture status information, material box position information, and workpiece identification information. The preprocessing includes time synchronization, noise reduction, normalization processing, removal of abnormal and missing data, and standardization of coordinates and format.

[0023] In this embodiment, the generation of the collaborative strategy model includes the following specific steps: The standardized operational dataset is divided into a state data subset, an action data subset, and a reward data subset, and input into a multi-agent collaborative control architecture. The state data subset includes the position, posture, speed, and workstation state parameters of each loading / unloading robot. The action data subset includes the execution instructions for each robot to grasp, transport, place, and adjust the cycle time. The reward data subset includes numerical indicators of task completion, energy consumption, and cycle time deviation. The multi-agent collaborative control architecture consists of multiple independent agents, each agent corresponding to one loading / unloading robot or one workstation node. Each agent is a control unit with independent perception, decision-making, and execution functions. Each agent includes a perception module for perceiving the operational status, a policy network for action decision-making, a value network for revenue estimation, and an interface module for communication and interaction. In a multi-agent cooperative control architecture, an independent policy network and value network are established for each agent. The policy network is used to predict the probability distribution of actions based on the state input, and the value network is used to estimate the expected cumulative revenue in the current state. The establishment of the policy network and value network for each agent includes: initializing the input layer nodes based on a subset of state data, normalizing the input state data and generating a state feature vector, setting up a multi-layer fully connected structure and a non-linear activation function in the intermediate layer to extract state features, inputting the extracted state features into the policy branch and the value branch respectively, calculating the probability distribution of each selectable action in the policy branch, calculating the cumulative reward estimate of the current state in the value branch, jointly optimizing the outputs of the policy branch and the value branch and updating the network parameters, and completing the initialization of the policy network and value network within each agent; An improved IPPO method is used to train the policy network of each agent. In each iteration, the expected return difference is calculated based on the actual interaction data. A loss function is constructed and a pruning constraint is added to limit the update magnitude. The improved IPPO method, based on the original policy optimization framework, introduces a dream-like self-training mechanism to expand the training samples, introduces a neural plasticity adjustment mechanism to adjust the change relationship of neuron connection weights in the policy network, and combines a joint update structure of shared feature layer and local decision layer to perform parameter synchronization and feature fusion. The actual interaction data includes the next state parameters and termination flags of each agent during the operation. The next state parameters are the updated state information after the action is executed, and the termination flag is used to indicate the task completion or abnormal interruption state. The mathematical expression for the loss function is: ; in, Indicates the policy network in parameters The overall loss function under, Indicates time step The expected operation on the upsampled data, This represents the ratio of the probability of actions taken under the old and new strategies in the same state. Represents the dominance function. This represents the untrunculated policy gradient term. This indicates that the policy update range will be limited to a certain range. Inside, This represents the policy gradient term after pruning constraints. Update the constraint coefficients for the strategy. These are the weighting coefficients for the error term of the value function. Representing the value network in relation to state The cumulative revenue estimate, This represents the actual cumulative revenue calculated based on actual interaction data. Here are the weighting coefficients for the entropy regularization term. This indicates that the agent is at time step The execution action, Indicates at time step The environmental state of the intelligent agent at all times. Indicates the policy network in parameters Below, in state Choose action The probability distribution, The entropy value represents the strategy; A dream-like self-training mechanism is introduced during the training phase. A potential environment model is generated based on historical interaction data. Virtual policy replay is performed in the potential environment model to generate an extended sample set and incorporate it into the training dataset. The extended sample set is used to improve the generalization ability of the policy under different production conditions. The generation of the potential environment model includes: extracting state transition samples based on historical interaction data, constructing an interaction dataset composed of state, action, and reward triples, standardizing and feature-encoding the state and action information, establishing state transition relationships to generate corresponding next state data, calculating and iteratively correcting the deviation between the generated state data and historical samples, and outputting a potential environment model for simulating the state evolution and interactive feedback relationship of the production line. The generation of the extended sample set includes: randomly selecting initial state samples from the latent environment model, generating virtual action sequences based on the action distribution of each agent's current policy network, inputting the virtual actions into the latent environment model to calculate the corresponding next state and reward value, forming a virtual interaction sample consisting of state, action, reward, next state and termination flag, standardizing and feature-aligning the virtual samples, and merging the generated virtual interaction samples with the actual interaction data to generate the extended sample set. Based on the updated training dataset, a neural plasticity adjustment mechanism is introduced during the parameter update phase. The change range of neuron connection weights in the policy network is adjusted according to the correlation between action and reward. The weight parameters are updated according to the inverse relationship between action reward ratio and error term. The neural plasticity adjustment mechanism includes calculating the initial gradient of connection weights based on the input-output relationship of each neuron in the policy network, normalizing the gradient and generating a plasticity adjustment factor by combining the correlation coefficient between action reward and error term. The plasticity adjustment factor is applied to the weight update formula to correct and iteratively update the weight parameters of each layer of the policy network layer by layer. At the same time, the parameter vector of the corresponding layer of the value network is updated simultaneously to obtain the updated policy network parameter and value network parameter set. Through multiple rounds of iterative training, the policy network parameters and value network parameters of each agent are integrated, and the shared feature layer and local decision layer are jointly updated under the framework of the cooperative policy model to generate a cooperative policy model that includes task allocation and timing control. The joint update includes: inputting the policy network parameters and value network parameters output by each agent into the shared feature layer of the cooperative policy model; weighting and fusing the parameters of each agent to generate a global feature representation; passing the global feature representation to the local decision layer corresponding to each agent; calculating local gradient information based on the cumulative revenue estimate of the value network of each agent; synchronously backpropagating and updating the weight parameters of the shared feature layer and the local decision layer; completing the parameter alignment and feature consistency adjustment among the agents; and generating the cooperative policy model.

[0024] In this embodiment, the update of the task status data includes: collecting workpiece spatial pose data output by the 3D vision perception system and real-time position data collected by the position sensor; performing attitude calculation and coordinate system calibration on the three-dimensional coordinate point cloud of the workpiece to obtain the attitude matrix and displacement vector of the workpiece in the global coordinate system of the production line; matching and synchronizing the attitude matrix and displacement vector with the pose parameters in the state information of each agent using timestamps; performing feature encoding and normalization processing on the synchronized data to generate a fused state feature vector; inputting the fused state feature vector into the state update module of the collaborative strategy model; performing inference calculation on the execution state of the current task stage to obtain the predicted action output and state evaluation results of each agent; calculating the state deviation based on the predicted action output and execution feedback data; weighting and correcting the state deviation with the original state parameters to generate an updated state vector; performing normalization processing and index encoding; and outputting task status data including workstation running status, loading and unloading operation progress parameters, and cycle deviation information. The workpiece spatial pose data includes the three-dimensional coordinate position, attitude matrix, displacement vector, and timestamp information of the workpiece in the global coordinate system of the production line.

[0025] In this embodiment, the loading and unloading operations include workpiece gripping, handling, and placement processes. The output of the updated task scheduling result includes: collecting feedback data from each loading and unloading robot during the execution process; organizing the feedback data into a time series and labeling it with status tags; constructing an execution feedback set including task completion, cycle deviation, and energy consumption information; comparing the execution feedback set with the predicted action sequence output by the collaborative strategy model; calculating the task execution deviation vector; performing weighted decomposition on the deviation vector and generating a task execution error matrix; inputting the task execution error matrix into the parameter update module of the collaborative strategy model; correcting the parameter vectors of each agent based on the cycle deviation information and task completion information in the error matrix; normalizing and fusing the corrected parameter vectors to form a new task scheduling parameter set; synchronously mapping the task scheduling parameter set to each agent node in the multi-agent collaborative control architecture; and outputting the updated task scheduling result. The task scheduling result includes the work order, task allocation weight, and cycle adjustment parameters of each workstation.

[0026] In this embodiment, the generation of the adaptive beat control command includes the following specific steps: Collect historical beat data and current load status data, perform time series smoothing and normalization on the beat data, and generate beat feature sequences; Initialize the chaotic frog jumping optimization method population, set the population size, maximum number of iterations and chaos factor, and randomly initialize the population individuals according to the beat feature sequence and divide them into several subpopulations; In each iteration, individuals within each subgroup are sorted according to the fitness function, the best and worst individuals are selected, and the beat adjustment parameters of the worst individual are updated based on the fitness difference between the two and the chaotic mapping function. The locally optimized beat adjustment parameters are then output. The fitness function is: ; in, Represents the global fitness function. This indicates the number of workstations participating in the cycle time matching optimization. Indicates the first Workstation cycle time deviation, Indicates the first Workstation load imbalance This is a trade-off factor between cycle time and load. During the clock parameter update process, the chaotic mapping function calculates chaotic variables based on the number of iterations, a preset chaotic factor, and fitness differences. It generates a chaotic sequence through nonlinear transformation and modulo operation, forming a uniformly distributed random perturbation value between zero and one. The perturbation value is applied to the clock adjustment parameter of the worst individual, and the parameter is subjected to nonlinear perturbation processing. Constraint correction is performed according to the clock parameter boundary threshold, and the updated clock adjustment parameter is output. Based on the locally optimized beat adjustment parameters, cross-group chaotic search is performed on individuals in each subgroup. The beat parameters are perturbed by chaotic variables and the global optimal solution is updated to obtain the optimized beat matching relationship. The optimized beat matching relationship is obtained by: summarizing the beat adjustment parameter sets output by the local optimization of each subgroup; performing cross-group chaotic search on the beat parameters of different subgroups; randomly exchanging and differentially calculating the chaotic sequence generated by the chaotic mapping function among the parameters of each subgroup to form a global beat parameter set; calculating the global fitness value of the global beat parameter set according to the fitness function and sorting it to determine the global optimal individual and the reference individual; generating a new chaotic sequence according to the chaotic mapping function; performing nonlinear perturbation and correction on the beat parameters of the reference individual to generate an updated beat parameter set; calculating the comprehensive error value of the beat deviation and load imbalance of each workstation on the updated beat parameter set; normalizing and weighting the error value; constructing a beat matching parameter matrix and performing boundary constraints and convergence detection; and outputting the optimized beat matching relationship including the beat correspondence and synchronization coefficient of each workstation. Based on the optimized beat matching relationship, parameter calculations are performed to generate adaptive beat control instructions. The beat control instructions include the beat adjustment coefficient, synchronization delay parameter, and task coordination cycle for each workstation. The parameter calculations include: normalizing and aligning the optimized beat matching relationship with the time series; calculating the beat difference between each workstation and generating the beat adjustment coefficient; statistically analyzing the beat offset of adjacent workstations and calculating the delay estimate to generate the synchronization delay parameter; weightedly fusing the beat adjustment coefficient and the synchronization delay parameter to calculate the overall task coordination cycle; performing smoothing and correction processing on the fused beat data; and outputting the adaptive beat control instructions.

[0027] In this embodiment, the generation of the comprehensive collaborative control command includes: aligning and normalizing the updated task scheduling results with the adaptive beat control command; fusing features of the task scheduling parameters, beat adjustment coefficients, and synchronization delay parameters of each workstation; constructing a joint optimization input matrix including execution order, beat matching relationship, and inter-workstation dependency relationship; dynamically calculating and adjusting the execution order of the loading and unloading robots based on the joint optimization input matrix; calculating coordination weights based on the workstation load balance and task completion time; weighted summing of the coordination weights and task scheduling parameters to generate a comprehensive control vector; performing inverse normalization and parameter mapping on the comprehensive control vector; and outputting a comprehensive collaborative control command including task allocation instructions for each workstation, execution order instructions for loading and unloading robots, and beat synchronization parameters.

[0028] Example 1: To verify the feasibility of this invention in practice, it was applied to the automated control scenario of loading and unloading parts in an automotive parts production line. This production line includes multiple processing stations and multiple loading and unloading robots. The transfer of workpieces between stations requires maintaining a stable cycle time and coordinated operation. Traditional production methods rely on fixed timing sequences and manual settings, which cannot cope with load changes and cycle time fluctuations, leading to waiting and congestion at some stations, resulting in low production efficiency and coordination. The multi-agent collaborative production line loading and unloading control system proposed in this invention achieves adaptive collaborative control in a multi-station, multi-robot environment through intelligent strategy modeling and optimization.

[0029] In the application, the production data acquisition module acquires operational data from each workstation and loading / unloading robot on the production line, including pose information, conveyor speed, fixture status, bin position, and workpiece identification information. This data is then processed for time synchronization, noise reduction, and normalization to generate a standardized operational dataset. Subsequently, the dataset is input into a multi-agent collaborative control architecture, and an improved IPPO method is used to train the mapping relationship between the state and action of each agent. Each agent corresponds to a workstation or robot and possesses independent perception and decision-making capabilities. The system introduces a dream-like self-training mechanism to generate a latent environment model, replaying the strategy in a virtual environment to expand the sample and improve the generalization ability of the strategy under various working conditions. Simultaneously, a neural plasticity adjustment mechanism is used to dynamically adjust the policy network weights, enabling the strategy to quickly adapt to changes in cycle time.

[0030] During production, the status update module uses a 3D vision perception system and position sensors to collect workpiece spatial pose data, which is then fused with the status information of each intelligent agent and input into the collaborative strategy model to update the task status data in real time. The task scheduling module controls the robot to perform loading and unloading operations based on the updated status data, collects feedback data to calculate task completion rate and cycle time deviation, inputs the results into the model for parameter updates, and outputs optimized task scheduling results. The system can dynamically adjust the sequence of actions based on real-time load and execution feedback, maintaining coordination and consistency between workstations.

[0031] When cycle time becomes unbalanced or workstation load changes, the cycle time optimization module activates a chaotic leapfrog optimization algorithm to perform global dynamic calculation and correction of the cycle time matching relationship between each workstation. It optimizes the cycle time parameters through chaotic mapping and cross-group search, generating adaptive cycle time control instructions that include cycle time adjustment coefficients, synchronization delay parameters, and task coordination cycles. The collaborative control module combines task scheduling results with the cycle time control instructions to jointly optimize the execution sequence of multiple robots and the relationships between workstations, generating comprehensive collaborative control instructions to drive the production line operation.

[0032] During continuous operation, the system performs online optimization by collecting real-time task completion and cycle time deviation information, gradually improving the coordination accuracy of task scheduling and cycle time matching. Results show that the production line can maintain a stable cycle time under load fluctuations and workpiece switching conditions, with smooth transitions between robot operations and significantly improved workstation utilization and collaboration efficiency. This invention's system achieves adaptive cycle time control and dynamic task allocation in complex multi-workstation environments, demonstrating high efficiency, stability, and flexibility, thus validating its application value in intelligent manufacturing scenarios.

[0033] Table 1. Performance Comparison of the Invention and Traditional Production Line Loading / Unloading Control Methods

[0034] As can be clearly seen from Table 1, the method of the present invention is superior to the traditional method in many indicators.

[0035] Regarding cycle time control accuracy, the average cycle time deviation of traditional methods is 0.85 seconds, while that of the system of this invention is only 0.32 seconds, a reduction of 62.4%. This is attributed to the chaotic frog-jump optimization algorithm introduced into the system, which dynamically corrects cycle time parameters through cross-group chaotic search and nonlinear perturbation mechanisms, making the cycle time between workstations more coordinated and consistent. The corresponding cycle time synchronization error also decreased from 5.3% to 1.7%, further demonstrating that the system can achieve higher time synchronization accuracy under multi-workstation parallel operation.

[0036] Secondly, in terms of resource utilization and task scheduling, the traditional system's workstation utilization rate is 82.6%, while the system of this invention improves it to 94.8%, an increase of 14.8%. Simultaneously, the task scheduling response time is reduced from 1.28 seconds to 0.53 seconds, an optimization of 58.6%. This is mainly due to the improved IPPO algorithm used in this invention for multi-agent joint policy updates. Each agent can adjust its action allocation in real time according to state changes, thereby reducing scheduling latency and improving decision-making speed.

[0037] Regarding robot operating efficiency, the average waiting time of the robot in this invention system was reduced from 3.6 seconds to 1.1 seconds, a decrease of 69.4%, and the load balance was improved from 84.1% to 96.3%. This is because the system achieves dynamic balance in task allocation among multiple robots through a joint update mechanism of shared feature layer and local decision layer, making the load of each workstation and robot more balanced and avoiding local overload or idleness.

[0038] Furthermore, the advantages of this invention are also very significant in terms of system robustness and energy efficiency. Fault recovery time was reduced from 7.4 seconds to 3.2 seconds, a reduction of 56.8%, indicating that the system can quickly recover to a cooperative state after an operational anomaly. Simultaneously, average energy consumption decreased from 1.46 to 1.09, saving approximately 25.3% of energy per cycle. This is because the system incorporates an energy consumption weight term in the reward function during action planning, enabling the control strategy to automatically tend towards energy-optimal performance while optimizing task efficiency.

[0039] In terms of communication performance, the system of this invention reduces data communication latency from 78 milliseconds to 29 milliseconds, a decrease of 62.8%, through a data sharing and unified synchronization mechanism among intelligent agents, significantly improving the real-time collaboration capability among multiple nodes. Considering all indicators, the overall system productivity increased from 87.5% to 97.2%, an increase of 11.1%, achieving stable, efficient, and low-energy-consumption collaborative control.

[0040] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A production line loading and unloading control system based on multi-agent cooperation, characterized in that, The application relates to a production line adaptive collaborative control system and method. The production data acquisition module is used for acquiring and preprocessing the running data of various workstations and multiple piece loading and unloading robots on a production line; The multi-agent collaborative control module is used for inputting the standardized running data set into a multi-agent collaborative control architecture, training the mapping relationship between the state and action of each agent by using an improved IPPO method, and generating a collaborative strategy model; The state updating module is used for acquiring the spatial pose information of a workpiece by a three-dimensional visual perception system and a position sensor, fusing the state information of the agent, inputting the fused information into the collaborative strategy model, and updating the task state data; The task scheduling module is used for controlling the piece loading and unloading robots to perform piece loading and unloading work according to the task state data, acquiring feedback data to calculate the task completion degree and the beat deviation, and updating the task scheduling result; The beat optimization module is used for calculating and adjusting the beat matching relationship of various workstations based on historical beat data and the current load state by using a chaotic frog leap optimization method, and generating adaptive beat control instructions; The collaborative control module is used for jointly optimizing the execution sequence of various workstations and piece loading and unloading robots according to the task scheduling result and the beat control instructions, generating comprehensive collaborative control instructions, and driving the adaptive collaborative operation of the production line.

2. The production line loading and unloading control system based on multi-agent cooperation according to claim 1, characterized in that, The modules are realized by the following methods: The running data of various workstations and multiple piece loading and unloading robots on a production line are acquired and preprocessed to generate a standardized running data set; The standardized running data set is input into a multi-agent collaborative control architecture, the mapping relationship between the state and action of each agent is trained by using an improved IPPO method, and a collaborative strategy model including piece loading and unloading task allocation and beat control is generated; The spatial pose information and real-time position data of a workpiece are acquired by a three-dimensional visual perception system and a position sensor, and are synchronously fused with the state information of each agent to input the fused information into the collaborative strategy model for reasoning and correction, and the task state data is updated; The piece loading and unloading robots are controlled to perform piece loading and unloading work according to the task state data, the execution feedback data is acquired to calculate the task completion degree and the beat deviation, the beat deviation information is input into the collaborative strategy model for parameter updating and task scheduling adjustment, and the updated task scheduling result is output; Based on historical beat data and the current load state, the beat matching relationship of various workstations is dynamically calculated and adjusted by using a chaotic frog leap optimization method, and adaptive beat control instructions are generated; According to the updated task scheduling result and the adaptive beat control instructions, the execution sequence of various workstations and piece loading and unloading robots is jointly optimized, comprehensive collaborative control instructions are generated, and the adaptive collaborative operation of the production line is driven.

3. The production line loading and unloading control system based on multi-agent cooperation according to claim 2, characterized in that, The running data includes robot pose information, conveying line speed information, hanger state information, magazine position information and workpiece identification information, and the preprocessing includes time synchronization, denoising, normalization processing, rejection of abnormal and missing data and unification of coordinates and formats.

4. The production line loading and unloading control system based on multi-agent cooperation according to claim 2, characterized in that, The generation of the collaborative strategy model includes the following specific steps: The standardized operation data set is divided into state data subset, action data subset and reward data subset, and is input into a multi-agent collaborative control architecture composed of multiple independent agents, each agent corresponding to a piece feeding and unloading robot or a station node, the agent being a control unit with independent perception, decision and execution functions, each agent including a perception module, a policy network, a value network and an interface module; In a multi-agent cooperative control architecture, an independent policy network and value network are established for each agent ； The policy network of each agent is trained by using an improved IPPO method, the expected return difference is calculated according to the actual interaction data in each iteration, a loss function is constructed and a clipping constraint is added to limit the update amplitude, the improved IPPO method introduces a dream-like self-training mechanism to expand the training samples, introduces a neural plasticity adjustment mechanism to adjust the change relationship of the neuron connection weights in the policy network, and combines the joint update structure of the shared feature layer and the local decision layer for parameter synchronization and feature fusion; The mathematical expression of the loss function is: ； wherein, denotes the overall loss function of the policy network under parameters denotes the expected operation on the up-sampled data at time step denotes the ratio of action probabilities of the new and old policies at the same state, denotes the advantage function, denotes the un-truncated policy gradient term, denotes the policy update magnitude is limited within the interval denotes the policy gradient term after the clipping constraint, is the policy update constraint coefficient, is the weight coefficient of the value function error term, denotes the cumulative reward estimate of the value network for state denotes the real cumulative reward calculated by the actual interaction data, is the weight coefficient of the entropy regularization term, denotes the executed action of the agent at time step denotes the environment state of the agent at time step denotes the probability distribution of the policy network under parameters selecting action at state denotes the entropy value of the policy;​​​​​​​ In the training phase, the dream-like self-training mechanism is introduced, a potential environment model is generated based on the historical interaction data, virtual policy playback is performed in the potential environment model, an extended sample set is generated and included in the training data set; Based on the updated training data set, the neural plasticity adjustment mechanism is introduced in the parameter update phase, the change amplitude of the neuron connection weights in the policy network is adjusted according to the correlation between the action and the reward, the weight parameters are updated according to the action return proportion and the inverse relationship of the error term, the neural plasticity adjustment mechanism includes calculating the initial gradient of the connection weights according to the input and output relationship of each neuron in the policy network, normalizing the gradient and combining the correlation coefficient of the action return and the error term to generate a plasticity adjustment factor, applying the plasticity adjustment factor to the weight update formula, and correcting and iteratively updating the weight parameters of each layer of the policy network, while synchronously updating the parameter vectors of the corresponding layers of the value network, to obtain the updated policy network parameter and value network parameter set; Through multiple iterations of training, the policy network parameters and value network parameters of each agent are integrated, the shared feature layer and the local decision layer are jointly updated under the collaborative strategy model framework, and a collaborative strategy model including piece feeding and unloading task allocation and beat control is generated.

5. The production line loading and unloading control system based on multi-agent cooperation according to claim 2, characterized in that, The updating of the task state data comprises: collecting workpiece spatial pose data output by a 3D visual perception system and real-time position data collected by a position sensor, performing pose solving and coordinate system calibration on a three-dimensional coordinate point cloud of the workpiece to obtain a pose matrix and a displacement vector of the workpiece in a global coordinate system of the production line, time stamp matching and synchronization of the pose matrix and the displacement vector with pose parameters in state information of each agent, feature encoding and normalization processing of the synchronized data to generate a fusion state feature vector, inputting the fusion state feature vector into a state updating module of the cooperative strategy model to perform inference calculation on an execution state of a current task stage to obtain predicted action outputs and state evaluation results of each agent, calculating a state deviation from the predicted action outputs and execution feedback data, generating an updated state vector by weighted correction of the state deviation and original state parameters, and performing normalization processing and index encoding to output task state data including work station running states, workpiece loading and unloading operation progress parameters and beat deviation information.

6. The production line loading and unloading control system based on multi-agent cooperation according to claim 2, characterized in that, The output of the updated task scheduling result comprises: collecting feedback data of each workpiece loading and unloading robot during execution, time series arrangement and state label annotation of the feedback data, constructing an execution feedback set including task completion degree, beat deviation and energy consumption information, comparing the execution feedback set with the predicted action sequence output by the cooperative strategy model to calculate a task execution deviation vector, performing weighted decomposition on the deviation vector to generate a task execution error matrix, inputting the task execution error matrix into a parameter updating module of the cooperative strategy model, correcting parameter vectors of each agent according to beat deviation information and task completion degree information in the error matrix, performing normalization and fusion calculation on the corrected parameter vectors to form a new task scheduling parameter set, synchronously mapping the task scheduling parameter set to each agent node in the multi-agent cooperative control architecture to output the updated task scheduling result.

7. The production line loading and unloading control system based on multi-agent cooperation according to claim 2, characterized in that, The generation of the adaptive beat control instruction comprises the following specific steps: Collecting historical beat data and current load state data, performing time series smoothing and normalization processing on the beat data to generate a beat feature sequence; Initializing a population of a chaotic frog leap optimization method, setting a population size, a maximum number of iterations and a chaotic factor, randomly initializing population individuals according to the beat feature sequence and dividing them into several subpopulations; In each iteration, sorting individuals in each subpopulation according to an adaptive function, selecting an optimal individual and a worst individual, updating beat adjustment parameters of the worst individual according to the difference in fitness between the two individuals and a chaotic mapping function, and outputting the locally optimized beat adjustment parameters; The adaptive function is: ； wherein, represents a global fitness function, represents the number of stations involved in the beat matching optimization, represents the beat deviation of the i-th station, represents the load imbalance of the i-th station, represents the load imbalance of the i-th station, represents the load imbalance of the i-th station, is a weighting coefficient between beat and load; Based on the locally optimized beat adjustment parameters, performing cross-group chaotic search on individuals of each subpopulation, disturbing beat parameters by using chaotic variables and updating a global optimal solution to obtain an optimized beat matching relationship; Performing parameter operation according to the optimized beat matching relationship to generate an adaptive beat control instruction, wherein the beat control instruction comprises beat adjustment coefficients, synchronization delay parameters and task coordination periods of each work station.

8. The production line loading and unloading control system based on multi-agent cooperation according to claim 2, characterized in that, The generation of the comprehensive cooperative control instruction comprises: data alignment and normalization processing of the updated task scheduling result and the adaptive beat control instruction, feature fusion of the task scheduling parameters, the beat adjustment coefficient and the synchronization delay parameters of each station, construction of a joint optimization input matrix comprising an execution sequence, a beat matching relationship and a dependence relationship between stations, dynamic calculation and arrangement adjustment of the upper and lower piece robot execution sequence based on the joint optimization input matrix, calculation of a coordination weight according to a station load balancing degree and a task completion time, weighted summation of the coordination weight and the task scheduling parameters to generate a comprehensive control vector, reverse normalization and parameter mapping processing of the comprehensive control vector, and output of a comprehensive cooperative control instruction comprising a task allocation instruction of each station, an upper and lower piece robot execution sequence instruction and a beat synchronization parameter.

Images