A Collaborative Control Method and System for Human-Robot Collaborative Assembly Robots Based on Digital Twins

Abstract

This invention discloses a collaborative control method and system for human-robot collaborative assembly robots based on digital twins. The method first constructs a digital twin simulation environment in the Unity engine to map the physical collaborative assembly scene to the virtual scene. Multimodal sensing technology is used to collect real-time data on human actions and the state of the assembled workpiece, introducing virtual-real mapping to achieve intent recognition. Then, based on reinforcement learning algorithms, path planning is performed on the collaborative robot in the digital twin environment to generate an optimal collaborative path sequence. Finally, the planned path is mapped to the real robot control commands through coordinate transformation and inverse kinematics algorithms, and an online adaptive strategy is used to dynamically optimize the motion trajectory. By combining digital twins and reinforcement learning, this invention improves the intelligence and flexible scheduling capabilities of the human-robot collaborative assembly system, enhances the system's rapid adaptability to unstructured changes and assembly accuracy, and overcomes the problems of cumbersome task programming and poor adaptability in traditional methods.

Description

Technical Field

[0001] This invention belongs to the field of intelligent manufacturing and human-machine collaborative control technology, specifically relating to a collaborative control method and system for human-machine collaborative assembly robots based on digital twins. Background Technology

[0002] With the rapid development of intelligent manufacturing and smart factories, complex assembly processes are gradually evolving from the traditional "manual operation + fixed tooling" to "human-robot collaboration + flexible units." In this process, collaborative robots are being widely introduced into assembly lines, aiming to improve both efficiency and quality by having robots handle repetitive, high-precision, or high-load operations, while human workers handle decision-making and flexible tasks. However, most existing human-robot collaborative assembly robot systems are still based on traditional industrial robot control architectures, heavily relying on pre-programmed offline fixed paths and static control rules, resulting in weak adaptability to dynamic changes on-site.

[0003] Specifically, traditional assembly robots typically generate trajectories based on teach-and-playback or offline programming, with assembly sequence, motion paths, and safety zones fixed during the planning phase. However, if changes occur on-site such as shifts in operator rhythm, temporary component replacements, tooling misalignment, or adjustments to assembly cycle time, engineers often need to replan the path and debug the program. This reconfiguration process is lengthy, labor-intensive, and makes it difficult to respond promptly to production changes. Furthermore, the assembly site contains a large amount of unstructured information, such as operator gestures, verbal commands, and intentions for temporary adjustments. Traditional control systems struggle to effectively perceive and understand this information, preventing the robot from forming a truly collaborative relationship with the operator; instead, it often simply performs tasks in parallel with the robot.

[0004] In recent years, digital twin technology has provided new approaches for modeling, simulating, and optimizing complex manufacturing systems. By constructing a digital twin in virtual space that is highly consistent with the physical assembly line, visualization, predictability, and traceability of equipment status, workpiece posture, and process flow can be achieved, supporting process planning and operation and maintenance decisions. However, existing applications are mostly limited to production line-level or equipment-level status monitoring and process verification. The coupling between the digital twin model and human-machine collaborative control strategies is weak, and a closed-loop online decision-making system that integrates the virtual and physical worlds has not yet been formed. Especially in collaborative assembly scenarios, how to utilize the digital twin environment to achieve unified modeling of people, machines, materials, and methods, and how to map the planning results in virtual space to physical robot control in real time, remains a challenge in existing technologies.

[0005] On the other hand, collaborative robot path planning still generally adopts static schemes based on CAD models or predefined rules, simply overlaying obstacle avoidance or deceleration logic within a limited range, lacking online learning and adaptive optimization capabilities. When the positions of surrounding personnel change, new obstacles appear in the environment, or the assembly target shifts slightly, the robot often can only cope with conservative safety distances and stopping strategies, affecting production cycle time and failing to fully leverage the flexibility of collaborative robots. Some studies have attempted to introduce optimization algorithms or classical path planning methods, but most are offline calculations, making it difficult to meet the requirements of real-time performance and continuous human-robot interaction.

[0006] In summary, existing human-robot collaborative assembly robot systems suffer from problems such as slow response, poor trajectory adaptability, and insufficient safety and collaboration capabilities. There is a lack of a unified framework that tightly couples digital twin technology with online path planning, making it impossible to fully utilize environmental and human-robot interaction information for intelligent decision-making in virtual space and to quickly and safely map the results to the physical robot execution layer. Summary of the Invention

[0007] To overcome the aforementioned technical deficiencies, this invention proposes a dynamic collaborative control method and system for human-robot collaborative assembly robots based on digital twins. A digital twin model of the assembly scene is constructed in a 3D simulation platform such as Unity. Combined with path planning and control algorithms, real-time recognition of human-robot collaborative intentions and adaptive adjustment of motion trajectories are achieved, fundamentally improving the efficiency, accuracy, and safety of human-robot collaboration in complex assembly scenarios. The technical solution is as follows:

[0008] A dynamic collaborative control method for human-machine collaborative assembly robots based on digital twins includes the following steps:

[0009] S100: Establish a digital twin environment in Unity, create digital twin models of the collaborative robot to be controlled and the assembly scene, calibrate the coordinate system of the virtual and physical spaces through sensors, and establish a two-way real-time communication channel between Unity and the physical robot controller to synchronize status information;

[0010] S200: In the digital twin environment, perform environmental perception, and based on sensor data such as vision and force, identify the pose and state of assembled parts, tools and collaborators, and generate the current environmental state of the system;

[0011] S300: Based on the state information and assembly task objectives in the digital twin environment, the SAC algorithm is used to optimize the path planning strategy. The reward function is constructed with the assembly task accuracy and safety as constraints. The strategy network and double Q-value network based on deep neural network are trained to output the robot cooperative path action sequence that meets the constraints.

[0012] S400: Map the virtual path obtained in step S300 to the real robot control system and generate robot execution code; during physical execution, adaptively adjust the trajectory based on real-time sensor feedback to achieve dynamic collaborative operation of the robot in the physical assembly environment. The adaptive adjustment involves: detecting anomalies based on real-time sensor feedback, triggering an online adjustment mechanism, replanning the remaining path using the SAC policy network, and generating compensation instructions to dynamically update the robot trajectory.

[0013] As a preferred option, in step S100, the coordinate systems of the virtual environment and the actual assembly environment are unified by using sensors and calibration technology. Based on this, the virtual simulation model is accurately docked with the real world in space, and the 3D model of the real robot and the motion capture data of the operator are imported to make the movements of the robot and the personnel in the virtual scene consistent with the physical entities. At the same time, a two-way data channel is established between Unity and the physical robot based on network protocols such as TCP / IP or ROS to realize the real-time transmission of robot motion data and control commands, thereby providing a reliable virtual-real synchronization foundation for subsequent path planning and control.

[0014] Preferably, step S200 includes: using an RGB camera, a depth camera, and an inertial measurement unit (IMU) to collect the operator's visual images, depth information, and skeletal key point data, respectively; performing noise reduction, filtering, and standardization preprocessing on the collected multi-source sensor data, and generating unified and accurate environmental perception information through a sensor fusion algorithm; using computer vision and deep learning algorithms to identify the pose of target objects in the scene, and combining action semantic information to identify the personnel's assembly intention and part status in the digital twin model.

[0015] Preferably, step S300 includes:

[0016] S310: Establish the path planning state space and motion space in the digital twin environment, and define the robot's current pose. Target pose and the distribution of environmental obstacles As input, through the state mapping function Constructing state vectors The robot's linear velocity, angular velocity, and / or joint increments are used to construct a continuous motion vector. This forms a continuous state-action description for path planning;

[0017] S320: Collecting samples in a digital twin environment ,in , Let be the state vectors at time k and time k+1. , , Let k be the action vector, immediate reward value, and termination flag; utilize the maximum entropy reinforcement learning objective function, and iteratively update the policy network parameters η and the double soft Q-value network parameters using gradient descent. And the temperature parameter β, to maximize the cumulative reward and maintain the policy entropy, finally obtain the optimized path planning strategy.

[0018] Preferably, the dual-value network in step S320 employs a soft Q-function update mechanism, updating each sample... Constructing the soft Q objective value and value network loss function Target value The definition is as follows:

[0019]

[0020] The loss function is defined as follows:

[0021]

[0022] In the above formula, At time step k, the agent performs an action. The immediate rewards obtained from the environment afterwards and For a two-way softQ value network, and For the corresponding target network, This is the discount factor. For temperature coefficient, For the end-of-round indicator, In the state The following actions are sampled from the current policy network. The negative logarithmic probability of the action. For parameters The policy network; by adjusting the parameters Perform gradient descent to minimize the loss function. Complete the update of the SoftQ value network.

[0023] Preferably, the updating of the strategy network and temperature parameters in step S320 includes:

[0024] Policy network loss function Defined as:

[0025]

[0026] In the above formula, In the state The actions obtained by sampling from the policy network are as follows. The negative logarithmic probability of the action. and This is a dual-soft-Q value network, achieved by adjusting the policy network parameters. Perform gradient descent minimization This is to improve the expected value of path planning while ensuring the strategy entropy;

[0027] Temperature parameters are in logarithmic form. The objective function for adaptive updates is defined as follows:

[0028]

[0029] In the above formula, To preset the target entropy level, The negative logarithmic probability of the action is determined by... Perform gradient descent minimization , and by The updated temperature coefficient is obtained, thereby enabling adaptive balancing of the exploration level and path quality of path planning in different collaborative assembly scenarios.

[0030] Preferably, step S400 includes: converting the three-dimensional coordinates in the digital twin environment into spatial coordinates in the base coordinate system of the real robot using known calibration parameters; solving the rotation angle corresponding to each joint of the robot using an inverse kinematics algorithm; inputting the angle parameters into the robot control code sub-function library to generate control instructions containing joint spatial motion and Cartesian spatial trajectory, wherein the control instructions support force control and impedance control modes.

[0031] Preferably, step S400 is as follows:

[0032] S410: Converts the motion trajectory point cloud generated in the digital twin simulation environment into a three-dimensional motion trajectory in the real coordinate system, and solves the joint space motion sequence through the forward and inverse kinematics algorithms of the robot model;

[0033] S420: The solved joint angles are sequentially called to generate execution code from the sub-functions in the robot control code library, including joint space motion sub-functions, Cartesian space linear interpolation sub-functions, and force control sub-functions. The corresponding sub-functions are automatically matched according to the assembly task category, and parameters such as joint angles, speeds, and safety constraints are brought in to realize the generation of complete motion control instructions.

[0034] Preferably, in step S400, the adaptive adjustment is as follows:

[0035] During robot movement, when force sensors or visual feedback detect part offset or collision anomalies, an online adjustment mechanism is triggered. The current robot state, environmental state, and remaining task objectives are input into the trained SAC policy network to recalculate the optimal or suboptimal remaining path sequence from the current state to the task objective. Based on the replanning results, the required joint angles or end-effector pose compensation amounts are calculated, and the control commands sent to the physical robot are updated in real time to correct the motion trajectory.

[0036] This invention also discloses a dynamic collaborative control system for a human-machine collaborative assembly robot based on digital twins, used to implement the above-mentioned dynamic collaborative control method for human-machine collaborative assembly robots. The system includes:

[0037] The digital twin simulation environment module, built on the Unity engine, is used to establish and maintain a virtual model and virtual-real mapping relationship consistent with the physical assembly scene;

[0038] The sensor module is used to collect multimodal data on human actions, assembly parts status, and robot status in the physical environment.

[0039] The path planning module runs the SAC reinforcement learning algorithm, receives the status information from the digital twin simulation environment module, performs online path planning and strategy optimization, and generates the motion path of the collaborative robot.

[0040] The robot control module is used to convert the virtual path generated by the path planning module into joint control commands for the physical robot through coordinate transformation and inverse kinematics calculation; and to perform online trajectory replanning and compensation when receiving abnormal feedback.

[0041] The communication module is used to establish a two-way real-time data communication link between the digital twin simulation environment module, the path planning module, the robot control module, and the physical robot.

[0042] Compared with existing technologies, this invention has the following advantages:

[0043] (1) By building a digital twin model of the collaborative robot and assembly scene in Unity, and combining calibration and network communication to achieve real-time synchronization between physical space and virtual space, the trajectory planning and control debugging are moved to the simulation environment, which greatly reduces the workload of teaching and on-site reprogramming, and shortens the production line reconfiguration and debugging time.

[0044] (2) Based on multimodal perception methods such as visual cameras and depth / inertial sensors, the system integrates and models personnel movements, part postures and contact states, and maps them into a digital twin model to achieve precise recognition of assembly intentions and part states, significantly improving the system's perception and response capabilities to unstructured disturbances such as changes in personnel rhythm and part offset.

[0045] (3) In the digital twin environment, the SAC algorithm based on maximum entropy reinforcement learning is adopted to model path planning as a cumulative reward maximization problem. Through iterative training of policy network and dual value network, the assembly accuracy and safety constraints are taken into account, enabling the collaborative robot to have online learning and adaptive optimization capabilities, which significantly improves the robustness and adaptability of path planning.

[0046] (4) The virtual planned trajectory is accurately mapped to the actual robot joint control command by calibration and inverse kinematics, and online compensation is triggered by force / visual feedback. The SAC strategy network performs incremental replanning on the remaining path to realize real-time correction of trajectory and velocity curve, which not only improves the consistency between virtual and real and the assembly accuracy, but also enhances the safety and recovery efficiency of human-machine collaboration under abnormal working conditions. Attached Figure Description

[0047] Figure 1 This is a system block diagram of a human-machine collaborative assembly dynamic cooperative control method based on digital twins according to an embodiment of the present invention;

[0048] Figure 2 This is a schematic diagram illustrating the construction and virtual-real mapping of a digital twin environment according to an embodiment of the present invention;

[0049] Figure 3 This is a flowchart of a multimodal perception and collaborative intent recognition embodiment of the present invention;

[0050] Figure 4 This is a diagram illustrating the internal optimization and reward mechanism of the SAC algorithm according to an embodiment of the present invention.

[0051] Figure 5 This is a flowchart illustrating the mapping of virtual paths to real robot instructions according to an embodiment of the present invention.

[0052] Figure 6 This is a flowchart of a robot dynamic adaptive cooperative control strategy according to an embodiment of the present invention;

[0053] Figure 7 This is a digital twin environment built using Unity, as described in one embodiment of the present invention. Detailed Implementation

[0054] The following specific embodiments further illustrate the dynamic collaborative control method and system for human-machine collaborative assembly robots based on digital twins described in this invention.

[0055] Example 1:

[0056] This embodiment discloses a collaborative control method for a human-machine collaborative assembly robot based on digital twins, such as... Figure 2As shown, it mainly includes the physical assembly site and the Unity digital twin environment. The two interact through the intermediate data interaction channel to form a collaborative control that integrates the virtual and the real.

[0057] Specifically, it includes the following steps:

[0058] S100: Construct a digital twin environment and achieve virtual-physical synchronization. Specifically,

[0059] S110: Unified Coordinate System. By using sensors such as cameras and IMUs for calibration, the transformation relationship between the reference coordinate systems of the virtual and real environments is determined, ensuring the consistency of the two coordinate systems.

[0060] S120: Precise Alignment of Virtual and Real Models. Based on the calibration results above, the positions and orientations of the virtual simulation model and the real physical equipment are aligned, ensuring that the spatial mapping accuracy between the virtual and real scenes reaches an acceptable error range, laying the foundation for path planning and control.

[0061] S130: Import robot and personnel models. Import the 3D models and motion data of actual collaborative robots and personnel into the digital twin environment. Use motion capture equipment to obtain information such as worker gestures and key points of the body skeleton, and drive the virtual model to achieve consistency between the virtual robot and virtual personnel movements and reality.

[0062] S140: Establish a communication channel. A real-time data channel is established between Unity and the physical robot controller using communication protocols (such as TCP / IP, ROS protocol). Robot motion parameters and control commands in the virtual environment can be sent to the real robot through this channel, and on-site sensor data can also be fed back to update the digital twin environment, ensuring real-time synchronization of virtual and real states.

[0063] like Figure 2 As shown, a digital twin simulation environment for the assembly scene is built in Unity, and 3D models of the collaborative robot to be controlled and the assembly scene are created respectively. Sensor and calibration technologies are used to ensure that the coordinate system of the virtual environment is consistent with that of the actual assembly environment. Based on this, precise docking ensures the spatial mapping accuracy between the virtual simulation model and the real world (step S110). Specifically, the coordinates of key points on the actual assembly line can be determined using methods such as camera calibration and IMU calibration, and aligned with the corresponding coordinates in the virtual scene, thus providing a reliable foundation for virtual-real synchronization for subsequent path planning and control (step S120). Simultaneously, the 3D model of the actual collaborative robot and the motion capture data of the operator are imported into Unity to ensure that the robot and personnel movements in the virtual environment are consistent with the physical entities (step S130). Figure 7As shown, a digital twin scene corresponding to the real assembly line was constructed in the Unity environment, including robot models, tooling fixtures, and assembly workpieces, achieving consistency in appearance and geometry. Furthermore, a bidirectional communication channel was established between Unity and the physical robot via network protocols (such as TCP / IP or ROS) (step S140). This communication channel ensures that the robot's motion data and control commands in the virtual environment can be transmitted to the physical robot in real time, while data collected by on-site sensors can also be updated to the digital twin environment in real time, achieving seamless integration between virtual simulation and actual operation.

[0064] S200: Multimodal Environmental Perception and Collaborative Intent Recognition. In a digital twin environment, it comprehensively perceives the assembly site environment and collaborating objects. It acquires data using various sensing devices such as visual cameras, inertial sensors, and force sensors, and combines deep learning and sensor fusion algorithms to achieve scene understanding, such as... Figure 3 As shown, the multimodal perception and collaborative intent recognition process integrates visual, inertial, and semantic information to accurately identify personnel actions and component states. Furthermore, the synchronization of the digital twin environment with the real world assists in subsequent path planning and control strategy development. Specifically,

[0065] S210: Multimodal data input. ROG vision and depth cameras capture the operator's movements and visual features of assembled parts, while inertial measurement units (IMUs) or human posture recognition devices acquire the operator's skeletal key points and motion information; optionally, tactile / force sensors are used to measure the contact force between the robot and the workpiece, acquiring physical interaction data.

[0066] S220: Data Preprocessing and Fusion. Noise reduction, standardization, and correction are performed on the data collected by sensors, such as distortion correction for images and filtering of inertial data, to obtain cleaned data. Then, a data fusion algorithm is used to perform spatiotemporal registration of the multi-source data, merging them into a unified sensing information source and outputting spatiotemporally aligned data, thereby improving the accuracy and robustness of environmental perception.

[0067] S230: State and Position Recognition. Based on computer vision and deep learning technologies (such as CNN semantic segmentation or object detection), the system analyzes the state of the scene to identify the positions and postures of assembled parts, tools, and collaborating personnel. Simultaneously, it utilizes sensor fusion algorithms to process human motion capture skeletal data in parallel to estimate personnel pose. By combining these two pieces of information, the system estimates the position of the robot's end effector, the workpiece pose, and the personnel's posture and motion state in real time, thereby updating the system's environmental state information and understanding the scene's semantics.

[0068] S240: Semantic Intent and Virtual-Real Synchronization. Through multimodal semantic feature fusion technology, visual information is fused with action semantic information (such as high-level instructions like "pick up the nut" or "connect the part") to achieve advanced collaborative intent recognition. The resulting semantic information drives state mapping, writing real-world scene perception data back to the corresponding virtual entity in the digital twin model, achieving real-time synchronous updates of the virtual and real scenes. Simultaneously, it forms intent-driven mechanisms, providing high-level instructions to decision-making and control modules in the digital twin environment, thereby assisting the system in understanding the operator's assembly intent and the current state of the parts. This aids in recognizing the operator's assembly intent and the current state of the parts (for example, by recognizing the operator's gaze direction and hand movements, the target part to be assembled can be inferred, and corresponding prompts can be provided in the digital twin).

[0069] The purpose of multimodal perception in this embodiment is not limited to target recognition and pose estimation; its core output is a high-level semantic state quantity that directly drives collaborative decision-making. This includes: assembly intentions or assembly stage labels inferred from fused visual and motion information (such as "pick up the nut" or "perform insertion"); a collaborative safety domain or risk field model dynamically calculated based on the real-time pose of the personnel; and contact force deviation and part position offset indices characterizing physical interaction anomalies. This state information is designed as a direct input to the path planning strategy network, enabling the robot's motion planning to respond in real-time to the operator's intentions, dynamically avoid safe areas, and adapt to physical contact disturbances.

[0070] S300: Path Planning and Strategy Optimization Based on SAC. Utilizing state information and assembly task objectives within the digital twin environment, the robot path planning problem is treated as a reinforcement learning problem. The path planning strategy is optimized using the Soft Actor-Critic (SAC) algorithm. First, the state space and action space for path planning are defined in the digital twin environment (step S310). The state space may include information such as the robot's current pose, target pose, and the location distribution of obstacles in the environment; the action space consists of continuous control quantities that the robot can execute, such as end-effector linear velocity, angular velocity, or joint angle increments, which are formed into continuous state vectors and action vectors through a state mapping function. Specifically,

[0071] S310: Establish a state-action space description. In the digital twin environment, acquire the robot's current pose and the 3D coordinates of the assembly task target, as well as information such as the positions of surrounding obstacles or personnel. Construct a state feature vector through a mapping function. That is, the robot's current pose... Target pose and the distribution of environmental obstacles As input, a state vector is constructed through a state mapping function. .

[0072] Simultaneously, the robot's end-effector linear velocity, angular velocity, and joint increments are used as continuous motion vectors. This forms a state-action description for reinforcement learning.

[0073] S320: Policy Training and Path Generation. This involves running the robot's interaction with the virtual environment within a digital twin simulation environment and collecting state transition samples. ,in , Let be the state vectors at time k and time k+1. , , Let k be the action vector, instantaneous reward value, and termination flag. State Next, select an action based on the current strategy. And execute it, then the environment will give a reward. And transition to the new state At the same time, determine whether the round has ended. These five elements are stored as a complete experience sample in the experience replay pool. The Policy Network and the Dual Value Network (Q-network) are trained using the SAC algorithm. During training, the assembly task's accuracy, trajectory smoothness, and safety are constrained according to a defined reward function. The policy is optimized to maximize the cumulative reward, and assembly accuracy and safety are evaluated in real time, providing positive / negative feedback based on the reward function.

[0074] The dual-value network employs a soft Q-function update mechanism for each sample. Constructing the soft Q objective value and value network loss function Target value The definition is as follows:

[0075]

[0076] The loss function is defined as follows:

[0077]

[0078] In the above formula, At time step k, the agent performs an action. The immediate rewards obtained from the environment afterwards and For a two-way softQ value network, and The corresponding parameters are divided into The target network, This is the discount factor. For temperature coefficient, For the end-of-round indicator, In the state The following actions are sampled from the current policy network. The negative logarithmic probability of the action. For parameters The policy network; by adjusting the parameters Perform gradient descent to minimize the loss function. Complete the update of the SoftQ value network.

[0079] Updates to the policy network and temperature parameters include:

[0080] Policy network loss function Defined as:

[0081]

[0082] In the above formula, In the state The actions obtained by sampling from the policy network are as follows. The negative logarithmic probability of the action. and This is a dual-soft-Q value network, achieved by adjusting the policy network parameters. Perform gradient descent minimization This is to improve the expected value of path planning while ensuring the strategy entropy;

[0083] Temperature parameters are in logarithmic form. The objective function for adaptive updates is defined as follows:

[0084]

[0085] In the above formula, To preset the target entropy level, The negative logarithmic probability of the action is determined by... Perform gradient descent minimization , and by The updated temperature coefficient is obtained, thereby enabling adaptive balancing of the exploration level and path quality of path planning in different collaborative assembly scenarios.

[0086] After training convergence, the policy network can directly output a sequence of path actions that satisfy the constraints. Finally, the cooperative path planning results obtained from the training are applied to the motion of the virtual robot, and the path sequence can be sent to the physical robot for execution.

[0087] like Figure 4As shown, the internal structure of SAC is a reward-driven, experience-replay-supported mechanism that iteratively updates network parameters through gradient descent. First, the reward function design module integrates three key performance indicators: assembly accuracy loss, trajectory smoothness loss, and safety distance loss, which are quantified and evaluated through their respective loss calculation submodules. These loss values ​​are combined and summed to form the total reward, which ultimately outputs a scalar reward value. This value serves as the core signal for evaluating the quality of the agent / robot's actions, driving the entire learning process. Second, the state transition experiences generated by the agent's interactions in the digital twin environment are stored as tuples in the experience replay pool. The experience replay pool provides batch data for network updates through random sampling, improving learning efficiency and stability. Finally, in gradient update and network optimization, sampled empirical data is used to compute multiple losses in parallel: the policy network (Actor) loss aims to maximize the expected reward and policy entropy, encouraging both the accuracy of the assembly task and enhancing the policy's exploration capability; the double-Q network (Critic) loss aims to minimize the temporal difference error of Q-value estimation, combining the reward at each step with the estimated value of the next state to calculate the current Q-value, and using gradient descent to minimize the loss function to update the network parameters; the temperature coefficient α loss is used to adaptively balance exploration and exploitation. The calculated losses are used to synchronously update the parameters of the policy network and the double-Q network through gradient descent and soft update steps, and the parameters of the target network are slowly updated in a soft update manner to stabilize training, while adaptively adjusting the temperature coefficient α. Through this iterative process, the algorithm enables the policy network to output better, safer, and more dynamically collaborative robot action sequences in a digital twin environment.

[0088] This embodiment utilizes deep reinforcement learning for path planning, enabling autonomous path learning and optimization. By validating assembly tasks in different scenarios within a digital twin environment, reusable collaborative trajectories and action strategies can be obtained for actual assembly.

[0089] In this embodiment, the SAC algorithm's state space, action space, and reward function are redesigned to focus on solving collaborative assembly problems rather than generalized navigation problems. In the state space, the state vector input to the policy network integrates assembly alignment error, dynamic safety domain parameters defined by the personnel's pose, contact force / deviation anomaly indicators from sensors, and current assembly stage information, ensuring the policy fully perceives the core constraints and context of the assembly task. The reward function is constructed closely around the core indicators of the assembly task, including at least: assembly accuracy rewards / penalties to encourage precise alignment, human-machine safety distance constraints to ensure personal safety, contact force threshold constraints to control interaction intensity, trajectory smoothness penalties to optimize motion quality, and stage completion rewards to mark task progress. This multi-objective reward design simultaneously optimizes assembly accuracy, operational safety, and motion smoothness during training.

[0090] S400: Path Mapping and Real-Time Dynamic Control. This maps the path planning results obtained in the digital twin environment to the real robot control system, completing the transformation from virtual path to physical execution (see...). Figure 5 Specifically, this includes coordinate transformation, inverse kinematics calculation of joint angles and generation of control commands, combined with an online adaptive mechanism to dynamically adjust the trajectory.

[0091] S410: Coordinate Transformation and Inverse Kinematics. Using known calibration parameters, the 3D coordinates of the target in the digital twin environment (such as the robot's end effector position) are transformed to the base coordinate system of the real robot, obtaining the target pose in physical space. Based on the transformed target position, the required rotation angles for each joint of the robot are calculated using inverse kinematics algorithms. This process ensures that the trajectory planned virtually is accurately replicated on the actual assembly line.

[0092] S420: Control Command Generation. The calculated joint angle parameters are input into the robot control code library to generate control commands containing joint spatial motion and Cartesian spatial trajectory. The control program supports multiple control modes (such as position control, force / resistance control, etc.) to meet the safety and flexibility requirements of the assembly process. For example, during assembly, an appropriate force control mode can be selected according to different processes, enabling the robot to automatically adjust its posture when subjected to force.

[0093] S421: Real-time Deviation Detection. During robot movement, the assembly process is continuously monitored via force sensors or visual feedback. When the force sensor detects abnormal contact force or the vision system detects workpiece position deviation, an online adjustment mechanism is triggered (see [link]). Figure 6 (See the control strategy flowchart shown).

[0094] S422: Online trajectory replanning. Once an anomaly is detected, the remaining path planning scheme is recalculated using the previously trained SAC policy network combined with the current assembly status. Based on the deviation type and assembly progress, compensation increments are generated by increasing or decreasing joint angle and speed parameters to adjust the path and achieve the correction purpose.

[0095] S423: Compensation Application and Trajectory Update. The calculated compensation amount is incorporated into the robot control commands to update the robot joint motion trajectory and velocity curve in real time. This allows for dynamic correction of assembly errors during execution, ensuring assembly accuracy and personnel safety during collaborative processes. Figure 6 The diagram shows an adaptive cooperative control strategy.

[0096] like Figure 5 As shown, the mapping process from digital twin path to robot instructions includes two steps: coordinate transformation and instruction generation. Specifically, this process starts from the Unity DT virtual trajectory points planned and generated in the Unity Digital Twin (DT) environment, which contains a series of raw pose data representing the desired pose. This data is first sent to the seamless virtual-real mapping core module, where coordinate transformation and joint angle calculation are performed. Coordinate transformation converts the raw pose in virtual space to a base coordinate system based on the robot's base, ensuring spatial reference consistency. Joint angles are calculated using inverse kinematics, converting the target pose into the joint angles required for each axis of the robot. Subsequently, the calculated base coordinates and joint angle information are passed to the instruction generation call control function library. According to the specific protocol and API of the robot controller, the above data is encapsulated into code that can be directly recognized and executed by the hardware, such as specific motion instructions. The generated instructions are sent to the real robot for execution, driving the robot entity to complete the assembly action.

[0097] To address potential model errors or environmental disturbances during execution, this embodiment also introduces dynamic deviation feedback. Sensors monitor the execution results in real time, and the deviation between the actual pose and the desired pose is fed back to the control link, triggering an online correction mechanism. The correction signal can be fed forward to the instruction generation stage for real-time compensation and adjustment of subsequent instructions, effectively ensuring high-precision and high-reliability mapping from virtual paths to physical actions.

[0098] like Figure 6As shown, this further demonstrates how a dynamic adaptive strategy of replanning and applying compensation is triggered by force sensing when a deviation is detected in real time. In step S421, which involves anomaly detection and determining whether to trigger the online adjustment mechanism, real-time feedback from force sensor and vision sensor data is received, and multi-sensor information fusion is performed. By applying threshold judgments to key physical quantities (such as contact force and positional deviation), the current state is classified as normal or abnormal. Once an abnormality is determined, indicating an unplanned situation such as accidental part movement, abnormal assembly resistance, or human intervention, the online adjustment mechanism is immediately triggered, sending an online adjustment signal. In step S422, the model adaptive planning uses the current assembly state and anomaly information as input, calling the pre-trained SAC (Soft Actor-Critic) model for rapid replanning. The core SAC policy network and value network work together, based on a maximum entropy reinforcement learning framework, to optimize the remaining path planning online and generate corresponding motion increment parameters (such as compensation amounts for joint angles or end-effector poses), outputting the optimal or suboptimal compensation strategy to adapt to the new state. In step S423, during real-time updating and execution, to ensure the feasibility and safety of the action, the planned compensation instructions are first smoothed and then subjected to rigorous safety constraint checks. If the check result indicates normal constraints, the drive instructions are directly updated and sent to the robot actuator to execute the compensation action, while also updating the physical and visual states. If the check result indicates abnormal constraints (e.g., the compensation action may cause a collision), the system will switch to executing the compensation action (e.g., emergency stop or retreat) and may re-trigger the planning process. In this embodiment, the dynamic adaptive strategy combined with this online adjustment mechanism enables the robot to respond promptly to unstructured changes (e.g., part movement, human intervention), improving the flexibility and robustness of assembly.

[0099] In summary, this embodiment constructs a digital twin model of the assembly scene in the Unity environment and synchronizes it with the physical system in real time. It utilizes multimodal perception methods such as vision, inertia, and semantics for environmental perception and human intent recognition, and performs path planning based on the Soft Actor-Critic algorithm. Finally, it maps the virtual path to the motion commands of the physical robot and performs real-time adaptive adjustments. The adaptive adjustment has explicit triggering conditions, such as the vision system detecting a workpiece position deviation exceeding a threshold, or the force sensor detecting abnormal contact force / collision. Once triggered, the system immediately invokes the pre-trained SAC policy network based on the updated digital twin state to perform rapid incremental replanning of the robot's remaining motion trajectory. The replanning outputs compensation amounts for joint angles and velocity parameters, and updates and sends control commands to the physical robot in real time. This mechanism directly solves the "online mismatch" problem caused by human intervention, workpiece movement, and contact disturbances in collaborative assembly, resulting in significant technical effects such as improved overall system safety, assembly success rate, environmental robustness, and real-time response speed. This method, through the fusion of digital twins and reinforcement learning, enhances the intelligence and adaptability of the human-machine collaborative assembly system, realizing dynamic collaborative control in complex assembly scenarios. The steps and measures described in all embodiments can be combined or adjusted according to actual needs, and their equivalent changes should also be included within the scope of protection of this invention.

[0100] Example 2:

[0101] This embodiment discloses a collaborative control system for a human-machine collaborative assembly robot based on digital twins, such as... Figure 1 As shown, the system includes a digital twin simulation environment module, a sensor module, a path planning module, a robot control module, and a communication module. The digital twin simulation environment module uses the Unity engine to construct a virtual environment for the assembly scene and maintains the virtual-real mapping relationship. The sensor module collects data on personnel actions and part states. The path planning module uses a reinforcement learning algorithm based on SAC to generate collaborative robot paths. The robot control module performs coordinate transformations and generates motion control commands based on the planning results. The communication module ensures real-time data interaction between the virtual and real worlds.

[0102] like Figure 7 As shown, a specific implementation method and device for a dynamic collaborative control system for human-machine collaborative assembly in a digital twin environment are disclosed. The system integrates components such as an embedded control unit, a multimodal sensing device, a path planning module, a robot control system, and a digital twin simulation platform, and can support real-time perception, intent recognition, path optimization, and online compensation functions in typical unstructured assembly scenarios.

[0103] The hardware components include:

[0104] Embedded controller: Deployed on the collaborative robot side, it is used to execute control strategies, receive planned paths, and output motion commands. It has ROS communication protocol support and integrates inverse kinematics and TCP / IP interface modules.

[0105] Multimodal sensor array: Deployed at assembly stations and collaborative areas, including depth cameras, RGB vision modules, inertial measurement units (IMUs) and force sensors, used to collect personnel movements, assembly part status and robot end effector feedback;

[0106] Collaborative robot platform: It features a seven-degree-of-freedom joint structure, coupled with a force-controlled end effector, supporting Cartesian trajectory planning and joint-level control;

[0107] The software component includes:

[0108] Digital Twin Platform: Built on Unity 3D, it achieves bidirectional communication and state synchronization with the real physical system through ROS bridging, and integrates functions such as assembly task modeling, perception visualization, and path testing in the simulation environment;

[0109] SAC Path Planning Module: Deployed on the server side, it uses reinforcement learning algorithms to train and infer collaborative paths, responds in real time to changes in assembly intent and outputs the optimal strategy.

[0110] Control code generation module: Based on the path results in the digital twin environment, it generates the corresponding robot control code and encapsulates it into a control sub-function library, including position mode, speed mode and force control mode.

[0111] The system operation process is as follows:

[0112] During initialization, the physical system collects workstation status and personnel actions through sensors, and the digital twin platform performs virtual-real coordinate mapping based on the calibration data, synchronizing the scene status to the simulation model in real time.

[0113] After the data collected by the sensors is fused and processed, it is pushed to the SAC strategy module for path planning to obtain the optimal motion trajectory and desired posture.

[0114] The control code module calculates the angles of each joint based on the trajectory points and converts them into robot control commands through inverse kinematics solutions, which are then sent to the controller.

[0115] If visual deviations or force control abnormalities occur during the movement, a dynamic adjustment mechanism is triggered, and the remaining path is replanned and compensated through the policy network.

[0116] Once the assembly task is completed, the digital twin model updates the task status to provide a reference for subsequent batch execution.

[0117] The control flow in this embodiment is as follows: Figure 6As shown, a closed-loop system structure of sensing perception, path generation, control mapping, and dynamic compensation is presented. This system is highly flexible and adaptable, suitable for complex assembly operations with multiple product types and small batches, and can significantly improve human-machine collaboration efficiency and assembly accuracy.

[0118] This invention addresses the complex and specific industrial scenario of "human-machine collaborative assembly" by implementing a series of engineering and systematic designs. These include: defining a state space and action space with practical physical meaning for this scenario; designing a reward function that integrates multiple constraints such as assembly accuracy, motion smoothness, and human-machine safety; and constructing a complete control closed loop from multimodal perception, virtual-real synchronization, strategy planning to online dynamic compensation. Through these targeted modifications and integrations, the standard SAC algorithm can achieve stable, reliable, and adaptive collaborative operation under real physical conditions involving dynamic obstacles, high-precision assembly requirements, strict personal safety constraints, and uncertain contact disturbances.

[0119] Although the present invention has been described in detail with reference to the accompanying drawings and the preferred embodiments described above, those skilled in the art can make various equivalent modifications or substitutions to its specific forms without departing from the spirit and essence of the invention. All such equivalent modifications or substitutions should be considered to fall within the protection scope of the present invention. The protection scope of the present invention is defined by the appended claims.

Claims

1. A collaborative control method for a human-machine collaborative assembly robot based on digital twins, characterized in that, Includes the following steps: S100: Establish a digital twin environment in Unity, and create digital twin models of the collaborative robot to be controlled and the assembly scene respectively; calibrate the coordinate system of the unified virtual and physical space through sensors; establish a two-way real-time communication channel between Unity and the physical robot controller to synchronize status information; S200: Perform environmental perception in the digital twin environment and generate the current environmental state of the system; S300: Based on the state information and assembly task objectives in the digital twin environment, path planning and policy optimization are performed using the SAC algorithm. A reward function is constructed with assembly task accuracy and safety as constraints. A policy network and a double Q-value network based on a deep neural network are trained, outputting a robot cooperative path action sequence that satisfies the constraints. Specifically, this includes: S310: Establish the path planning state space and motion space in the digital twin environment, and define the robot's current pose. Target pose and the distribution of environmental obstacles As input, through the state mapping function Constructing state vectors The robot's linear velocity, angular velocity, and / or joint increments are used to construct a continuous motion vector. This forms a continuous state-action description for path planning; S320: Collect samples in the digital twin environment. ,in , Let be the state vectors at time k and time k+1. , , Let k be the action vector, immediate reward value, and termination flag; utilize the maximum entropy reinforcement learning objective function, and iteratively update the policy network parameters η and the double soft Q-value network parameters using gradient descent. And the temperature parameter β, to maximize the cumulative reward and maintain the policy entropy, finally obtain the optimized path planning policy; whereby, the update of the dual soft Q-value network is based on the soft Q objective value. and value network loss function Target value The definition is as follows: ， The loss function is defined as follows: ， In the formula, At time step k, the agent performs an action. The immediate rewards obtained from the environment afterwards and For a two-way softQ value network, and For the corresponding target network, This is the discount factor. For temperature coefficient, For the end-of-round indicator, In the state The following actions are sampled from the current policy network. The negative logarithmic probability of the action. For parameters The policy network; by adjusting the parameters Perform gradient descent to minimize the loss function. Complete the update of the softQ value network; S400: Map the virtual path obtained in step S300 to the real robot control system and generate robot execution code; during physical execution, adaptively adjust the trajectory based on real-time sensor feedback to realize dynamic collaborative operation of the robot in the physical assembly environment.

2. The method according to claim 1, characterized in that, In step S100, realizing the synchronization status information specifically includes: importing the three-dimensional model of the physical collaborative robot and the human motion data obtained through motion capture equipment into the digital twin simulation environment, driving the virtual model to maintain consistency with the physical entity; and using network protocols to establish a bidirectional data channel between Unity and the physical robot controller for transmitting motion data and control commands.

3. The method according to claim 1, characterized in that, Step S200 includes: using an RGB camera, a depth camera, and an inertial measurement unit (IMU) to collect visual images, depth information, and skeletal key point data of the operator, respectively; performing noise reduction, filtering, and standardization preprocessing on the collected multi-source sensor data, and generating unified and accurate environmental perception information through a sensor fusion algorithm; using computer vision and deep learning algorithms to identify the pose of target objects in the scene, and combining action semantic information to identify the personnel's assembly intention and the state of parts in the digital twin model.

4. The method according to claim 1, characterized in that, In step S320, the policy network is updated as follows: Policy network loss function Defined as: ， In the formula, For temperature coefficient, In the state The actions obtained by sampling from the policy network are as follows. The negative logarithmic probability of the action. and This is a dual-soft-Q value network, achieved by adjusting the policy network parameters. Perform gradient descent minimization This aims to improve the expected value of path planning while ensuring policy entropy.

5. The method according to claim 1, characterized in that, In step S320, the temperature parameter is in logarithmic form. The objective function for adaptive updates is defined as follows: ， In the formula, To preset the target entropy level, In the state The actions obtained by sampling from the policy network are as follows. The negative logarithmic probability of the action is determined by... Perform gradient descent minimization , and by The updated temperature coefficient is obtained, thereby enabling adaptive balancing of the exploration level and path quality of path planning in different collaborative assembly scenarios.

6. The method according to claim 1, characterized in that, Step S400 includes: converting the three-dimensional coordinates in the digital twin environment into spatial coordinates in the base coordinate system of the real robot using known calibration parameters; solving the rotation angle corresponding to each joint of the robot using an inverse kinematics algorithm; inputting the angle parameters into the robot control code sub-function library to generate control instructions containing joint spatial motion and Cartesian spatial trajectory, wherein the control instructions support force control and impedance control modes.

7. The method according to claim 1, characterized in that, In step S400, the adaptive adjustment is as follows: During robot movement, when force sensors or visual feedback detect part misalignment or collision anomalies, an online adjustment mechanism is triggered. The current robot state, environment state, and remaining task objectives are input into the trained SAC policy network to recalculate the optimal or suboptimal remaining path sequence from the current state to the task objective. Based on the replanning results, the required joint angles or end-effector pose compensations are calculated, and the control commands sent to the physical robot are updated in real time to correct the motion trajectory.

8. A collaborative control system for a human-machine collaborative assembly robot based on digital twins, characterized in that, The system for implementing the method as described in any one of claims 1 to 7 comprises: The digital twin simulation environment module, built on the Unity engine, is used to establish and maintain a virtual model and virtual-real mapping relationship consistent with the physical assembly scene; The sensor module is used to collect multimodal data on human actions, assembly parts status, and robot status in the physical environment. The path planning module runs the SAC reinforcement learning algorithm, receives the status information from the digital twin simulation environment module, performs online path planning and strategy optimization, and generates the motion path of the collaborative robot. The robot control module is used to convert the virtual path generated by the path planning module into joint control commands for the physical robot through coordinate transformation and inverse kinematics calculation; and to perform online trajectory replanning and compensation when receiving abnormal feedback. The communication module is used to establish a two-way real-time data communication link between the digital twin simulation environment module, the path planning module, the robot control module, and the physical robot.

Images