Human-machine cooperation assembly track generation method, device and equipment based on digital human

Abstract

The invention provides a man-machine cooperation assembly track generation method, device and equipment based on a digital human, and relates to the technical field of man-machine cooperation, and the method comprises the steps: constructing a digital human model, a cooperation robot model and an assembly environment based on musculoskeletal on a target simulation platform, defining the attributes of each model, and forming a man-machine cooperation assembly simulation platform; taking the man-machine cooperation assembly simulation platform as an interaction environment, training a digital human control strategy network based on a deep reinforcement learning algorithm, and generating a digital human assembly track in an assembly process; introducing the trained digital human control strategy network into a man-machine collaborative assembly simulation platform, taking a dynamic assembly environment as a training environment of a collaborative robot, training the obstacle avoidance capability of the collaborative robot to avoid dynamic actions of a digital human through a deep reinforcement learning algorithm, generating a robot obstacle avoidance strategy network, and performing robot obstacle avoidance simulation on the robot. And outputting a robot assembly track based on the robot obstacle avoidance strategy network. By means of the method, man-machine safety cooperation assembly can be achieved.

Description

Technical Field

[0001] This application relates to the field of human-machine collaboration technology, and in particular to a method, apparatus and equipment for generating human-machine collaborative assembly trajectories based on digital humans. Background Technology

[0002] Currently, manufacturing systems are increasingly demanding greater flexibility and intelligence. Human-robot collaborative assembly, as a new manufacturing model, helps combine the strengths of both humans and robots to achieve flexible production. The human-robot collaborative assembly environment is unstructured; the robot's control strategy is not pre-set and needs to learn and optimize through collaboration with humans. To achieve natural collaboration between humans and robots, a large amount of data from the human-robot collaborative assembly process needs to be collected for training the robot's control strategy.

[0003] However, the current method of collecting data in actual human-machine collaborative assembly suffers from low efficiency and small data volume, making it difficult to meet the requirements of human-machine collaborative assembly in complex scenarios. Summary of the Invention

[0004] In view of this, this application proposes a method, apparatus and device for generating human-machine collaborative assembly trajectories based on digital humans.

[0005] Firstly, this application provides a method for generating human-machine collaborative assembly trajectories based on digital humans, including:

[0006] Based on the acquired human upper limb model and human simulation model, a musculoskeletal digital human model is constructed on the target simulation platform;

[0007] A collaborative robot model and assembly environment are constructed on the target simulation platform, and the attributes of the digital human model, the collaborative robot model, and the entities in the assembly environment are defined to form a human-machine collaborative assembly simulation platform.

[0008] Using the human-machine collaborative assembly simulation platform as an interactive environment, a digital human control strategy network based on a deep reinforcement learning algorithm is trained based on a predefined first state space, first action space, and first reward function, and the digital human assembly trajectory during the assembly process is generated based on the trained digital human control strategy network.

[0009] The trained digital human control strategy network is introduced into the human-machine collaborative assembly simulation platform to construct a dynamic assembly environment. The dynamic assembly environment is used as the training environment for the collaborative robot. Based on the predefined second state space, second action space and second reward function, the collaborative robot is trained to avoid obstacles by avoiding the dynamic actions of the digital human through a deep reinforcement learning algorithm. This generates a robot obstacle avoidance strategy network, and the robot assembly trajectory with dynamic obstacle avoidance capability is output based on the robot obstacle avoidance strategy network.

[0010] In one embodiment, the construction of a musculoskeletal-based digital human model on a target simulation platform based on the acquired human upper limb model and human simulation model includes:

[0011] The acquired human upper limb model and human simulation model are converted to the target simulation platform to construct a musculoskeletal morphology model;

[0012] A mathematical model of muscle activation and muscle force is constructed based on the Hill-type muscle model, and a dynamic model of the musculoskeletal morphology model is established.

[0013] The musculoskeletal morphology model, the mathematical model of muscle activation and muscle force, and the dynamic model are integrated to form the digital human model.

[0014] In one embodiment, constructing the collaborative robot model and assembly environment on the target simulation platform includes:

[0015] The body structure, dynamic parameters, and control interface of the collaborative robot are defined on the target simulation platform. The components of the collaborative robot are organized by tag collaboration and a hierarchical nesting relationship is constructed to establish the collaborative robot model.

[0016] Import the 3D model file of the corresponding assembly environment into the target simulation platform, set the attribute parameters of the assembly environment, and establish the assembly environment on the target simulation platform.

[0017] In one embodiment, defining the attributes of the digital human model, the collaborative robot model, and the entities in the assembly environment includes:

[0018] Acquire the first position and posture information of the digital human model and the collaborative robot model, as well as the second position information of the assembled parts;

[0019] Based on the first position information, the attitude information, and the second position information, the position, attitude, and collision attributes of each entity are defined by the tags in the target simulation platform configuration file.

[0020] In one embodiment, the first state space includes the joint angles, joint velocities, muscle activation values, end-effector positions, and target distances of the digital human model.

[0021] The first action space is defined as a muscle activation signal composed of multidimensional continuous control variables;

[0022] The first reward function includes a target completion reward, a target distance reward, and a muscle energy consumption penalty; the target completion reward is positive when the digital human completes the target task and zero when the task is not completed; the target distance reward is obtained by calculating the Euclidean distance between the current position and the target position and taking the negative value; the muscle energy consumption penalty is the L2 norm average of the muscle activation signal per unit time.

[0023] In one embodiment, the deep reinforcement learning algorithm is the SAC algorithm; the step of using the human-machine collaborative assembly simulation platform as the interactive environment and training a digital human control policy network based on the deep reinforcement learning algorithm according to a predefined state space, action space, and reward function includes:

[0024] Obtain the basic training parameters and the initial state parameters of the round;

[0025] Based on the initial state parameters of the round, the initial joint angle of the right upper limb of the digital human model is set at the beginning of the training round, and random perturbation is introduced;

[0026] For each training round, an exploration operation is performed until the iteration termination condition corresponding to the training basic parameters is reached. The exploration operation includes: determining the current strategy based on the current state space of the interactive environment, sampling the action corresponding to the muscle activation signal of the digital human arm, executing the action in the interactive environment to control the movement of the digital human, calculating the reward value through the reward function and feeding it back to the next state space, and storing the interaction sample in the experience replay pool.

[0027] When the number of samples in the experience replay pool reaches a preset threshold, a network update operation is initiated after each interaction. The network update operation includes dual Critic network update and Actor network update.

[0028] In one embodiment, there is a second state space, a second action space, and a second reward function;

[0029] The second state space includes the collaborative robot's own state, motion target, and environmental constraints;

[0030] The second motion space is defined as the incremental control quantities of the six joints of the collaborative robot;

[0031] The second reward function includes a target distance penalty, a task completion reward, a first obstacle avoidance penalty, and a second obstacle avoidance penalty. The first obstacle avoidance penalty is for obstacle avoidance between the robot and the digital human, and the second obstacle avoidance penalty is for obstacle avoidance between the collaborative robot and a stationary obstacle. The target distance penalty is calculated based on the Euclidean distance between the robot's end effector and the target position. The task completion reward is positive when the collaborative robot completes the task and zero when it fails to complete the task. The first obstacle avoidance penalty is determined based on the human-robot safe distance threshold, the human-robot collision detection distance, and the minimum Euclidean distance between the collaborative robot and the digital human's upper limb. The second obstacle avoidance penalty is determined based on the collision safety distance and the minimum Euclidean distance between the collaborative robot and a stationary obstacle.

[0032] Secondly, this application also provides a human-machine collaborative assembly trajectory generation device based on a digital human, comprising:

[0033] The first building module is used to construct a musculoskeletal digital human model on the target simulation platform based on the acquired human upper limb model and human simulation model.

[0034] The second construction module is used to build a collaborative robot model and assembly environment on the target simulation platform, and to define the attributes of the digital human model, the collaborative robot model and the entities in the assembly environment to form a human-machine collaborative assembly simulation platform.

[0035] The first training module is used to use the human-machine collaborative assembly simulation platform as an interactive environment to train a digital human control strategy network based on a deep reinforcement learning algorithm based on a predefined first state space, first action space and first reward function, and to generate a digital human assembly trajectory during the assembly process based on the trained digital human control strategy network.

[0036] The second training module is used to introduce the trained digital human control strategy network into the human-machine collaborative assembly simulation platform to construct a dynamic assembly environment. The dynamic assembly environment is used as the training environment for the collaborative robot. Based on the predefined second state space, second action space and second reward function, the collaborative robot is trained to avoid obstacles by using a deep reinforcement learning algorithm to avoid the dynamic actions of the digital human. The robot obstacle avoidance strategy network is generated, and the robot assembly trajectory with dynamic obstacle avoidance capability is output based on the robot obstacle avoidance strategy network.

[0037] Thirdly, this application also provides an electronic device, including a processor and a memory; the memory has a computer program stored thereon, wherein the computer program, when executed by the processor, implements the human-machine collaborative assembly trajectory generation method based on digital humans as described in the first aspect.

[0038] Fourthly, this application also provides a computer storage medium storing a computer program thereon, wherein the computer program, when executed by a processor, implements the human-machine collaborative assembly trajectory generation method based on digital humans as described in the first aspect.

[0039] The human-computer collaborative assembly trajectory generation method based on digital humans proposed in this application has the following advantages over related technologies:

[0040] 1. The human-machine collaborative assembly trajectory generation method of this application constructs a human-machine collaborative assembly simulation platform on a target simulation platform, forming a human-machine collaborative assembly simulation environment, which can also serve as an interactive environment for reinforcement learning training. Based on this, the human-machine collaborative assembly simulation platform is used as the interactive environment to train a digital human control policy network based on a deep reinforcement learning algorithm, using a predefined first state space, first action space, and first reward function. The trained digital human control policy network is then used to generate the digital human assembly trajectory during the assembly process. This digital human assembly trajectory is a natural motion trajectory that conforms to the constraints of the real human body structure, effectively simulating the motion trajectory of the human body. Furthermore, the trained digital human control policy network is introduced into the human-machine collaborative assembly simulation platform to construct a dynamic assembly environment. This dynamic assembly environment can serve as a training environment for the collaborative robot. Based on a predefined second state space, second action space, and second reward function, a deep reinforcement learning algorithm is used to train the collaborative robot's obstacle avoidance ability to avoid the dynamic actions of the digital human, generating a robot obstacle avoidance policy network. Based on this robot obstacle avoidance policy network, a robot assembly trajectory with dynamic obstacle avoidance capabilities can be output, thereby achieving safe collaborative assembly between humans and collaborative robots in a shared space, improving the efficiency and safety of the human-machine collaborative assembly process.

[0041] 2. By converting the acquired human upper limb model and human simulation model to the target simulation platform to construct a musculoskeletal morphological model, core data conforming to human anatomy from existing models can be fully reused. This ensures that the structure and morphology of the digital human's upper limb are highly consistent with the real human body, avoiding deviations from physiological limits in subsequent motion simulations due to morphological distortion. This provides a precise structural carrier for the digital human to have a realistic movement foundation. A mathematical model of muscle activation and muscle force is constructed based on the Hill-type muscle model, which can accurately quantify the biomechanical mapping relationship from neural activation signals to muscle contraction force. The dynamic model of the musculoskeletal morphological model can then link muscle contraction force and joint motion state through mechanical equations, ensuring that the digital human's movement conforms to physical laws and avoiding problems that do not conform to the movement trajectory of the real human body. This provides key functional support for the realism and controllability of the digital human's movement. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a flowchart illustrating a human-machine collaborative assembly trajectory generation method based on digital humans in one embodiment of this application;

[0044] Figure 2 This is a flowchart illustrating step S103 in one embodiment of this application;

[0045] Figure 3 This is a schematic diagram of the digital human training process based on the SAC algorithm in one embodiment of this application;

[0046] Figure 4 This is a schematic diagram of the reward value change curve during the digital human training process in one embodiment of this application;

[0047] Figure 5 This is a schematic diagram comparing the joint trajectories of a digital human and a real human body in one embodiment of this application;

[0048] Figure 6 This is a schematic diagram showing the error distribution between the virtual trajectory and the real trajectory of each joint in one embodiment of this application;

[0049] Figure 7 This is a schematic diagram of the Fitts Law verification fitting curve for a digital human in one embodiment of this application;

[0050] Figure 8 This is a schematic diagram of the dynamic obstacle avoidance training process of a collaborative robot in one embodiment of this application;

[0051] Figure 9 This is a schematic diagram of the reward value change curve during the training process of a collaborative robot in one embodiment of this application;

[0052] Figure 10 This is a schematic diagram of the joint angle changes of a collaborative robot after training is completed in one embodiment of this application;

[0053] Figure 11 This is a schematic diagram of human-machine collaboration trajectory planning during a certain assembly process in one embodiment of this application;

[0054] Figure 12 This is a schematic diagram of the structure of a human-machine collaborative assembly trajectory generation device based on digital humans in one embodiment of this application. Detailed Implementation

[0055] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0056] In some embodiments, such as Figure 1 As shown, this application discloses a human-machine collaborative assembly trajectory generation method based on digital humans, which includes the following steps S101 to S104.

[0057] S101: Based on the acquired human upper limb model and human simulation model, construct a musculoskeletal digital human model on the target simulation platform.

[0058] Among these, the human upper limb model typically serves as a basic morphological template. Its core value lies in providing structural data that conforms to human anatomy, including the geometry of the upper limb skeleton, the connection methods and range of motion of the joints, as well as the attachment points, muscle fiber orientation, and physiological length of the muscles. This data is a prerequisite for ensuring that the digital human's upper limbs resemble those of a real human. The human simulation model, on the other hand, focuses more on supplementing dynamic and kinematic parameters, such as physical properties like bone mass and moment of inertia, motion constraint parameters like joint damping coefficients and friction coefficients, and mechanical parameters like muscle contraction characteristics. This provides the functional foundation for the digital human to subsequently achieve movements that closely resemble those of a real human.

[0059] S102: Construct a collaborative robot model and assembly environment on the target simulation platform, and define the attributes of entities in the digital human model, collaborative robot model and assembly environment to form a human-machine collaborative assembly simulation platform.

[0060] In the application, 3D modeling software can be used to create 3D models of the geometry in the assembly environment. These models are then imported into the target simulation platform, where the assembly environment is constructed. Basic attributes, such as physical, spatial, and interactive attributes, can be set for each environmental entity, transforming the assembly environment from a static geometric model into a dynamic scene that can participate in simulation interactions. The collaborative robot model can be built using the modeling tools or configuration files of the target simulation platform, based on the platform's corresponding construction rules. Furthermore, the attributes of the digital human model, collaborative robot model, and assembly environment entities can be uniformly defined, eliminating interaction logic conflicts between different models through standardized attribute definitions. For example, the attributes of the digital human model, collaborative robot model, and assembly environment entities may include position parameters and collision parameters.

[0061] S103: Using the human-machine collaborative assembly simulation platform as the interactive environment, a digital human control policy network based on a deep reinforcement learning algorithm is trained based on a predefined first state space, first action space, and first reward function, and the digital human assembly trajectory during the assembly process is generated based on the trained digital human control policy network.

[0062] Among them, the deep reinforcement learning algorithm can be the Soft Actor-Critic (SAC) algorithm. The SAC algorithm is a continuous control policy algorithm based on the maximum entropy reinforcement learning theory, which can improve the exploratory nature and stability of the policy while ensuring learning efficiency.

[0063] It is understandable that using a human-machine collaborative assembly simulation platform as an interactive environment can provide a high-fidelity, controllable virtual scenario for training digital human control strategies. Replacing real-world scenarios with simulation significantly reduces hardware investment and safety risks during training, and allows for flexible setting of environmental parameters, improving the generalization ability of the strategy network. This avoids the hardware wear and tear, safety risks, and excessive costs associated with relying on real physical environments. Deep reinforcement learning algorithms are used to train the digital human control strategy network. During training, the network gradually learns the motion strategies of the digital human during assembly. When generating assembly trajectories based on the trained strategy network, the network dynamically outputs continuous control actions according to the real-time state during assembly. These actions are chained together in a time sequence, ultimately generating a digital human assembly trajectory that conforms to real human motion.

[0064] S104: The trained digital human control strategy network is introduced into the human-machine collaborative assembly simulation platform to construct a dynamic assembly environment. The dynamic assembly environment is used as the training environment for the collaborative robot. Based on the predefined second state space, second action space and second reward function, the collaborative robot is trained to avoid obstacles by avoiding the dynamic actions of the digital human through a deep reinforcement learning algorithm. The robot obstacle avoidance strategy network is generated, and the robot assembly trajectory with dynamic obstacle avoidance capability is output based on the robot obstacle avoidance strategy network.

[0065] It is understandable that introducing the trained digital human control strategy network into a human-robot collaborative assembly simulation platform can transform the originally static assembly environment into a dynamic assembly environment that includes the real-time movement of the digital human. This environment can realistically simulate the uncertainty of the digital human's actions in human-robot collaboration, providing a practical interactive foundation for the obstacle avoidance training of collaborative robots. Using this dynamic assembly environment as the training environment for collaborative robots allows the robot to directly face the core obstacle avoidance target—the dynamic movements of the digital human—rather than static obstacles, ensuring that the training objective aligns with the actual needs of human-robot collaboration. Based on this, by training the collaborative robot's obstacle avoidance ability to avoid the dynamic movements of the digital human using deep reinforcement learning algorithms, a robot obstacle avoidance strategy network can be generated. Based on the robot assembly trajectory output by this network, the robot can adjust its movement path in real time under the interference of the dynamic movements of the digital human, satisfying both assembly accuracy and efficiency requirements while ensuring safety in human-robot collaboration.

[0066] The aforementioned human-robot collaborative assembly trajectory generation method constructs a human-robot collaborative assembly simulation platform on a target simulation platform, forming a human-robot collaborative assembly simulation environment that can also serve as an interactive environment for reinforcement learning training. Based on this, the human-robot collaborative assembly simulation platform, as an interactive environment, trains a digital human control policy network based on a deep reinforcement learning algorithm, using a predefined first state space, first action space, and first reward function. The trained digital human control policy network then generates the digital human assembly trajectory during the assembly process. This digital human assembly trajectory is a natural motion trajectory that conforms to the constraints of the real human body structure, effectively simulating human motion trajectories. Furthermore, introducing the trained digital human control policy network into the human-robot collaborative assembly simulation platform to construct a dynamic assembly environment allows this dynamic assembly environment to serve as a training environment for the collaborative robot. Based on a predefined second state space, second action space, and second reward function, a deep reinforcement learning algorithm trains the collaborative robot to avoid obstacles in the dynamic actions of the digital human, generating a robot obstacle avoidance policy network. This robot obstacle avoidance policy network can output a robot assembly trajectory with dynamic obstacle avoidance capabilities, thereby achieving safe collaborative assembly between humans and collaborative robots in a shared space, improving both efficiency and safety in the human-robot collaborative assembly process.

[0067] In some embodiments, step S101, constructing a musculoskeletal-based digital human model on a target simulation platform based on the acquired human upper limb model and human simulation model, includes: converting the acquired human upper limb model and human simulation model to the target simulation platform to construct a musculoskeletal morphological model; constructing a mathematical model of muscle activation and muscle force based on the Hill-type muscle model, and establishing a dynamic model of the musculoskeletal morphological model; and integrating the musculoskeletal morphological model, the mathematical model of muscle activation and muscle force, and the dynamic model to form a digital human model.

[0068] For example, the target simulation platform can be the MuJoCo platform, which can convert the MoBL human upper limb model and MyoHand model in OpenSim to MuJoCo, and then define parameters to build a musculoskeletal digital human model. The model is mainly composed of bones, joints and muscles, where muscles are mainly composed of muscle fibers and tendons. Muscle fibers are activated and controlled by nerves to generate contractile force, which is transmitted to the bones through tendons to form joint movement.

[0069] The Hill-type muscle model can be used to describe the process of muscle contraction driven by neural signals. This model controls the degree of muscle activation through neural stimulation signals, and the dynamic process of muscle activation can be represented by the following differential equation:

[0070]

[0071] in, Represents current muscle activation. The signals representing nerve stimulation of muscles are both within the range of [0,1]. The activation time constant is the time constant between the activation and deactivation processes. and A related nonlinear function.

[0072] Based on this, muscles generate muscle force after being activated by signals. for:

[0073]

[0074] in, This represents the maximum isometric contractile force of the muscle. Muscle length The changing driving force-length relationship function Speed ​​of muscle contraction or extension The changing driving force-velocity relationship function Indicates muscles in Activation value at time, It is a passive-length relationship function.

[0075] To describe the movement of a digital human driven by muscle activation, a dynamic model of the musculoskeletal digital human model is established to describe the dynamic relationship between muscle control, joint movement, and external perturbations. Its expression is as follows:

[0076]

[0077] in, , , These represent the angle, angular velocity, and angular acceleration of the joint, respectively. Represents the mass-inertia matrix. Represents the Coriolis force and centrifugal force terms. Represents the term of gravity. It is the joint-driving torque generated by muscle activation. It is interference or contact torque from the external environment.

[0078] In this embodiment, the acquired human upper limb model and human simulation model are converted to the target simulation platform to construct a musculoskeletal morphology model. This allows for the full reuse of core data from existing models that conform to human anatomy, ensuring that the structure and morphology of the digital human's upper limb are highly consistent with the real human body. This avoids deviations from physiological limits in subsequent motion simulations due to morphological distortion, providing a precise structural carrier for the digital human to have a realistic movement foundation. A mathematical model of muscle activation and muscle force is constructed based on the Hill-type muscle model, which can accurately quantify the biomechanical mapping relationship from neural activation signals to muscle contraction force. The dynamic model of the musculoskeletal morphology model can then link muscle contraction force and joint motion state through mechanical equations, ensuring that the digital human's movement conforms to physical laws and avoiding problems that do not conform to the movement trajectory of the real human body. This provides crucial functional support for the realism and controllability of the digital human's movement.

[0079] In some embodiments, step S102, constructing a collaborative robot model and assembly environment on the target simulation platform, includes: defining the body structure, dynamic parameters and control interface of the collaborative robot on the target simulation platform; organizing the components of the collaborative robot through tag collaboration and constructing a hierarchical nesting relationship to establish a collaborative robot model; importing the three-dimensional model file of the corresponding assembly environment into the target simulation platform and setting the attribute parameters of the assembly environment to establish the assembly environment on the target simulation platform.

[0080] The attribute parameters can include position parameters, attitude parameters, and collision parameters.

[0081] In applications, the body structure of the collaborative robot can be defined using the configuration file of the target simulation platform. Each joint, link, end effector, and other component is organized through corresponding tags, and their hierarchical nesting relationship is constructed, setting dynamic parameters and control interfaces. Then, 3D modeling software can be used to pre-create 3D models of the geometry of the assembly workbench, parts, and tools, obtaining 3D model files. These files are then imported into the physical simulation environment of the target simulation platform. The mesh data of the 3D model files is called through the configuration file, and parameters such as position, mass, color, and collision attributes are set.

[0082] For example, the body structure of a collaborative robot is defined using MuJoCo's XML files, with each joint, link, end effector, and other component connected via... <geom>and <joint>The system organizes tags and establishes their hierarchical nesting relationships, setting dynamic parameters and control interfaces. 3D model files can be STL format mesh files, which are imported into the MuJoCo physics simulation environment. External mesh data is then called via XML configuration files, setting parameters such as position, mass, color, and collision attributes.

[0083] In some embodiments, step S102 defines the attributes of the digital human model, the collaborative robot model, and the entities in the assembly environment, including: obtaining the first position information and posture information of the digital human model and the collaborative robot model, as well as the second position information of the assembly parts; and defining the position, posture, and collision attributes of each entity based on the first position information, posture information, and second position information through the tags in the target simulation platform configuration file.

[0084] In this application, the positions of the digital human model, collaborative robot model, and assembly parts need to be rationally arranged. The position, pose, and collision attributes of each entity are defined using tags in the configuration file. By adjusting the position parameters of the digital human and the initial joint angles of its upper limbs, it is ensured that its upper limbs can complete the target actions and tasks within a defined range. The relative position of the collaborative robot's base on the workbench is set to ensure that it is in a non-contact state with the digital human model in the initial state. During collision detection, collisions between the digital human, robot, and parts are set as detectable to avoid physical interference or geometric overlap in the initial state.

[0085] For example, this can be achieved through the XML configuration file and <geom>The labels define the position, orientation, and collision attributes of each entity.

[0086] In one embodiment, the first state space includes the joint angles, joint velocities, muscle activation values, end-effector positions, and target distances of the digital human model. The first motion space is defined as the muscle activation signal composed of multidimensional continuous control variables.

[0087] The first reward function includes a target completion reward, a target distance reward, and a muscle energy consumption penalty. The target completion reward is positive when the digital human completes the target task and zero when the task is not completed. The target distance reward is obtained by calculating the Euclidean distance between the current position and the target position and taking the negative value. The muscle energy consumption penalty is the L2 norm average of the muscle activation signal per unit time.

[0088] The digital human model has a complete skeletal structure. The muscle drive system is mainly built on the right upper limb model of the digital human. Based on the muscle drive modeling and reinforcement learning strategy training of the right upper limb, the activation signals of the right upper limb muscles of the digital human model are controlled by reinforcement learning to generate the corresponding assembly action joint trajectory.

[0089] To comprehensively encompass the motion states, control signals, and relationship with the task objective of the digital human model during training, a multi-dimensional state space, known as the first state space, is constructed as the input to the policy network. This first state space contains information such as the digital human model's joint angles, joint velocities, muscle activation values, end-effector positions, and target distances, specifically defined as follows:

[0090]

[0091] in, and This represents the angles of the seven joints in the right upper limb of the musculoskeletal digital human model, along with the corresponding angular velocities. Represents the three-dimensional spatial coordinates of the target point. This indicates the current position coordinates of the hand's end in space. The distance between the palm and the target point is expressed in Euclidean form. This represents the 24-dimensional muscle activation signals of the right upper limb.

[0092] The motion space of the digital human, namely the first motion space, is defined as a muscle activation signal consisting of 24-dimensional continuous control variables, used to drive the corresponding muscles to generate muscle force, as specifically defined below:

[0093]

[0094] Each action Indicates the first A muscle in The degree of activation at any given moment.

[0095] To ensure that the motion trajectories generated by the musculoskeletal human model in the interactive environment have good physiological rationality and task completion effect, a composite reward function, namely the first reward function, was designed, as follows:

[0096]

[0097] in, This represents the reward for achieving the objective, used to guide the strategy to complete the objective task. Its specific definition is as follows:

[0098]

[0099] This represents a reward for distance to the target point, used to encourage the distal end of the arm to gradually approach the target point, improving the guidance and convergence speed of the control strategy. It is calculated by determining the current position. With the target location The Euclidean distance between them, with a negative value as the reward:

[0100]

[0101] This is a muscle energy expenditure penalty term, representing the amount of muscle activation signals per unit time. Normal average:

[0102]

[0103] in, This indicates the current activation value of the muscle. This value is used to punish unnecessary high-energy output and encourage the generation of low-energy, physiologically-compliant muscle activation signals.

[0104] In some embodiments, such as Figure 2 As shown, when the deep reinforcement learning algorithm is the SAC algorithm, in step S103, the human-machine collaborative assembly simulation platform is used as the interactive environment, and the digital human control policy network based on the deep reinforcement learning algorithm is trained based on the predefined state space, action space and reward function, including the following steps S201 to S204.

[0105] S201: Obtain the basic training parameters and the initial state parameters of the round.

[0106] The basic training parameters may include the success threshold for completing the digital human task, the maximum number of training rounds for the digital human's virtual motion trajectory, and the maximum number of steps per round. For example, during training, the success threshold for completing the digital human task can be set to 0.02m, the digital human's virtual motion trajectory can be trained for a maximum of 10,000 rounds, and each round can execute a maximum of 500 steps. When the task is completed, the round will terminate early. The initial state parameters for each round may include the initial joint angles of the digital human model's right upper limb.

[0107] S202: Set the initial joint angle of the right upper limb of the digital human model at the beginning of the training round based on the initial state parameters of the round, and introduce random perturbation.

[0108] In each round, at the beginning of the state, the initial joint angle of the right upper limb of the digital human model can be set, and a random perturbation of ±0.05 rad can be introduced on this basis to increase the diversity of states and improve the generalization ability of the strategy.

[0109] S203: Perform exploration operations for each training round until the iteration termination condition corresponding to the basic training parameters is reached. The exploration operations include: determining the current policy based on the current state space of the interactive environment, sampling the action corresponding to the muscle activation signal of the digital human's arm, executing the action in the interactive environment to control the movement of the digital human, calculating the reward value through the reward function and feeding it back to the next state space, and storing the interaction sample in the experience replay pool.

[0110] In applications, such as Figure 3 As shown, in each training iteration, the current policy Based on the current state space Actions that sample muscle activation signals from a digital human arm in a human-computer collaborative interactive environment. The system then executes this action within the environment to control the movement of the digital human model. The interactive environment calculates a reward value based on a reward function. And reward numerical values ​​and the next state space. These interaction samples It is stored in the experience replay pool for subsequent training of the network and policy updates.

[0111] S204: When the number of samples in the experience replay pool reaches a preset threshold, a network update operation is initiated after each interaction. The network update operation includes dual Critic network update and Actor network update.

[0112] In application, the Critic network uses a minimum double-Q structure to estimate the target value, avoiding training instability caused by Q-value overestimation. The target Q-value is defined by the following formula:

[0113]

[0114] in, The discount factor is set to 0.99. Entropy weights are used to balance policy entropy and expected return, and are dynamically adjusted using an automatic adjustment mechanism. To maintain the target entropy level; For the policy network in state The next step after downsampling. The optimization objective of the Critic network is to minimize the loss function:

[0115]

[0116] in, This represents the mathematical expectation summation symbol, which calculates the average value of the loss function for the states sampled in the experience pool.

[0117] Then, the Actor network updates based on the objective of maximizing reward and entropy, that is, encouraging the policy to choose actions with high Q-values ​​and high entropy, which is equivalent to minimizing the policy network loss function:

[0118]

[0119] in, This represents the entropy weighting coefficient, which controls the trade-off between exploration and reward maximization. The entropy represents the action; a larger value indicates a more random action. The estimated state-action values ​​for the current Critic network. The state sampling distribution in the experience pool.

[0120] To improve training stability and convergence speed, SAC uses the target network to perform soft updates on the Critic network. Through continuous iteration of the above process, the policy network gradually learns the motion strategies of the digital human during the assembly process, generating a digital human assembly trajectory that conforms to real human motion.

[0121] It is understandable that the SAC policy network gradually learns the optimal mapping relationship between states and actions. When training converges, the value of the reward function reaches a stable state, such as... Figure 4 As shown. The trained policy model is saved. After receiving the current environmental state, the trained policy network model can output corresponding muscle activation signals and action commands to drive the movement of each joint of the digital human's upper limbs, thereby generating the digital human's motion trajectory during the assembly process.

[0122] To verify the accuracy of the virtual digital human trajectory generation, real human motion videos were captured using a camera. Combined with a multi-camera calibration method, the actual motion trajectory was extracted using the Pose2Sim library, which is based on pose recognition and 3D reconstruction. This enabled pose recognition and 3D reconstruction of the real human motion videos captured by the camera. For the same assembly action, 10 sets of real human motion data and 10 sets of digital human motion trajectory data were collected. Five key joints of the right upper limb (including shoulder flexion and extension, shoulder adduction and abduction, shoulder rotation, elbow flexion and extension, and forearm pronation and supination) were aligned point-by-point. The mean and standard deviation of the angle data at different positions of each joint were calculated, obtaining the change curves of the real and virtual trajectories. The mean angle curves and their standard deviation bands of the real and virtual trajectories were then visualized and overlaid for comparison. Figure 5 As shown. To further analyze the differences between virtual trajectories and real human motion trajectories, the mean error and error range at each joint are analyzed, as shown below. Figure 6 As shown, the virtual joint trajectory generally follows the same trend as the real data, although some joint movements deviate to a certain extent. The overall error is mainly distributed between 0.1 and 0.2 rad, with a few joints showing slightly higher deviations at specific movement stages.

[0123] The comparison shows that the overall trend of the virtual human trajectory matches the distribution of real data, covering the main changes in human movement. This indicates that the reinforcement learning-based digital human motion method can generate assembly motion trajectories with reasonable realism and good adaptability. Looking at the generated joint motion trajectories, the joint angles exhibit smooth and continuous changes throughout the motion sequence. Each joint angle transitions smoothly over time without significant abrupt changes or jitter. The virtual trajectories of most joints maintain good consistency with the real trajectories in terms of overall trend and joint inflection points, reflecting the stability of the musculoskeletal system's coordinated control. The digital human motion trajectory reflects the coordinated control ability learned by the digital human model through muscle control, especially showing consistent activation patterns in joints such as shoulder elevation, elbow flexion and extension, and forearm rotation. The stable changes in joint angles demonstrate that the strategy model, driven by muscle activation signals, can produce motion outputs that conform to biological physiological laws. This verifies that the musculoskeletal digital human driven by muscle activation signals can achieve upper limb coordinated movements similar to those of a real human, and the proposed digital human trajectory generation method has a certain degree of representativeness and stability.

[0124] To verify the ergonomic rationality of the digital human assembly trajectory generated in this invention, the motion process of the digital human is analyzed based on Fitts' Law, a human factors ergonomics model. Fitts' Law is an empirical model describing the linear relationship between movement time (MT) and index of difficulty (ID) in point-to-point motion tasks. Its mathematical expression is:

[0125]

[0126] Where a and b are fitting coefficients, the formula for calculating the exercise difficulty index ID is:

[0127]

[0128] Where D is the target distance and W is the target width.

[0129] During the verification process, four typical motion difficulty levels (ID=1, 2, 3, 4) were pre-defined. For each ID value, different values ​​of D (10cm, 20cm, 30cm, 40cm) were selected, and the corresponding W was determined according to the motion difficulty index calculation formula. Using the trained digital human strategy model, assembly movements were simulated under different (D, W) task conditions to achieve movement from the starting position to the target position.

[0130] For each (D, W) parameter combination, 50 experimental tests were conducted, and the duration of movement (MT) was recorded for each test. At the same difficulty index (ID) level, the mean and standard deviation of the MT results for all (D, W) combinations were calculated as representative statistics for that ID. The least squares regression method was used to fit the linear relationship between MT and different IDs, yielding Fitts' law regression parameters a = 0.0285, b = 0.0434, and a coefficient of determination R² = 0.9920. The fitted curve is shown below. Figure 7 As shown in the figure, the box plot of MT distribution at each ID level is illustrated. The boxes represent the interquartile range, the midline represents the median, the whiskers represent the non-outlier range, the blue crosses represent the median MT values ​​used for fitting, and the red solid line represents the fitted regression curve, demonstrating the linear growth trend of MT with increasing ID. The results indicate that the digital human assembly motion trajectory generation method used in this invention conforms to the linear characteristics of Fitts' Law, and the motion of the digital human exhibits consistency and rationality with the kinematic and ergonomic characteristics of a real human.

[0131] In some embodiments, the second state space includes the collaborative robot's own state, motion target, and environmental constraints. The second motion space is defined as the incremental control quantities of the collaborative robot's six joints.

[0132] The second reward function includes a target distance penalty, a task completion reward, a first obstacle avoidance penalty, and a second obstacle avoidance penalty. The first obstacle avoidance penalty is for obstacle avoidance between the robot and the digital human, and the second obstacle avoidance penalty is for obstacle avoidance between the collaborative robot and a stationary obstacle. The target distance penalty is calculated based on the Euclidean distance between the robot's end effector and the target position. The task completion reward is positive when the collaborative robot completes the task and zero when it fails to complete the task. The first obstacle avoidance penalty is determined based on the human-robot safe distance threshold, the human-robot collision detection distance, and the minimum Euclidean distance between the collaborative robot and the digital human's upper limb. The second obstacle avoidance penalty is determined based on the collision safety distance and the minimum Euclidean distance between the collaborative robot and a stationary obstacle.

[0133] Using the trained digital human strategy model as a foundation, a dynamic obstacle avoidance training environment for collaborative robots is built. By loading the trained digital human strategy model to drive the movement of the digital human's upper limbs, and inputting this as a dynamic obstacle source into the collaborative robot training environment, the robot needs to consider the digital human's movement behavior during obstacle avoidance, thus forming a realistic and complex human-robot collaborative scenario.

[0134] Next, to ensure the collaborative robot can safely avoid obstacles during the assembly process with the digital human, a geometry-based minimum distance detection method is adopted. By calling the distance detection function in MuJoCo in real time, the minimum Euclidean distance between each link geometry of the collaborative robot from the base to the end effector and the geometry of the right upper limb skeleton (such as the shoulder, elbow, forearm, and wrist) of the digital human model is calculated, and the minimum value among all calculation results is taken as the minimum human-robot distance at the current moment. :

[0135]

[0136] in, The right upper limb of the digital human model is represented by the first... The coordinates of the position on the surface of the skeletal geometry geom where the minimum contact distance occurs between the geometry and the robot. In the collaborative robot model, the first... The coordinates of corresponding points on the surface of the link geometry geom are dynamically calculated by MuJoCo's built-in functions.

[0137] To achieve real-time obstacle avoidance control for collaborative robots in dynamic environments, it is necessary to rationally construct the state space and action space during the training process to accurately describe the robot's own state, task objectives, and key information about the external environment. This setup directly determines the comprehensiveness of the agent's environmental perception and the accuracy of its task decisions during reinforcement learning training. The second state space of the collaborative robot... Including its own state, the target of motion, and the constraints in the environment, it is defined as follows:

[0138]

[0139] in, and This represents the angles and angular velocities of the robot's six joints. Indicates the spatial position of the robot's end effector. Indicates the target location. Indicates the position of a stationary obstacle. This represents the Euclidean distance between the robot's end effector and the target. and This represents the minimum distance between the robot and the upper limbs of the digital human model, as well as surrounding stationary obstacles.

[0140] Next, the second motion space of the collaborative robot will be... Defined as the incremental control quantity of its 6 joints :

[0141]

[0142] To enable collaborative robots to perform dynamic obstacle avoidance assembly tasks in a shared space with digital humans, a composite reward function is designed that simultaneously considers task completion efficiency and safety. This reward function consists of four sub-items, measuring target proximity, task completion status, and the robot's obstacle avoidance safety, respectively. The overall reward function, also known as the second overall reward function, is defined as follows:

[0143]

[0144] in, This is a target distance penalty term. By penalizing the Euclidean distance between the robot's end effector and the target position, it guides the robot to approach the target position more quickly. As the robot gets closer to the target, this penalty value decreases, encouraging the actuator to move in the target direction, as follows:

[0145]

[0146] For task completion rewards, a substantial positive reward is given when the robot's end effector is close enough to the target, indicating task completion. Conversely, no reward is given if the target is not reached. Details are as follows:

[0147]

[0148] The obstacle avoidance penalty for the robot and the digital human is determined by the minimum distance between them, ensuring the robot doesn't get too close to the digital human model. If the robot and the digital human are too close but a collision doesn't occur, a linear penalty is applied; if they come into contact too close, a fixed large penalty is applied to avoid a learning tendency towards dangerous situations, as detailed below:

[0149]

[0150] in, This represents the minimum Euclidean distance between the collaborative robot and the digital human's upper limb; This indicates the human-machine safe distance threshold, which can be set to 0.1m; This indicates the human-machine collision detection distance, which can be set to 0.02m.

[0151] This is a penalty for obstacle avoidance between collaborative robots and stationary obstacles. No penalty is imposed when the distance between the collaborative robot and the stationary obstacle is greater than the safe distance; conversely, a significant negative reward is given to punish the dangerous behavior, as detailed below:

[0152]

[0153] in, This represents the minimum Euclidean distance between a collaborative robot and a stationary obstacle. This indicates the collision distance between collaborative robots and can be set to 0.

[0154] Training for obstacle avoidance strategies in collaborative robots, such as Figure 9 As shown, the SAC algorithm is also used as the reinforcement learning framework. The collaborative robot is based on the dynamic assembly environment and observes its own state, target position and minimum Euclidean distance between itself and digital humans and obstacles in real time to construct a suitable state space. The action space is defined as the incremental control of six joint angles to realize the output of continuous actions.

[0155] During training, the trained digital human motion strategy network is integrated into the environment as dynamic obstacles faced by the collaborative robot, and collision and distance detection are performed using MuJoCo's built-in functions. The success threshold for the collaborative robot to complete the task is set at 0.02m, and a total of 6000 training rounds are conducted, with a maximum of 500 steps per round. At the beginning of each training round, the initial posture of each joint of the collaborative robot is at a preset angle. A random perturbation of ±0.05 rad is added during initialization to improve the robustness of the strategy. The round ends early if the robot successfully reaches the target position or collides with it.

[0156] The SAC algorithm jointly optimizes the policy output actions by using a policy network (Actor) and two state-action value networks (Critic) to maximize the expected reward. The collaborative robot continuously samples the current state from the environment, generates actions based on the policy, executes the actions to obtain rewards, and then transitions to the next state, forming a tuple of state-action-reward-next state, which is stored in the experience replay pool. The Critic network updates its parameters using the target Q-value and the minimum double-Q policy, while the Actor network optimizes the entropy regularization objective of the policy distribution to improve the policy's randomness and robustness. An automatic entropy adjustment mechanism is employed, adaptively adjusting the entropy coefficient based on the difference between the current policy entropy and the target entropy.

[0157] Next, the collaborative robot gradually learns to adjust its movement strategy in real time based on the dynamic changes in the digital human's upper limb movements. At each moment, the robot dynamically makes decisions and updates its action output based on the perceived state space of the digital human's joints and their relative positions, thereby proactively avoiding potential collision risks. With each training iteration, the policy network iterates and optimizes during interaction with the environment, gradually improving its adaptability to complex human-robot collaborative scenarios. The reward value change curve is shown below. Figure 9 As shown, it can be observed that after several rounds of policy updates, the cumulative reward value tends to fluctuate steadily, indicating that the policy has basically converged. After 6000 rounds of training, the policy model is saved.

[0158] The changes in joint angles during dynamic obstacle avoidance by collaborative robots are as follows: Figure 10 As shown, the six joints of the collaborative robot exhibit a continuous and smooth trend of change during the dynamic obstacle avoidance task, without any sudden changes. This indicates that the trained policy network has good stability and can stably control the robot's joint movements.

[0159] Within a shared assembly space in a virtual environment, the entire process of collaborative robots and digital humans working together to complete assembly tasks is as follows: Figure 11 As shown, (a) represents the initial moment when the collaborative robot and the digital human are in their respective starting poses; in stages (b) and (c), the robot gradually approaches the assembly task area, while the digital human and the robot continuously move closer; in stages (d) and (e), the robot actively avoids the dynamic movements of the digital human and adjusts its trajectory during the assembly process; (f) represents the moment the task execution ends, with the human and robot in their respective end poses of the assembly task. The overall path change demonstrates the robot's ability to dynamically adjust to the dynamic behavior of the digital human. As the robot approaches the target, it actively avoids potential conflict paths with the digital human's upper limbs and selects safer action outputs within the permissible spatial range. The overall trajectory is relatively smooth, without any violent shaking or conflict behavior, providing support for safe collaborative assembly between the human and robot.

[0160] In some embodiments, please refer to Figure 12 This application provides a human-machine collaborative assembly trajectory generation device 120 based on digital humans, including: a first construction module 121, a second construction module 122, a first training module 123, and a second training module 124; wherein,

[0161] The first construction module 121 is used to construct a musculoskeletal digital human model on the target simulation platform based on the acquired human upper limb model and human simulation model.

[0162] The second building module 122 is used to build a collaborative robot model and assembly environment on the target simulation platform, and to define the attributes of entities in the digital human model, collaborative robot model and assembly environment to form a human-machine collaborative assembly simulation platform.

[0163] The first training module 123 is used to use the human-machine collaborative assembly simulation platform as an interactive environment to train a digital human control strategy network based on a deep reinforcement learning algorithm based on a predefined first state space, first action space and first reward function, and to generate the digital human assembly trajectory during the assembly process based on the trained digital human control strategy network.

[0164] The second training module 124 is used to introduce the trained digital human control strategy network into the human-machine collaborative assembly simulation platform to construct a dynamic assembly environment. The dynamic assembly environment is used as the training environment for the collaborative robot. Based on the predefined second state space, second action space and second reward function, the collaborative robot is trained to avoid obstacles by avoiding the dynamic actions of the digital human through a deep reinforcement learning algorithm. The robot obstacle avoidance strategy network is generated, and the robot assembly trajectory with dynamic obstacle avoidance capability is output based on the robot obstacle avoidance strategy network.

[0165] It should be noted that the human-machine collaborative assembly trajectory generation device 120 based on digital human provided in this application embodiment and the human-machine collaborative assembly trajectory generation method based on digital human provided in this application embodiment are based on the same application concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned human-machine collaborative assembly trajectory generation method based on digital human, and the repeated parts will not be described again.

[0166] In some embodiments, an electronic device provided in this application includes a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements the above-described human-machine collaborative assembly trajectory generation method based on digital humans.

[0167] Specifically, the processor may include, for example, a general-purpose microprocessor, an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor may also include onboard memory for caching purposes. The processor may be a single processing unit or multiple processing units for performing different actions of the method flow according to embodiments of this application.

[0168] Memory can be any medium capable of containing, storing, transmitting, propagating, or transmitting instructions. For example, memory can include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, instruments, or propagation media. Specific examples of memory include: magnetic storage devices such as magnetic tape or hard disk drives (HDDs); optical storage devices such as optical discs (CD-ROMs); and also random access memory (RAM) or flash memory; and / or wired / wireless communication links.

[0169] This application also provides a computer-readable medium storing a computer program thereon, which, when executed by a processor, implements the aforementioned method for generating human-machine collaborative assembly trajectories based on digital humans. This computer-readable medium may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not be assembled into that device / apparatus / system. The aforementioned computer-readable medium carries one or more programs, which, when executed, implement the method as described in the embodiments of this application.

[0170] According to embodiments of this application, a computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wired, optical fiber, radio frequency signals, etc., or any suitable combination thereof.

[0171] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application. Therefore, the scope of this application should not be limited to the above embodiments. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.< / geom> < / joint> < / geom>

Claims

1. A method for generating assembly trajectories based on digital humans through human-computer collaboration, characterized in that, include: Based on the acquired human upper limb model and human simulation model, a musculoskeletal digital human model is constructed on the target simulation platform; A collaborative robot model and assembly environment are constructed on the target simulation platform, and the attributes of the digital human model, the collaborative robot model, and the entities in the assembly environment are defined to form a human-machine collaborative assembly simulation platform. Using the human-machine collaborative assembly simulation platform as an interactive environment, a digital human control strategy network based on a deep reinforcement learning algorithm is trained based on a predefined first state space, first action space, and first reward function, and the digital human assembly trajectory during the assembly process is generated based on the trained digital human control strategy network. The trained digital human control strategy network is introduced into the human-machine collaborative assembly simulation platform to construct a dynamic assembly environment. The dynamic assembly environment is used as the training environment for the collaborative robot. Based on the predefined second state space, second action space and second reward function, the collaborative robot is trained to avoid obstacles by avoiding the dynamic actions of the digital human through a deep reinforcement learning algorithm. This generates a robot obstacle avoidance strategy network, and the robot assembly trajectory with dynamic obstacle avoidance capability is output based on the robot obstacle avoidance strategy network.

2. The human-machine collaborative assembly trajectory generation method based on digital humans as described in claim 1, characterized in that, The digital human model based on the acquired human upper limb model and human simulation model is constructed on the target simulation platform based on the musculoskeletal system, including: The acquired human upper limb model and human simulation model are converted to the target simulation platform to construct a musculoskeletal morphology model; A mathematical model of muscle activation and muscle force is constructed based on the Hill-type muscle model, and a dynamic model of the musculoskeletal morphology model is established. The musculoskeletal morphology model, the mathematical model of muscle activation and muscle force, and the dynamic model are integrated to form the digital human model.

3. The human-machine collaborative assembly trajectory generation method based on digital humans as described in claim 1, characterized in that, The construction of the collaborative robot model and assembly environment on the target simulation platform includes: The body structure, dynamic parameters, and control interface of the collaborative robot are defined on the target simulation platform. The components of the collaborative robot are organized by tag collaboration and a hierarchical nesting relationship is constructed to establish the collaborative robot model. Import the 3D model file of the corresponding assembly environment into the target simulation platform, set the attribute parameters of the assembly environment, and establish the assembly environment on the target simulation platform.

4. The human-machine collaborative assembly trajectory generation method based on digital humans as described in claim 1, characterized in that, The definition of attributes for the digital human model, the collaborative robot model, and the entities in the assembly environment includes: Acquire the first position and posture information of the digital human model and the collaborative robot model, as well as the second position information of the assembled parts; Based on the first position information, the attitude information, and the second position information, the position, attitude, and collision attributes of each entity are defined by the tags in the target simulation platform configuration file.

5. The human-machine collaborative assembly trajectory generation method based on digital humans as described in claim 1, characterized in that, The first state space includes the joint angles, joint velocities, muscle activation values, end-effector position, and target distance of the digital human model; The first action space is defined as a muscle activation signal composed of multidimensional continuous control variables; The first reward function includes a target completion reward, a target distance reward, and a muscle energy consumption penalty; the target completion reward is positive when the digital human completes the target task and zero when the task is not completed; the target distance reward is obtained by calculating the Euclidean distance between the current position and the target position and taking the negative value; the muscle energy consumption penalty is the L2 norm average of the muscle activation signal per unit time.

6. The human-machine collaborative assembly trajectory generation method based on digital humans as described in claim 5, characterized in that, The deep reinforcement learning algorithm is the SAC algorithm; the step of using the human-machine collaborative assembly simulation platform as the interactive environment and training a digital human control policy network based on the deep reinforcement learning algorithm according to a predefined state space, action space, and reward function includes: Obtain the basic training parameters and the initial state parameters of the round; Based on the initial state parameters of the round, the initial joint angle of the right upper limb of the digital human model is set at the beginning of the training round, and random perturbation is introduced; For each training round, an exploration operation is performed until the iteration termination condition corresponding to the training basic parameters is reached. The exploration operation includes: determining the current strategy based on the current state space of the interactive environment, sampling the action corresponding to the muscle activation signal of the digital human arm, executing the action in the interactive environment to control the movement of the digital human, calculating the reward value through the reward function and feeding it back to the next state space, and storing the interaction sample in the experience replay pool. When the number of samples in the experience replay pool reaches a preset threshold, a network update operation is initiated after each interaction. The network update operation includes dual Critic network update and Actor network update.

7. The method for generating human-machine collaborative assembly trajectory based on digital humans as described in any one of claims 1 to 6, characterized in that, The second state space includes the collaborative robot's own state, motion target, and environmental constraints; The second motion space is defined as the incremental control quantities of the six joints of the collaborative robot; The second reward function includes a target distance penalty, a task completion reward, a first obstacle avoidance penalty, and a second obstacle avoidance penalty. The first obstacle avoidance penalty is for obstacle avoidance between the collaborative robot and the digital human, and the second obstacle avoidance penalty is for obstacle avoidance between the collaborative robot and a stationary obstacle. The target distance penalty is calculated based on the Euclidean distance between the robot's end effector and the target position. The task completion reward is positive when the collaborative robot completes the task and zero when the task is not completed. The first obstacle avoidance penalty is determined based on the human-robot safe distance threshold, the human-robot collision detection distance, and the minimum Euclidean distance between the collaborative robot and the digital human's upper limb. The second obstacle avoidance penalty is determined based on the collision safety distance and the minimum Euclidean distance between the collaborative robot and a stationary obstacle.

8. A human-machine collaborative assembly trajectory generation device based on digital humans, characterized in that, include: The first building module is used to construct a musculoskeletal digital human model on the target simulation platform based on the acquired human upper limb model and human simulation model. The second construction module is used to build a collaborative robot model and assembly environment on the target simulation platform, and to define the attributes of the digital human model, the collaborative robot model and the entities in the assembly environment to form a human-machine collaborative assembly simulation platform. The first training module is used to use the human-machine collaborative assembly simulation platform as an interactive environment to train a digital human control strategy network based on a deep reinforcement learning algorithm based on a predefined first state space, first action space and first reward function, and to generate a digital human assembly trajectory during the assembly process based on the trained digital human control strategy network. The second training module is used to introduce the trained digital human control strategy network into the human-machine collaborative assembly simulation platform to construct a dynamic assembly environment. The dynamic assembly environment is used as the training environment for the collaborative robot. Based on the predefined second state space, second action space and second reward function, the collaborative robot is trained to avoid obstacles by using a deep reinforcement learning algorithm to avoid the dynamic actions of the digital human. The robot obstacle avoidance strategy network is generated, and the robot assembly trajectory with dynamic obstacle avoidance capability is output based on the robot obstacle avoidance strategy network.

9. An electronic device, characterized in that, It includes a processor and a memory; the memory stores a computer program, wherein the computer program, when executed by the processor, implements the human-machine collaborative assembly trajectory generation method based on a digital human as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that, It stores a computer program, wherein when the computer program is executed by a processor, it implements the human-machine collaborative assembly trajectory generation method based on a digital human as described in any one of claims 1 to 7.

Images