Robot control method and system based on long-term and short-term human motion intentions

Abstract

The invention discloses a robot control method and system based on long-term and short-term human motion intentions, and relates to the technical field of robot control. According to the method, based on a time sequence trajectory prediction module of a Transform neural network, multi-step accurate prediction of a short-term trajectory of human motion is realized. And establishing a dynamic probability distribution model of a human long-term moving target by adopting an iterative Bayesian probabilistic reasoning method to identify a human final target point. And through real-time updating of the contact force and the weight, the period changing intention is calculated. Through the synergistic effect of short-term time sequence trajectory prediction, long-term intention probabilistic reasoning and haptic feedback, accurate prediction of a human short-term motion trajectory, robust recognition of a long-term target intention and dynamic response to intention sudden change are realized, and the prediction accuracy of the short-term trajectory is ensured while the prediction accuracy of the long-term target intention is improved. The robustness of long-term target recognition and intention sudden change response is remarkably improved, the defects of an existing human motion intention prediction method are overcome, and the intelligence and efficiency in a man-machine cooperation task are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of robot control, and particularly to a robot control method and system based on long-term and short-term human motion intentions. Background Art

[0002] With the wide application of collaborative robot technology in fields such as industrial assembly and rehabilitation medicine, human-robot collaborative operations have put forward higher requirements for the real-time performance and accuracy of motion intention recognition. Traditional intention prediction methods based on fixed time windows or static models often struggle to cope with the inherent nonlinear and time-varying characteristics of human motion. Existing technologies mostly adopt trajectory prediction algorithms with a single time scale. Although this architecture can maintain a certain accuracy in short-term trajectory tracking, it cannot effectively model the long-period behavior patterns of human motion, resulting in lag and deviation in prediction results when facing complex operation scenarios. Especially in multi-objective switching tasks, static probability models lack the ability to dynamically model the potential preferences and decision-making processes of operators, making it difficult to stably accumulate the confidence of long-term intention recognition and seriously affecting the decision-making reliability of the collaborative system.

[0003] Although current human motion intention prediction models based on deep learning show advantages in processing time series data, they generally have adaptability defects to sudden intention changes. Most neural network models rely on offline pre-trained data. When an operator suddenly changes the motion trajectory due to environmental interference or task adjustment, the system often needs to be retrained or manually intervened to resume effective tracking. This mechanism defect is particularly prominent in collaborative scenarios that require continuous physical interaction, easily leading to robot response delays and even potential safety hazards. There is still a blank in existing research on real-time dynamic adjustment strategies.

[0004] In the dimension of long-term motion intention prediction, traditional Bayesian inference methods often use fixed prior distributions, ignoring the dynamic evolution law of human motion preferences with the progress of the task. This static modeling method is difficult to accurately capture the strategy transfer characteristics shown by the operator in multi-stage tasks, resulting in the update of the target probability distribution lagging behind the actual motion state, that is, insufficient prediction accuracy due to time-varying scenarios of human motion intentions, and lagging long-term target recognition leading to robot response delays. Summary of the Invention

[0005] In order to solve the technical problems of insufficient prediction accuracy and lagging long-term target recognition in time-varying scenarios of human motion intentions existing in the prior art, embodiments of the present invention provide a robot control method and system based on long-term and short-term human motion intentions. The technical solutions are as follows:

[0006] On the one hand, a robot control method based on long - short - term human motion intention is provided. This method is implemented by a robot control device based on long - short - term human motion intention. The robot control system based on long - short - term human motion intention includes a data processing platform, an admittance controller, and a robot;

[0007] The method includes:

[0008] S1. The data processing platform acquires the motion data set of the human hand during the human - robot collaborative handling process; the motion data set includes the contact force between the human hand and the robot;

[0009] S2. The data processing platform inputs the motion data set into a time - series trajectory prediction module constructed based on the Transformer neural network to obtain a short - term trajectory prediction result;

[0010] S3. The data processing platform determines multiple pending target points according to the motion data set and the prior knowledge base, and determines the target point among the multiple pending target points according to the dynamic probability distribution model constructed by the iterative Bayesian probability inference method to obtain a long - term intention prediction result;

[0011] S4. The data processing platform updates the shared control weights of the human and the robot according to the contact force, and calculates the deviation vector according to the shared control weights, the contact force, and the short - term trajectory prediction result;

[0012] S5. The data processing platform inputs the short - term trajectory prediction result, the deviation vector, and the long - term intention prediction result into the admittance controller. According to the short - term trajectory prediction result, the deviation vector, and the long - term intention prediction result, the admittance controller controls the motion of the robot.

[0013] On the other hand, a robot control system based on long - short - term human motion intention is provided. This system is applied to the robot control method based on long - short - term human motion intention. The system includes a data processing platform, an admittance controller, and a robot; wherein:

[0014] The data processing platform is used to acquire the motion data set of the human hand during the human - robot collaborative handling process; the motion data set includes the contact force between the human hand and the robot; input the motion data set into a time - series trajectory prediction module constructed based on the Transformer neural network to obtain a short - term trajectory prediction result; determine multiple pending target points according to the motion data set and the prior knowledge base, and determine the target point among the multiple pending target points according to the dynamic probability distribution model constructed by the iterative Bayesian probability inference method to obtain a long - term intention prediction result; update the shared control weights of the human and the robot according to the contact force, calculate the deviation vector according to the shared control weights, the contact force, and the short - term trajectory prediction result; input the short - term trajectory prediction result, the deviation vector, and the long - term intention prediction result into the admittance controller;

[0015] The admittance controller is used to control the movement of the robot according to the short-term trajectory prediction result, the deviation vector, and the long-term intention prediction result;

[0016] The robot is used to move according to the control of the admittance controller.

[0017] On the other hand, a robot control device based on long-term and short-term human motion intentions is provided. The robot control device based on long-term and short-term human motion intentions includes: a processor; a memory, and computer-readable instructions are stored on the memory. When the computer-readable instructions are executed by the processor, any one of the methods in the above-mentioned robot control method based on long-term and short-term human motion intentions is implemented.

[0018] On the other hand, a computer-readable storage medium is provided. At least one instruction is stored in the storage medium, and the at least one instruction is loaded and executed by a processor to implement any one of the methods in the above-mentioned robot control method based on long-term and short-term human motion intentions.

[0019] The beneficial effects brought by the technical solutions provided in the embodiments of the present invention at least include:

[0020] The time-series trajectory prediction module based on the Transformer neural network realizes multi-step accurate prediction of short-term human motion trajectories. Taking human motion preferences as prior modeling, an iterative Bayesian probability inference method is adopted to establish a dynamic probability distribution model of long-term human motion goals, and the human final target point is identified through a confidence accumulation mechanism. By real-time updating of contact force and weight, the variable-term intention is calculated. Through the synergistic effect of short-term time-series trajectory prediction, long-term intention probability inference, and force-tactile feedback, accurate prediction of short-term human motion trajectories, robust recognition of long-term target intentions, and dynamic response to intention mutations are realized. While ensuring the accuracy of short-term trajectory prediction, the robustness of long-term target recognition and intention mutation response is significantly improved, solving the deficiencies of existing human motion intention prediction methods, improving the intelligence and efficiency in human-robot collaboration tasks, and can be effectively applied to fields such as industrial assembly and rehabilitation medicine that require high-precision human-robot collaborative operations, improving the intelligence and safety of collaborative robots. Description of the Drawings

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for description in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0022] Figure 1It is a flowchart of a robot control method based on long - term and short - term human motion intentions provided by an embodiment of the present invention;

[0023] Figure 2 It is a schematic structural diagram of a human - robot collaborative handling scenario provided by an embodiment of the present invention;

[0024] Figure 3 It is a flowchart of long - term, short - term and variable intention prediction provided by an embodiment of the present invention;

[0025] Figure 4 It is a schematic diagram of the fusion of contact force and predicted trajectory provided by an embodiment of the present invention;

[0026] Figure 5 It is a block diagram of a robot control system based on long - term and short - term human motion intentions provided by an embodiment of the present invention;

[0027] Figure 6 It is a schematic structural diagram of a robot control device based on long - term and short - term human motion intentions provided by an embodiment of the present invention. Detailed implementation manners

[0028] Next, the technical solutions in the present invention will be described with reference to the accompanying drawings.

[0029] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design solution described as an "example" in the present invention should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of the word "example" is intended to present concepts in a specific manner. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one of the two.

[0030] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference is not emphasized, the meanings they express are the same. "(of)", "corresponding", and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference is not emphasized, the meanings they express are the same.

[0031] In the embodiments of the present invention, sometimes subscripts such as W1 may be written in a non - subscript form such as W1. When the difference is not emphasized, the meanings they express are the same.

[0032] To make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the following will be described in detail with reference to the accompanying drawings and specific embodiments.

[0033] An embodiment of the present invention provides a robot control method based on long - short - term human motion intentions. This method can be implemented by a robot control device based on long - short - term human motion intentions, and the robot control device based on long - short - term human motion intentions can be a terminal or a server. As Figure 1 shown in the flowchart of the robot control method based on long - short - term human motion intentions, the processing flow of this method can include the following steps:

[0034] S1. The data processing platform acquires the motion data set of the human hand during the human - robot collaborative handling process.

[0035] Among them, the motion data set includes the contact force between the human hand and the robot.

[0036] In a feasible implementation, the three - dimensional coordinate data of the human hand joint points are captured in real - time by a depth camera, and the human - robot contact force information is synchronously collected in combination with a six - dimensional force sensor. Then, these data are pre - processed to obtain the motion data set of the human hand.

[0037] The data pre - processing includes: converting the human motion trajectory from the camera coordinate system to the robot base coordinate system, eliminating noise interference through Kalman filtering, and segment - enhancing the trajectory data using a sliding time window to construct a multi - modal data set including spatio - temporal position, contact force amplitude, and direction.

[0038] Figure 2 depicts a human - robot collaborative handling scenario, where humans and robots jointly move an object to a specified target. In this case, the present invention defines the short - term intention as the expected position of the human hand in the next moment (i.e., the trajectory of the human hand), and the long - term intention as the final target of the object to be transported (i.e., target A, B, or C).

[0039] At the beginning of the task, the target point is unknown, may be known through prediction, and the human intention may change during the dynamic process. When the long - term intention is unknown, the robot must assist in the joint transportation under the guidance of the human. In addition, when the robot obtains the long - term intention through estimation, the robot should show greater initiative and may even play a leading role in the task.

[0040] The dynamic model of the robot is as follows in formula (1):

[0041]

[0042] where x o 、 respectively represent the position, velocity, and acceleration of the robot end - effector; M o ∈R nxn is the inertia matrix, C o ∈R nxn is the Coriolis matrix and centrifugal matrix, Go ∈R n is the gravitational vector, F h ∈R n is the force input by human, F r ∈R n is the force input by the robot, F vir ∈R n is the virtual gravitational force generated by the target.

[0043] The control input of the human arm is simply defined as a stiffness model, as shown in Equation (2) below:

[0044] F h = K h (x p - x i )(2)

[0045] where x p , x i ∈R n are the current position and the desired position of the human hand respectively. K h ∈R nxn is the stiffness matrix.

[0046] To achieve flexible transportation, the robot should exhibit spring-like characteristics under external forces. Therefore, the admittance controller shown in Equation (3) is considered in the embodiments of the present invention:

[0047]

[0048] where, F h ∈R n is the force input by human, that is, the contact force between the human and the robot in the embodiments of the present invention, F vir ∈R n is the virtual gravitational force generated by the target, that is, the virtual attractive force determined in the subsequent steps of the embodiments of the present invention, x d , are the position, velocity and acceleration of the desired trajectory respectively, which can be obtained from the short-term trajectory prediction results determined in the subsequent steps of the embodiments of the present invention; M r ∈R nxn , D r ∈R nxn , K r ∈R nxn are the desired inertia, damping and stiffness matrices respectively, x r , are the reference position, velocity and acceleration of the robot end effector respectively.

[0049] In this model, the inputs include the desired trajectory x d , the external force F h from the human, and the virtual force F vir, the output is the reference trajectory x r . As Figure 3 shown, the subsequent steps respectively determine the short-term trajectory prediction result, the long-term intention prediction result, and the variable-term intention prediction result.

[0050] S2. The data processing platform inputs the motion data set into the time-series trajectory prediction module constructed based on the Transformer neural network to obtain the short-term trajectory prediction result.

[0051] Optionally, the time-series trajectory prediction module constructed based on the Transformer neural network adopts a stacked encoder structure, including 8 hidden layers and 8 attention heads, uses the ReLU activation function, the input sequence length T is 50 historical time steps, the output sequence length is 20 time steps, and the output dimension is the same as the input dimension.

[0052] In a feasible implementation manner, considering the time series of the human hand movement trajectory, the embodiment of the present invention uses the Transformer network for short-term trajectory prediction. The input of the network includes the historical position X of the human hand = [x p (t), x p (t - T),..., x p (t - nT)], and the output is the predicted future position of the human hand x i , that is, a string of predicted human hand coordinates.

[0053] Calculate the spatio-temporal dependence weight of the joint trajectory through the multi-head attention mechanism, and the formula is expressed as the following formula (4):

[0054]

[0055] Among them, Q, K, and V are the query, key, and value matrices respectively, is the dimension scaling factor.

[0056] The loss function is defined as the mean square error between the predicted position x i (t) of the hand at the future time step and the actual position x p (t + mT). The loss function of the neural network is designed as the following formula (5):

[0057]

[0058] Among them, N is the number of training samples, x i (t) is the predicted position of the human hand at time t, and x p (t + mT) is the actual position of the human hand at time t + mT.

[0059] S3. The data processing platform determines multiple target points to be determined based on the motion data set and the prior knowledge base, and determines the target point from the multiple target points to be determined according to the dynamic probability distribution model constructed by the iterative Bayesian probability inference method, so as to obtain the long-term intention prediction result.

[0060] Optionally, the iterative Bayesian probability inference method satisfies the following relationship (6):

[0061] P(g h |x 0:n )∝P(g h |x 0:n-1 )P(x n |g h ,x 0:n-1 )∝P(g h |x 0:n-1 )P(x n |g h ,x n-1 )(6)

[0062] where x represents the position information corresponding to a time point, and the subscript represents time. x 0：n-1 represents all the positions from the 0th to the (n - 1)th moment, which together form a trajectory. P(g h |x 0:n-1 ) is the posterior distribution of the target g h at the (n - 1)th moment. P(x n |g h ,x 0:n-1 ) is the likelihood of the new observation x h given the target point g n and the historical trajectory. P(x n |g h ,x n-1 ) is the likelihood of the new observation x n after introducing the Markov assumption, ensuring that the target at the nth moment depends only on the target and the position x n-1 at the (n - 1)th moment.

[0063] Optionally, the dynamic probability distribution model is represented by a Gaussian mixture model. The Gaussian mixture model is the Gaussian distribution N(0,σ 2 ) of the deviation angle of the force from the predicted trajectory, where the mean of the Gaussian distribution is 0 and the standard deviation is σ. The likelihood P(x n |g h ,x n-1 ) is expressed as the following formula (7):

[0064]

[0065] where θ is the deviation angle of the force from the predicted trajectory.

[0066] In a feasible implementation, if the robot realizes the long-term intention of the human, the human can be freed from the task and the robot can execute the task with higher precision. The objective of the embodiments of the present invention is to infer the distribution of the target g h ∈G h by analyzing the historical trajectory.

[0067] Let the prior probability distribution of the target be P(h). Given the historical trajectory x 0:n , the objective of the embodiments of the present invention is to infer the posterior distribution of the target P(g h |x 0:n ). According to Bayes' theorem, the posterior distribution of the target can be expressed as the following formula (8):

[0068]

[0069] where P(x 0:n |g h ) is the likelihood of the trajectory given the target, and P(x 0:n ) is the evidence. To simplify the calculation, the embodiments of the present invention represent the posterior probability of the target as: P(g h |x 0:n ) ∝ P(x 0:n |g h )P(g h ). According to the iterative Bayes' idea, the embodiments of the present invention can also be expressed as the following formula (9):

[0070] P(g h |x 0:n ) ∝ P(g h |x 0:n-1 )P(x n |g h ,x 0:n-1 )(9)

[0071] where P(g h |x 0:n-1 ) is the posterior distribution of the target at the (n - 1)th moment, and P(x n |g h ,x 0:n-1 ) is the likelihood of the new observation x n given the target and the historical trajectory.

[0072] To reduce the computational complexity, the embodiments of the present invention introduce the Markov assumption, making the target at the nth moment depend only on the target and the position x n-1 at the (n - 1)th moment. The posterior probability of the target can be further expressed as the following formula (10):

[0073] P(g h |x 0:n ) ∝ P(g h|x 0:n-1 )P(x n |g h ,x n-1 )(10)

[0074] To calculate the likelihood P(x n |g h ,x n-1 ), the embodiments of the present invention use the result x of short-term intention estimation i to replace the position x at time n n , because using future trajectory points can better express human motion intention than current trajectory points. It is approximately a Gaussian distribution, with a mean of N(0,σ 2 ) with respect to the angle θ, where σ is the standard deviation of the prediction error. θ is defined as the angle between the vector g h -x i and the vector x i -x n-1 . Therefore, the likelihood P(x n |g h ,x n-1 ) can be expressed as the above formula (7).

[0075] Then, the result is normalized, the target with the highest posterior probability is selected, and the most likely target is determined

[0076] S4. The data processing platform updates the shared control weights of the human and the robot according to the contact force, and calculates the deviation vector according to the shared control weights, the contact force, and the short-term trajectory prediction result.

[0077] Optionally, the data processing platform updates the shared control weights of the human and the robot according to the contact force, including:[[]]

[0078] The data processing platform updates the shared control weight α of the human and the robot according to the contact force and the following formula (11):

[0079]

[0080] where F h is the contact force, and β is a sensitivity factor used to adjust the sensitivity of the shared control weight α to the contact force.

[0081] Optionally, calculating the deviation vector according to the shared control weight, the contact force, and the short-term trajectory prediction result includes:[[]]

[0082] Calculating the deviation vector t according to the shared control weight, the contact force, the short-term trajectory prediction result, and the following formula (12):

[0083]

[0084] where x i is the predicted position in the short-term trajectory prediction result, and x p is the actual position of the human hand.

[0085] In a feasible implementation, when an operator attempts to change the motion of an object during a collaborative task, human effort is the most direct indicator of the current change state. However, it is insufficient in predicting future changes. To better predict human-robot interaction, it is beneficial to combine human effort with the short-term human-robot interaction prediction discussed earlier.

[0086] The direction vector t ∈ R n is defined as the convex combination of human force and short-term intention, and the expression is as shown in Equation (12) above.

[0087] A schematic diagram of the human motion direction t is as shown in Figure 4 . Let g i represent the target point that this person expects to reach. Assume that the expected goal in the first stage of the transportation task is g1, and the short-term prediction direction will naturally converge to g1.

[0088] As the short-term prediction accuracy improves, the resistance to changing the expected goal also intensifies. Since force provides a real-time reflection of the current expected direction of a human, combining these two factors can more accurately represent the human motion intention. When an object obscures the hand and vision fails, force can still provide feedback on the human motion intention.

[0089] In addition, the θ i angle represents the divergence between the shortest path from the current position to the target point and x p - x i and will be used for subsequent long-term intention analysis.

[0090] Then t is normalized to obtain the required human motion direction. The coefficient α ∈ [0, 1] varies over time and is used to adjust the relative influence between the robot and the human. The value of α is defined as shown in Equation (11) above.

[0091] If the force is relatively large, α > 0.5 indicates that the human is in control of the task and the robot follows the human's adjustment of the object's motion. On the contrary, when the force is small (α < 0.5), the robot takes control of the task, and the human mainly assists by supporting the load of the object. At the same time, the magnitude of α can also reflect the urgency of the change in the human motion intention. The larger α is, the faster t converges to the direction of the acting force.

[0092] It should be noted that there is a certain delay in the inference process of the neural network, which may lead to the untimely update of the expected position x d of the robot. When α fluctuates around 0.5, human effort can be used to subtly adjust the delay introduced by this lag, thereby reducing its impact on task performance.

[0093] Once the desired movement direction of the human is determined, the robot can adjust its desired position x d = x o + th, where h is a fixed step size. d

[0094] In this way, for the case of sudden changes in the movement intention of the human in the experiment, the embodiment of the present invention adopts an online adaptive strategy using real-time force feedback, realizing the autonomous tracking of the changing dynamic movement intention of the human by the robot without pre-training.

[0095] S5. The data processing platform inputs the short-term trajectory prediction result, the deviation vector, and the long-term intention prediction result into the admittance controller. According to the short-term trajectory prediction result, the deviation vector, and the long-term intention prediction result, the admittance controller controls the movement of the robot.

[0096] Optionally, the specific operation process of S5 is as follows:

[0097] Update the desired trajectory according to the short-term trajectory prediction result and the deviation vector, adjust the virtual attraction according to the long-term intention prediction result, and the admittance controller controls the movement of the robot according to the desired trajectory with the assistance of the virtual attraction.

[0098] The expression of the virtual attraction is the following formula (13):

[0099]

[0100] where g h is the target point of the long-term intention prediction result, x o is the end pose of the current robot, K vir represents the virtual stiffness coefficient, P thr represents the threshold probability of the desired target, α thr represents the weight threshold, and α thr is the threshold of α.

[0101] In a feasible implementation manner, after obtaining the required target point, the virtual force F vir is defined as the following formula (14):

[0102]

[0103] where K vir is the virtual stiffness coefficient, and P thr is the threshold probability.

[0104] ​If the target point is not determined in advance during the experiment and the corresponding trajectory is not trained, the accuracy of short-term prediction may be compromised. Therefore, the value of the weight α will be relatively large, which can be used to provide a rough estimate of the potential target. In this case, it is more appropriate to set the virtual force to zero. Therefore, formula (14) can be updated to the above formula (13). The setting of the virtual force ensures that the robot can still move toward the target autonomously when the external force of the human hand is withdrawn.

[0105] In an embodiment of the present invention, a time series trajectory prediction module based on a Transformer neural network is used to achieve multi-step accurate prediction of short-term human motion trajectories. Human motion preferences are used as a priori modeling, and an iterative Bayesian probabilistic reasoning method is used to establish a dynamic probability distribution model of human long-term motion goals, and the final target point of humans is identified through a confidence accumulation mechanism. The variable period intention is calculated by real-time updating of contact force and weight. Through the synergistic effect of short-term time series trajectory prediction, long-term intention probability reasoning and force tactile feedback, accurate prediction of human short-term motion trajectory, robust recognition of long-term target intentions and dynamic response to intention mutations are achieved. While ensuring the accuracy of short-term trajectory prediction, the robustness of long-term target recognition and intention mutation response is significantly improved, which solves the shortcomings of existing human motion intention prediction methods, improves the intelligence and efficiency in human-machine collaborative tasks, and can be effectively applied to industrial assembly, rehabilitation medicine and other fields that require high-precision human-machine collaborative operations, and improves the intelligence and safety of collaborative robots.

[0106] Figure 5 1 is a block diagram of a robot control system based on long-term and short-term human motion intentions provided by an embodiment of the present invention, and the system is used for a robot control method based on long-term and short-term human motion intentions. Figure 5 The system comprises:

[0107] The data processing platform 510 is used to obtain a motion data set of a human hand during human-robot collaborative handling; the motion data set includes the contact force between the human hand and the robot; the motion data set is input into a time series trajectory prediction module constructed based on a Transformer neural network to obtain a short-term trajectory prediction result; multiple undetermined target points are determined based on the motion data set and a priori knowledge base, and a target point is determined from multiple undetermined target points based on a dynamic probability distribution model constructed by an iterative Bayesian probability reasoning method to obtain a long-term intention prediction result; the shared control weight of the human and the robot is updated based on the contact force, and a deviation vector is calculated based on the shared control weight, the contact force and the short-term trajectory prediction result; the short-term trajectory prediction result, the deviation vector and the long-term intention prediction result are input into an admittance controller;

[0108] The admittance controller 520 is used to control the robot motion according to the short-term trajectory prediction result, the deviation vector and the long-term intention prediction result;

[0109] The robot 530 is configured to move according to the control of an admittance controller.

[0110] In an embodiment of the present invention, a temporal trajectory prediction module based on a Transformer neural network is used to achieve multi-step accurate prediction of short-term trajectories of human movements. The motion preferences of a person are modeled as a prior, and an iterative Bayesian probabilistic inference method is adopted to establish a dynamic probability distribution model of the long-term motion goals of a human. The final target point of a human is identified through a confidence accumulation mechanism. The variable-term intention is calculated through real-time updates of contact forces and weights. Through the coordinated action of short-term temporal trajectory prediction, long-term intention probabilistic inference, and force-tactile feedback, accurate prediction of short-term human motion trajectories, robust recognition of long-term target intentions, and dynamic response to intention mutations are achieved. While ensuring the accuracy of short-term trajectory prediction, the robustness of long-term target recognition and intention mutation response is significantly improved, solving the deficiencies of existing human motion intention prediction methods, improving the intelligence and efficiency in human-robot collaboration tasks, and can be effectively applied to fields such as industrial assembly and rehabilitation medicine that require high-precision human-robot collaborative operations, improving the intelligence and safety of collaborative robots.

[0111] Figure 6 is a schematic structural diagram of a robot control device based on long-term and short-term human motion intentions provided by an embodiment of the present invention, as Figure 6 shown, the robot control device based on long-term and short-term human motion intentions may include the above-mentioned Figure 5 shown robot control system based on long-term and short-term human motion intentions. Optionally, the robot control device 510 based on long-term and short-term human motion intentions may include a first processor 2001.

[0112] Optionally, the robot control device 510 based on long-term and short-term human motion intentions may further include a memory 2002 and a transceiver 2003.

[0113] Among them, the first processor 2001, the memory 2002, and the transceiver 2003 may be connected through a communication bus, for example.

[0114] Next, in conjunction with Figure 6 each component of the robot control device 510 based on long-term and short-term human motion intentions will be specifically introduced:

[0115] Among them, the first processor 2001 is the control center of the robot control device 510 based on long-term and short-term human motion intentions. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or it can be an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. For example: one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs).

[0116] Optionally, the first processor 2001 can execute various functions of the robot control device 510 based on long-term and short-term human motion intentions by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0117] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 6 the CPU0 and CPU1 shown in

[0118] In a specific implementation, as an embodiment, the robot control device 510 based on long-term and short-term human motion intentions may also include multiple processors, such as Figure 6 the first processor 2001 and the second processor 2004 shown in. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, the processor can refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0119] Among them, the memory 2002 is used to store the software program for implementing the solution of the present invention and is controlled by the first processor 2001 for execution. The specific implementation method can refer to the above method embodiments and will not be elaborated here.

[0120] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or may also be an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently and be coupled to the first processor 2001 through an interface circuit ( Figure 6 not shown) of the robot control device 510 based on long-term and short-term human motion intentions. The embodiments of the present invention do not make specific limitations in this regard.

[0121] The transceiver 2003 is used to communicate with a network device or with a terminal device.

[0122] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 6 not shown separately). Among them, the receiver is used to implement the receiving function, and the transmitter is used to implement the sending function.

[0123] Optionally, the transceiver 2003 may be integrated with the first processor 2001 or may exist independently and be coupled to the first processor 2001 through an interface circuit ( Figure 6 not shown) of the robot control device 510 based on long-term and short-term human motion intentions. The embodiments of the present invention do not make specific limitations in this regard.

[0124] It should be noted that Figure 6 the structure of the robot control device 510 based on long-term and short-term human motion intentions shown in does not constitute a limitation to the router. The actual knowledge structure recognition device may include more or fewer components than shown in the figure, or combine some components, or have different component arrangements.

[0125] In addition, the technical effects of the robot control device 510 based on long-term and short-term human motion intentions may refer to the technical effects of the robot control method based on long-term and short-term human motion intentions described in the above method embodiments, and will not be elaborated here.

[0126] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0127] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0128] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware, or any arbitrary combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable systems. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that contains one or more collections of available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, or a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0129] It should be understood that the term "and / or" in this document is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Here, A and B can be singular or plural. In addition, the character " / " in this document generally represents an "or" relationship between the associated objects before and after, but it may also represent an "and / or" relationship, which can be specifically understood with reference to the context before and after.

[0130] In the present invention, "at least one" means one or more, and "a plurality" means two or more. "At least one of the following" or its similar expressions refer to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, and c can be single or multiple.

[0131] It should be understood that in various embodiments of the present invention, the magnitudes of the sequence numbers of the above processes do not imply the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.

[0132] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Skilled professionals can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of the present invention.

[0133] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described devices, systems, and units can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0134] In several embodiments provided by the present invention, it should be understood that the disclosed devices, systems, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of systems or units can be in electrical, mechanical, or other forms.

[0135] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0136] In addition, the functional units in each embodiment of the present invention can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0137] When the above-described functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs that can store program codes.

[0138] As described above, the above are only specific implementation manners of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A robot control method based on long-term and short-term human motion intentions, characterized in that, The robot control method based on long - short - term human motion intention is implemented by a robot control system based on long - short - term human motion intention. The robot control system based on long - short - term human motion intention includes a data processing platform, an admittance controller, and a robot; The method includes: S1. The data processing platform acquires the motion data set of the human hand during the human - robot collaborative handling process; the motion data set includes the contact force between the human hand and the robot; S2. The data processing platform inputs the motion data set into the time - series trajectory prediction module constructed based on the Transformer neural network to obtain the short - term trajectory prediction result; S3. The data processing platform determines multiple pending target points according to the motion data set and the prior knowledge base, and determines the target point among the multiple pending target points according to the dynamic probability distribution model constructed by the iterative Bayesian probability inference method to obtain the long - term intention prediction result; S4. The data processing platform updates the shared control weights of the human and the robot according to the contact force, and calculates the deviation vector according to the shared control weights, the contact force, and the short - term trajectory prediction result; S5. The data processing platform inputs the short - term trajectory prediction result, the deviation vector, and the long - term intention prediction result into the admittance controller. According to the short - term trajectory prediction result, the deviation vector, and the long - term intention prediction result, the admittance controller controls the movement of the robot.

2. The robot control method based on long-short-term human motion intention according to claim 1, wherein The time - series trajectory prediction module constructed based on the Transformer neural network adopts a stacked encoder structure, includes 8 hidden layers and 8 attention heads, uses the ReLU activation function, the input sequence length T is 50 historical time steps, the output sequence length is 20 time steps, and the output dimension is the same as the input dimension.

3. The robot control method based on long short-term human motion intention according to claim 1, wherein The iterative Bayesian probability inference method satisfies the following relational expression (1): P(g h |x 0:n ) ∝ P(g h |x 0:n-1 ) P(x n |g h , x 0:n-1 ) ∝ P(g h |x 0:n-1 ) P(x n |g h , x n-1 ) (1) Among them, x represents the position information corresponding to a time point, and the subscript represents time. x 0：n-1 represents all positions from the 0th to the (n - 1)th moment. P(g h |x 0:n-1 ) is the posterior distribution of the target g h at the (n - 1)th moment. P(x n |g h ,x 0:n-1 ) is the likelihood of the new observation x h given the target point g n and the historical trajectory. P(x n |g h ,x n-1 ) is the likelihood of the new observation x n after introducing the Markov assumption, ensuring that the target at the nth moment only depends on the target and the position x n-1 .

4. The robot control method based on long - short - term human motion intention according to claim 3, characterized in that, The dynamic probability distribution model is represented by a Gaussian mixture model. The Gaussian mixture model is the Gaussian distribution N(0,σ 2 ) of the deviation angle between the force and the predicted trajectory. The mean of the Gaussian distribution is 0, and the standard deviation is σ. The likelihood P(x n |g h ,x n-1 ) is expressed as the following formula (2): where θ is the deviation angle between the force and the predicted trajectory.

5. The robot control method based on long-short-term human motion intention according to claim 1, wherein, The data processing platform updates the shared control weights of the human and the robot according to the contact force, including: The data processing platform updates the shared control weight α of the human and the robot according to the contact force and the following formula (3): Among them, F h is the contact force, and β is a sensitivity factor used to adjust the sensitivity of the shared control weight α to the contact force.

6. The robot control method based on long - short - term human motion intention according to claim 5, wherein, The calculation of the deviation vector according to the shared control weights, the contact force, and the short - term trajectory prediction result includes: Calculating the deviation vector t according to the shared control weights, the contact force, the short - term trajectory prediction result, and the following formula (4): where x i is the predicted position in the short-term trajectory prediction result, and x p is the actual position of the human hand.

7. The robot control method based on long - short - term human motion intention according to claim 1, wherein, The admittance controller controls the movement of the robot according to the short - term trajectory prediction result, the deviation vector, and the long - term intention prediction result, including: Updating the desired trajectory according to the short - term trajectory prediction result and the deviation vector, adjusting the virtual attraction according to the long - term intention prediction result. The admittance controller controls the movement of the robot according to the desired trajectory with the assistance of the virtual attraction; The expression of the virtual attraction is the following formula (5): Among them, g h is the target point of the long-term intention prediction result, x o is the end pose of the current robot, K vir represents the virtual stiffness coefficient, P thr represents the threshold probability of the expected target, α thr represents the weight threshold, α thr is the threshold of α.

8. A robot control system based on long-term and short-term human motion intentions, the robot control system based on long-term and short-term human motion intentions is used to implement the robot control method based on long-term and short-term human motion intentions as described in any one of claims 1-7, characterized in that The system includes a data processing platform, an admittance controller, and a robot; where: The data processing platform is used to obtain the motion data set of the human hand during the human-robot collaborative handling process; the motion data set includes the contact force between the human hand and the robot; input the motion data set into the time-series trajectory prediction module constructed based on the Transformer neural network to obtain the short-term trajectory prediction result; determine multiple pending target points according to the motion data set and the prior knowledge base, and determine the target point among the multiple pending target points according to the dynamic probability distribution model constructed by the iterative Bayesian probability inference method to obtain the long-term intention prediction result; update the shared control weights of the human and the robot according to the contact force, and calculate the deviation vector according to the shared control weights, the contact force and the short-term trajectory prediction result; input the short-term trajectory prediction result, the deviation vector and the long-term intention prediction result into the admittance controller; The admittance controller is used to control the robot motion according to the short-term trajectory prediction result, the deviation vector and the long-term intention prediction result; The robot is used to move according to the control of the admittance controller.

9. A robot control device based on long-term and short-term human motion intentions, characterized in that, The robot control device based on long-term and short-term human motion intentions includes: A processor; A memory, on which computer-readable instructions are stored. When the computer-readable instructions are executed by the processor, the method described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that, Program code is stored in the computer-readable storage medium, and the program code can be called by the processor to execute the method described in any one of claims 1 to 7.

Images