Motion trajectory prediction data processing method and device, electronic equipment and storage medium

Abstract

The application provides a motion trajectory prediction data processing method and device, electronic equipment and storage medium, relates to the technical field of digital data processing, and through a data processing process of phase perception, hybrid modeling, uncertainty quantification and joint optimization, not only can the accuracy and physical consistency of motion trajectory prediction be greatly improved, but also the downstream control can adaptively adjust the safety strategy according to the predicted confidence, significantly reduces the collision risk in the scene of human-machine cooperation, and improves the safety and adaptability of the robot. The method can select one or more motion trajectory prediction sub-models corresponding to the dominant motion phase from the pre-constructed model library, can select the most suitable motion trajectory prediction sub-model for different motion phases, and can quickly adapt to different target objects or motion modes. In addition, the data processing process of the method does not depend on specific hardware and can be widely applied to fields requiring timing prediction and decision-making cooperation.

Description

Technical Field

[0001] This invention relates to the field of electronic digital data processing technology, and in particular to a motion trajectory prediction data processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the rapid development of the Internet of Things, wearable computing, embodied AI, and robotics, accurately and in real-time predicting the future movement trajectory of a target object has become a core data processing task for many critical applications. For example, in human-robot collaboration scenarios, robots need to predict pedestrian movements to safely avoid them; in rehabilitation exoskeleton control scenarios, robots need to provide timely assistance based on the patient's current gait trends.

[0003] In terms of technical implementation, existing prediction methods for time-series motion data mainly suffer from the following data processing deficiencies:

[0004] First, existing methods mostly employ a single statistical learning model or a single physical model. Statistical models lack physical constraints on the laws of motion, and the prediction results may not conform to physical laws, thus belonging to black box prediction; while physical models are difficult to handle complex and variable individual behavior patterns.

[0005] Secondly, existing deep learning-based action recognition methods mostly output deterministic categories or trajectories, which prevents downstream decision-making systems from adjusting their strategies, posing a significant risk in safety-critical applications such as assisted driving and human-machine collaboration.

[0006] Finally, current data processing workflows typically separate the prediction module from the control module, with the prediction module's output serving only as a fixed input reference for the control module, failing to create a data-driven closed-loop optimization. For example, in robot following scenarios, prediction errors are directly transmitted to downstream controllers, which can severely impact the overall system's reliability, safety, and adaptability. Summary of the Invention

[0007] This invention provides a method, apparatus, electronic device, and storage medium for processing motion trajectory prediction data, in order to overcome the deficiencies existing in related technologies.

[0008] This invention provides a method for processing motion trajectory prediction data, comprising:

[0009] Acquire multi-source temporal motion data of the target object at the current moment, and based on the multi-source temporal motion data, identify the dominant motion phase of the target object at the current moment;

[0010] One or more motion trajectory prediction sub-models corresponding to the dominant motion phase are selected from the pre-built model library, and the motion trajectory prediction sub-models are applied based on the multi-source time-series motion data to generate the motion trajectory prediction results of the target object at future times; the model library includes data-driven sub-models and physics-driven sub-models;

[0011] Uncertainty quantification is performed on the prediction results output by the data-driven sub-model to obtain the probability distribution parameters of the prediction results output by the data-driven sub-model.

[0012] Based on the predicted motion trajectory and the probability distribution parameters, a model prediction control optimization problem is constructed.

[0013] The model predictive control optimization problem is solved online to obtain the optimal control command for the control objective.

[0014] According to a motion trajectory prediction data processing method provided by the present invention, the step of selecting one or more motion trajectory prediction sub-models corresponding to the dominant motion phase from a pre-built model library includes:

[0015] If the dominant motion phase is the gait support phase, then the motion trajectory prediction sub-model is determined to be the physical driving sub-model, and the physical parameters of the physical driving sub-model are estimated online based on the historical motion data of the target object;

[0016] If the dominant motion phase is the gait swing phase, then the motion trajectory prediction sub-model is determined to be the data-driven sub-model, which is a pre-trained Transformer encoder-decoder model.

[0017] According to the present invention, a motion trajectory prediction data processing method is provided, wherein the data-driven sub-model is obtained by end-to-end training based on motion time series data samples with true trajectory labels and applying a composite loss function;

[0018] The composite loss function includes prediction error loss, physical consistency loss, and uncertainty calibration loss. The prediction error loss is calculated based on the predicted trajectory data output by the data-driven sub-model and the true trajectory label.

[0019] The physical consistency loss is calculated based on the predicted acceleration and the theoretical acceleration. The predicted acceleration is determined based on the second derivative of the predicted trajectory data with respect to time, and the theoretical acceleration is determined based on the predicted trajectory data by applying parameterized dynamic equations.

[0020] The uncertainty calibration loss is determined based on the initial diagonal covariance matrix output by the data-driven sub-model, the predicted trajectory data, and the ground truth trajectory label.

[0021] According to the present invention, a motion trajectory prediction data processing method is provided, wherein the probability distribution parameters include the prediction mean and the prediction diagonal covariance matrix;

[0022] The uncertainty quantification of the prediction results output by the data-driven sub-model to obtain the probability distribution parameters of the prediction results output by the data-driven sub-model includes:

[0023] Based on the multi-source temporal motion data, the data-driven sub-model performs multiple random forward propagations to obtain multiple prediction results and the initial diagonal covariance matrix corresponding to each prediction result.

[0024] The mean of the multiple prediction results is calculated as the prediction mean, and the diagonal covariance matrix of the multiple prediction results is calculated as the divergence diagonal covariance matrix. Based on the divergence diagonal covariance matrix and the initial diagonal covariance matrix corresponding to each prediction result, the prediction diagonal covariance matrix is ​​determined.

[0025] According to a motion trajectory prediction data processing method provided by the present invention, the cost function of the model predictive control optimization problem includes a tracking error term, a control cost term, and an uncertainty penalty term; the step of constructing the model predictive control optimization problem based on the motion trajectory prediction result and the probability distribution parameters includes:

[0026] The tracking error term is calculated based on the difference between the predicted motion trajectory result and the predicted mean in the probability distribution parameters.

[0027] The control cost term is calculated based on the sequence of control instructions to be optimized;

[0028] The uncertainty penalty term is determined based on the trace of the predicted diagonal covariance matrix in the probability distribution parameters.

[0029] According to the motion trajectory prediction data processing method provided by the present invention, the cost function of the model predictive control optimization problem further includes a safety risk term;

[0030] The step of constructing a model prediction control optimization problem based on the motion trajectory prediction result and the probability distribution parameters further includes:

[0031] Based on the trajectory point position data in the multi-source temporal motion data, the actual distance between the control target and the target object is determined.

[0032] The safety risk term is determined based on the difference between the actual distance and the minimum absolute safe distance, and the trace of the predicted diagonal covariance matrix in the probability distribution parameters.

[0033] According to a motion trajectory prediction data processing method provided by the present invention, the step of identifying the dominant motion phase of the target object at the current moment based on the multi-source time-series motion data includes:

[0034] The multi-source temporal motion data is preprocessed to obtain a standardized input sequence;

[0035] The standardized input sequence is input into the phase recognition model to obtain the dominant motion phase output by the phase recognition model.

[0036] The present invention also provides a motion trajectory prediction data processing device, comprising:

[0037] The phase recognition module is used to acquire multi-source temporal motion data of the target object at the current moment, and based on the multi-source temporal motion data, identify the dominant motion phase of the target object at the current moment;

[0038] The trajectory prediction module is used to select one or more motion trajectory prediction sub-models corresponding to the dominant motion phase from a pre-built model library, and apply the motion trajectory prediction sub-models based on the multi-source time-series motion data to generate the motion trajectory prediction result of the target object at a future time; the model library includes data-driven sub-models and physics-driven sub-models.

[0039] The uncertainty quantification module is used to quantify the uncertainty of the prediction results output by the data-driven sub-model to obtain the probability distribution parameters of the prediction results output by the data-driven sub-model.

[0040] The problem construction module is used to construct a model prediction control optimization problem based on the motion trajectory prediction results and the probability distribution parameters.

[0041] The problem-solving module is used to solve the model predictive control optimization problem online to obtain the optimal control command for the control objective.

[0042] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the motion trajectory prediction data processing method as described above.

[0043] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the motion trajectory prediction data processing method as described above.

[0044] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the motion trajectory prediction data processing method as described above.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] The motion trajectory prediction data processing method, apparatus, electronic device, and storage medium provided by this invention, through a data processing flow of phase sensing, hybrid modeling, uncertainty quantification, and joint optimization, can not only significantly improve the accuracy and physical consistency of motion trajectory prediction, but also enable downstream control to adaptively adjust safety strategies based on the prediction confidence level, significantly reducing collision risks in human-robot collaboration scenarios and improving robot safety and adaptability. This method selects one or more motion trajectory prediction sub-models corresponding to the dominant motion phase from a pre-built model library, allowing for the selection of the most suitable motion trajectory prediction sub-model for different motion phases, thus enabling rapid adaptation to different target objects or motion patterns. Furthermore, the data processing flow of this method is not dependent on specific hardware and can be widely applied in fields requiring time-series prediction and decision-making collaboration, such as robot following, autonomous driving trajectory planning, industrial process control, and intelligent rehabilitation assessment. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in this invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart illustrating the motion trajectory prediction data processing method provided by the present invention.

[0049] Figure 2 This is a schematic diagram of the motion trajectory prediction data processing device provided by the present invention.

[0050] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0052] Figure 1 This is a flowchart illustrating a motion trajectory prediction data processing method provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes:

[0053] S1, acquire multi-source temporal motion data of the target object at the current moment, and identify the dominant motion phase of the target object at the current moment based on the multi-source temporal motion data;

[0054] S2, Select one or more motion trajectory prediction sub-models corresponding to the dominant motion phase from the pre-built model library, and apply the motion trajectory prediction sub-models based on the multi-source time-series motion data to generate the motion trajectory prediction result of the target object at future times; the model library includes data-driven sub-models and physics-driven sub-models;

[0055] S3, perform uncertainty quantification on the prediction results output by the data-driven sub-model to obtain the probability distribution parameters of the prediction results output by the data-driven sub-model;

[0056] S4. Based on the motion trajectory prediction results and the probability distribution parameters, construct a model prediction control optimization problem;

[0057] S5. Solve the model predictive control optimization problem online to obtain the optimal control command for the control objective.

[0058] Specifically, the motion trajectory prediction data processing method provided in this embodiment of the invention is executed by a motion trajectory prediction data processing device. This device can be configured within the controller of a control target, which can be a robot in a human-machine collaboration scenario or a rehabilitation exoskeleton control scenario, a vehicle in an autonomous driving scenario, or industrial equipment in an industrial process control scenario, etc. The controller can be an embedded computing unit (NVIDIA Jetson AGXXavier).

[0059] In this embodiment of the invention, the software module implementing the method can be compiled into a function package of the operating system (ROS).

[0060] First, execute step S1 to obtain the multi-source temporal motion data of the target object at the current moment. Multi-source temporal motion data refers to a data stream acquired from multiple different types of sensors that changes continuously over time and is related to the motion state of a target object. For example, it can include multi-dimensional temporal motion data such as the target object's trajectory point location data, velocity data, acceleration data, and foot contact force. The target object can be a person; the trajectory point location data can be the target object's positioning data; the velocity data can be measured by a velocity sensor installed on the target object; the acceleration can be measured by an accelerometer installed on the target object; and the foot contact force can be measured by a pressure sensor array distributed on the smart insole. In practical implementation, the controller receives temporal motion data from each sensor through a communication interface.

[0061] To ensure data accuracy, the multi-source temporal motion data can be preprocessed. Preprocessing may include denoising, normalization, and time alignment using timestamps to form a standardized input sequence with a fixed time step and feature dimensions. Here, a low-pass filter can be used for denoising, and Z-Score normalization can be used for normalization.

[0062] The normalized input sequence can be represented as ,in For time step, The time step and feature dimension can both be set as needed. The time step can be 50, and the feature dimension can be 12.

[0063] Furthermore, the dominant motion phase of the target object at the current moment can be identified using a standardized input sequence. This dominant motion phase refers to the discrete action state that plays a major role at the current moment, defined based on the kinematic or dynamic characteristics of the target object. The dominant motion phase of the target object at the current moment can be identified through plantar contact force in multi-source time-series motion data, and can be determined by plantar contact markers. This is indicated by the following: For example, if at least one foot contact force is greater than the contact force threshold, then the dominant motion phase of the target object at the current moment is determined to be the gait support phase, i.e., there is... Otherwise, the dominant motion phase of the target object at the current moment is the gait swing phase, that is, there is Simultaneously, the duration of the dominant motion phase is recorded to filter out jitter caused by noise.

[0064] This step, by acquiring multi-source temporal motion data and extracting the dominant motion phase, enables accurate perception of the current complex motion state of the target object, laying a reliable data foundation for subsequent adoption of targeted prediction models.

[0065] Next, step S2 is performed, selecting one or more motion trajectory prediction sub-models corresponding to the dominant motion phase from a pre-built model library. The pre-built model library may include at least two motion trajectory prediction sub-models, which can be represented as follows: N represents the number of motion trajectory prediction sub-models in the model library. This represents the nth motion trajectory prediction sub-model in the model library. The model library includes data-driven sub-models and physics-driven sub-models; that is, the motion trajectory prediction sub-models in the model library can be either physics-driven or data-driven, and both are used to predict motion trajectory data for the next H time steps. The motion trajectory data for the next H time steps can include the trajectory point position data for each of the H time steps, where H time steps can correspond to 1 second, and H can be 20.

[0066] The physics-driven sub-model is suitable for motion phases with well-defined and constrained motion laws that can be modeled, such as the gait support phase. The physics-driven sub-model is based on parameterized dynamic equations. Make a prediction:

[0067] ;

[0068] in, The motion trajectory data predicted by the physics-driven sub-model for the next H time steps. For physical parameter vectors, For equivalent leg length, This is the acceleration due to gravity.

[0069] Parameterized dynamic equations It can be a discrete kinematic model of a differentially driven robot.

[0070] Data-driven sub-models are suitable for motion phases with complex and flexible motion patterns, such as the gait swing phase. Data-driven sub-models can make predictions based on deep learning networks.

[0071] ;

[0072] in, The data-driven sub-model predicts the motion trajectory data for the next H time steps, which may include the trajectory point location data for each of the H time steps. The initial diagonal covariance matrix is ​​the synchronous output of the data-driven sub-model, representing data noise, that is, the inherent and unavoidable randomness of the data itself, which can be called accidental uncertainty. For deep learning networks, These are network parameters.

[0073] Each motion trajectory prediction sub-model in the model library is associated with one or more motion phases. For example, a gait support phase can correspond to a specific physical driving sub-model. When a gait swing phase contains highly nonlinear and flexible motion patterns, and a single model cannot cover all situations, the gait swing phase can be associated with multiple data-driven sub-models. When multiple gait swing phases are highly similar in features, multiple gait swing phases can be associated with the same data-driven sub-model.

[0074] By utilizing the dominant motion phase, one or more motion trajectory prediction sub-models associated with it can be identified from the model library, and the standardized input sequence can be... The input is fed into one or more motion trajectory prediction sub-models, and the predicted motion trajectory of the target object at future times is obtained from the output of each sub-model. For the physics-driven sub-model within the motion trajectory prediction sub-model, the motion trajectory prediction result is... For the data-driven sub-model in the motion trajectory prediction sub-model, the motion trajectory prediction result is: .

[0075] In this step, the motion trajectory prediction sub-model is dynamically selected by the dominant motion phase. This not only utilizes the statistical regularity in the data, but also ensures that the prediction results conform to basic physical constraints, such as energy conservation and motion continuity. It also takes into account the specific complexity of different motion stages. This can effectively overcome the problem of poor adaptability of a single prediction model in complex dynamic environments and improve prediction accuracy and physical rationality.

[0076] Next, step S3 is executed, employing Bayesian deep learning techniques to quantify the uncertainty of the prediction results output by the data-driven sub-model, obtaining the probability distribution parameters of the prediction results. Specifically, the uncertainty quantification operation can be implemented using Monte Carlo dropout, deep ensemble methods, or direct learning of covariance noise. The probability distribution parameters of the prediction results output by the data-driven sub-model can include the prediction mean and the prediction diagonal covariance matrix, which together form a probability distribution. , Represents probability. This represents the output variable of the data-driven sub-model. For the input variables of the data-driven sub-model, It follows a normal distribution. To predict the mean, To predict the diagonal covariance matrix.

[0077] This step quantifies the cognitive blind spots of the data-driven sub-model when facing unknown or complex scenarios, enabling the controller to not only know the predicted trajectory location but also grasp the reliability of the prediction, providing crucial safety boundary information for downstream decision-making.

[0078] Next, step S4 is executed to construct a Model Predictive Control (MPC) optimization problem using the motion trajectory prediction results and probability distribution parameters. The cost function of the MPC optimization problem can include a tracking error term, a control cost term, and an uncertainty penalty term, and may also include a safety risk term. Here, the casadi library in Python can be used to construct the MPC optimization problem.

[0079] The model predictive control optimization problem can be expressed as:

[0080] ;

[0081] ;

[0082] in, The optimal control sequence can include control commands for the next H time steps. Let the cost function be the model predictive control optimization problem. This indicates the sequence of control instructions to be optimized. The predicted mean in the probability distribution parameters. Let be the predicted diagonal covariance matrix in the probability distribution parameters. These constraints represent the following: system dynamics constraints mean that the next trajectory point position data of the controlled target must conform to its own mechanical and physical laws, such as the kinematic equations of a differential drive robot; control input constraints mean that the control commands output to the motor must be within the maximum and minimum physical limits allowed by the hardware; and state and safety constraints mean that the trajectory of the controlled target cannot exceed the boundaries of the physical environment and cannot collide with obstacles.

[0083] This step enables data-driven closed-loop integration of the prediction and control processes, breaking down the barriers between prediction and control in traditional solutions and allowing the controller to perceive risks in preceding stages. Introducing uncertainty quantification results into the model predictive control optimization problem allows the controller to automatically adopt a more conservative strategy when prediction confidence is low.

[0084] Finally, step S5 is executed, which calls the IPOPT solver to solve the model predictive control optimization problem online, obtaining the optimal control commands for the robot. The solution frequency can be 10Hz. The obtained optimal control commands are motor control commands, which may include voltage control commands, linear velocity control commands, and angular velocity control commands, etc.

[0085] To ensure the real-time performance of the method, the model predictive control optimization problem can be coded in C++ first, and the number of iterations can be fixed to ensure that each solution is completed within 50ms.

[0086] It is understandable that the optimal control command could be The first control command, namely , which represents the optimal control command for the next time step from the current time step.

[0087] Optimal control commands allow the target to move and track objects. Once the target's trajectory point position data is updated, the process returns to step S1 for the next round of data processing. This step ensures real-time response to complex optimization problems within strict time constraints, enabling the target to smoothly and safely follow or avoid objects.

[0088] The motion trajectory prediction data processing method provided in this embodiment of the invention, through a data processing flow of phase perception, hybrid modeling, uncertainty quantification, and joint optimization, can not only significantly improve the accuracy and physical consistency of motion trajectory prediction, but also enable downstream control to adaptively adjust safety strategies based on the prediction confidence level, significantly reducing collision risks in scenarios such as human-robot collaboration and improving the safety and adaptability of robots. This method selects one or more motion trajectory prediction sub-models corresponding to the dominant motion phase from a pre-built model library, allowing for the selection of the most suitable motion trajectory prediction sub-model for different motion phases, thus enabling rapid adaptation to different target objects or motion patterns. Furthermore, the data processing flow of this method does not depend on specific hardware and can be widely applied in fields requiring time-series prediction and decision-making collaboration, such as robot following, autonomous driving trajectory planning, industrial process control, and intelligent rehabilitation assessment.

[0089] Based on the above embodiments, the step of selecting one or more motion trajectory prediction sub-models corresponding to the dominant motion phase from a pre-built model library includes:

[0090] If the dominant motion phase is the gait support phase, then the motion trajectory prediction sub-model is determined to be the physical driving sub-model, and the physical parameters of the physical driving sub-model are estimated online based on the historical motion data of the target object;

[0091] If the dominant motion phase is the gait swing phase, then the motion trajectory prediction sub-model is determined to be the data-driven sub-model, which is a pre-trained Transformer encoder-decoder model.

[0092] Specifically, when the dominant motion phase is the gait support phase, the motion trajectory prediction sub-model can be defined as a physical driving sub-model. This physical driving sub-model can be an inverted pendulum model, and its physical parameters can be estimated online using the historical motion data of the target object. The historical motion data can be motion data from the past 3 seconds. The physical parameters in the parameterized dynamic equations are then back-calculated and updated in real time using recursive least squares (RLS) or a Kalman filter. This allows the physical driving sub-model to quickly adapt to different target objects' height, weight, and specific gait characteristics; that is, physical parameters such as equivalent leg length and gravitational acceleration are corrected in real time.

[0093] When the dominant motion phase is the gait swing phase, the motion trajectory prediction sub-model can be defined as a data-driven sub-model, which can be a pre-trained Transformer encoder-decoder model. This data-driven sub-model can be trained using publicly available gait datasets.

[0094] To ensure the real-time performance of the method, the data-driven sub-model can be converted into the TensorRT engine to utilize hardware acceleration.

[0095] In this embodiment of the invention, by selectively scheduling the physics-driven sub-model and the data-driven sub-model, the advantages of physics rules and data-driven approaches can be complemented, which improves the generalization ability for complex behaviors while ensuring the rationality of the predicted physics.

[0096] Based on the above embodiments, the data-driven sub-model is obtained by end-to-end training using a composite loss function based on motion time-series data samples with true trajectory labels.

[0097] The composite loss function includes prediction error loss, physical consistency loss, and uncertainty calibration loss. The prediction error loss is calculated based on the predicted trajectory data output by the data-driven sub-model and the true trajectory label.

[0098] The physical consistency loss is calculated based on the predicted acceleration and the theoretical acceleration. The predicted acceleration is determined based on the second derivative of the predicted trajectory data with respect to time, and the theoretical acceleration is determined based on the predicted trajectory data by applying parameterized dynamic equations.

[0099] The uncertainty calibration loss is determined based on the initial diagonal covariance matrix output by the data-driven sub-model, the predicted trajectory data, and the ground truth trajectory label.

[0100] Specifically, publicly available gait datasets can include ground truth trajectory labels from multiple objects and scenarios. The dataset contains temporal motion data samples. The gait dataset can be divided into training, validation, and test sets in an 8:1:1 ratio. The training set is used for model training, the validation set for model validation after training, and the test set for model testing after validation. Here, the Adam optimizer can be used to perform model training on the training set and stop early on the validation set to obtain the trained data-driven sub-model.

[0101] During model training, motion time-series data samples are input into the data-driven sub-model to obtain the predicted trajectory data and the initial diagonal covariance matrix output by the data-driven sub-model. The predicted trajectory data can be trajectory data for the next H time steps, including the trajectory point position data for each time step. Similarly, the ground truth trajectory labels... It can also include trajectory point location data for each time step.

[0102] Subsequently, the composite loss function is calculated using the predicted trajectory data, ground truth trajectory labels, and the initial diagonal covariance matrix. This composite loss function is then used to perform end-to-end training on the data-driven sub-model, resulting in the trained data-driven sub-model. The ultimate goal of model training is to make the composite loss function as close to 0 as possible.

[0103] The composite loss function includes prediction error loss, physical consistency loss, and uncertainty calibration loss. Prediction error loss represents the pure data error between the prediction result of the data-driven sub-model and the true trajectory label. Physical consistency loss represents the degree to which the prediction result of the data-driven sub-model violates established physical laws. Uncertainty calibration loss represents the deviation between the data-driven sub-model's confidence in the accuracy of its prediction result and the actual error.

[0104] The composite loss function can be expressed by the following formula:

[0105] ;

[0106] in, For composite loss function, To predict error loss, This is the Physical Consistency Loss. Uncertainty Calibration Loss. and These are all weighting coefficients, which can be manually set to adjust the ratio between the three loss terms.

[0107] Prediction error loss can be calculated using the mean squared error (MSE) formula:

[0108] ;

[0109] in, To predict the position of the trajectory point at the k-th time step in the trajectory data, This represents the position data of the trajectory point at the k-th time step in the true trajectory label. It is an L2 norm.

[0110] The physical consistency loss can be calculated using the following formula:

[0111] ;

[0112] in, for The second derivative with respect to time t represents the predicted acceleration. for The first derivative with respect to time t represents the predicted velocity. To parametrically compose the dynamic equations The theoretical acceleration obtained.

[0113] The larger the difference between the predicted acceleration and the theoretical acceleration, the more likely the data-driven sub-model is predicting a physically impossible motion, such as a person gaining a huge thrust out of thin air. In this case, the physical consistency loss will increase to penalize the data-driven sub-model.

[0114] Uncertainty calibration loss can be calculated using the following formula:

[0115] ;

[0116] in, The initial diagonal covariance matrix output by the data-driven sub-model corresponds to the initial diagonal covariance at the k-th time step. The smaller the initial diagonal covariance, the more confident the data-driven sub-model is; the larger the initial diagonal covariance, the less confident the data-driven sub-model is.

[0117] Uncertainty calibration loss forces the initial diagonal covariance matrix of the data-driven submodel to match the actual error. If the data-driven submodel makes a significant error at a certain step—that is, the actual error is large, but its initial diagonal covariance is small—the uncertainty calibration loss will be larger, forming a substantial penalty. After training with this uncertainty calibration loss, the data-driven submodel learns to honestly output a larger initial diagonal covariance when encountering unfamiliar or complex actions.

[0118] On the test set, the initial diagonal covariance matrix can be calibrated by temperature scaling on the trained data-driven sub-model, so that the prediction confidence and accuracy are better matched, and the data-driven sub-model is not too overconfident in its prediction results.

[0119] In this embodiment of the invention, by guiding model training through a composite loss function, it is possible to effectively prevent black-box models from outputting results that violate physical common sense and improve the honesty of the data-driven sub-model in self-evaluating uncertainty.

[0120] Based on the above embodiments, the probability distribution parameters include the predicted mean and the predicted diagonal covariance matrix;

[0121] The uncertainty quantification of the prediction results output by the data-driven sub-model to obtain the probability distribution parameters of the prediction results output by the data-driven sub-model includes:

[0122] Based on the multi-source temporal motion data, the data-driven sub-model performs multiple random forward propagations to obtain multiple prediction results and the initial diagonal covariance matrix corresponding to each prediction result.

[0123] The mean of the multiple prediction results is calculated as the prediction mean, and the diagonal covariance matrix of the multiple prediction results is calculated as the divergence diagonal covariance matrix. Based on the divergence diagonal covariance matrix and the initial diagonal covariance matrix corresponding to each prediction result, the prediction diagonal covariance matrix is ​​determined.

[0124] Specifically, when quantifying the uncertainty of the prediction results output by the data-driven sub-model, the Monte Carlo Dropout method is used. Dropout activation is maintained throughout the inference process of the data-driven sub-model, and the dropout rate can be set to 0.1, meaning that 10% of neurons are randomly muted during each random forward propagation (parameter calculation not required). Therefore, when facing the same input, the output of the data-driven sub-model is slightly different each time because different neurons are randomly muted within the data-driven sub-model. Furthermore, by inputting the standardized input sequence into the data-driven sub-model, it can perform multiple random forward propagations to obtain multiple prediction results and the initial diagonal covariance matrix corresponding to each prediction result. These multiple prediction results can be represented as... , Let M be the m-th prediction result, where M is the number of random forward propagations, which is the number of prediction results and can be 30.

[0125] The predicted mean can be calculated using the following formula:

[0126] ;

[0127] ;

[0128] in, This is the initial diagonal covariance matrix corresponding to the m-th prediction result, which is the initial diagonal covariance matrix obtained when the data-driven sub-model performs the m-th random forward propagation. The diagonal covariance matrix of the M prediction results, also known as the divergence diagonal covariance matrix, represents the cognitive uncertainty of the data-driven sub-model due to the fact that the data has not been seen before.

[0129] In this embodiment of the invention, by calculating the probability distribution parameters, the inherent random uncertainty of the data itself and the cognitive uncertainty caused by insufficient understanding of the model can be fully captured, providing a high-precision probabilistic description for trajectory prediction.

[0130] Based on the above embodiments, the cost function of the model predictive control optimization problem includes a tracking error term, a control cost term, and an uncertainty penalty term; the construction of the model predictive control optimization problem based on the motion trajectory prediction result and the probability distribution parameters includes:

[0131] The tracking error term is calculated based on the difference between the predicted motion trajectory result and the predicted mean in the probability distribution parameters.

[0132] The control cost term is calculated based on the sequence of control instructions to be optimized;

[0133] The uncertainty penalty term is determined based on the trace of the predicted diagonal covariance matrix in the probability distribution parameters.

[0134] Specifically, the cost function of the model predictive control optimization problem includes a tracking error term, a control cost term, and an uncertainty penalty term, which can be expressed as:

[0135] ;

[0136] in, This indicates the sequence of control instructions to be optimized. This represents the control instruction at the k-th time step in the future. This represents the position data at the k-th time step in the motion trajectory prediction result. This represents the predicted mean at the k-th time step in the future. This represents the predicted diagonal covariance matrix for the k-th time step in the future. express traces, For uncertainty penalty weighting coefficient, and These are the positive definite weight matrices for the corresponding terms. To track the error term, To control the cost item, This is a penalty item for uncertainty.

[0137] In this embodiment of the invention, the uncertainty of prediction is directly converted into a penalty factor in control optimization, which can prompt the controller to automatically adopt a more stable and conservative action strategy when faced with unreliable predictions.

[0138] Based on the above embodiments, the cost function of the model predictive control optimization problem also includes a security risk term;

[0139] The step of constructing a model prediction control optimization problem based on the motion trajectory prediction result and the probability distribution parameters further includes:

[0140] Based on the trajectory point position data in the multi-source temporal motion data, the actual distance between the control target and the target object is determined.

[0141] The safety risk term is determined based on the difference between the actual distance and the minimum absolute safe distance, and the trace of the predicted diagonal covariance matrix in the probability distribution parameters.

[0142] Specifically, the cost function of the model predictive control optimization problem can also include a safety risk term, then the cost function can be expressed as:

[0143] ;

[0144] in, For the weighting coefficient of the safety risk item, An exponential function that makes safety constraints tighten as uncertainty increases. This is the minimum absolute safe distance.

[0145] It can be calculated using the following formula:

[0146] ;

[0147] in, The actual distance between the control target and the target object at the k-th time step can be determined by combining the trajectory point position data from multi-source time-series motion data with the current position data of the control target. When it increases, The decay slows down, which means that even Greater than The controller will also be subject to a greater risk penalty, which forces the optimizer to calculate a control strategy that moves the control target further away from the target object.

[0148] In this embodiment of the invention, by introducing a safety risk item, the dynamic adaptive adjustment of the safety distance threshold can be achieved, which can greatly enhance the active collision avoidance capability and overall safety in the human-machine collaboration process.

[0149] Based on the above embodiments, identifying the dominant motion phase of the target object at the current moment based on the multi-source temporal motion data includes:

[0150] The multi-source temporal motion data is preprocessed to obtain a standardized input sequence;

[0151] The standardized input sequence is input into the phase recognition model to obtain the dominant motion phase output by the phase recognition model.

[0152] Specifically, when identifying the dominant motion phase of a target object at the current moment, a phase recognition model can be introduced. The preprocessed, standardized input sequence is input into the phase recognition model, which then obtains and outputs the dominant motion phase. This phase recognition model can be represented as follows: , Let K be a predefined, mutually exclusive set of phase states, and K be the number of phase states contained in the set. The resulting dominant motion phase can be represented as: And there are .

[0153] Here, the phase recognition model can be a regular state machine, a supervised learning classifier, or a hidden Markov model. For example, the phase recognition model can be a one-dimensional convolutional neural network (1D-CNN). The phase recognition model can be trained using labeled motion phase data and a cross-entropy loss function.

[0154] Based on the above embodiments, an independent security monitoring thread can also be introduced to monitor in real time. The size. If If the set threshold is exceeded, an emergency stop will be triggered or the system will switch to manual remote operation mode.

[0155] Based on the above embodiments, the motion trajectory prediction data processing method provided in this invention was tested in an indoor environment. The controlled object was a robot. The robot successfully predicted the target object's intention to move straight, turn, and stop, and made smooth following adjustments 0.3-0.8 seconds in advance. In the test where the target object suddenly turned, the controller decelerated in advance due to the sudden increase in uncertainty, avoiding a potential collision. Compared with existing methods, the motion trajectory prediction data processing method provided in this invention reduced the average tracking error by 28% in sharp turns and obstacle avoidance scenarios, and no collisions occurred.

[0156] like Figure 2As shown, based on the above embodiments, this embodiment of the invention provides a motion trajectory prediction data processing device, including:

[0157] Phase recognition module 21 is used to acquire multi-source temporal motion data of the target object at the current moment, and based on the multi-source temporal motion data, identify the dominant motion phase of the target object at the current moment;

[0158] The trajectory prediction module 22 is used to select one or more motion trajectory prediction sub-models corresponding to the dominant motion phase from a pre-built model library, and apply the motion trajectory prediction sub-models based on the multi-source time-series motion data to generate the motion trajectory prediction result of the target object at a future time; the model library includes data-driven sub-models and physics-driven sub-models.

[0159] Uncertainty quantification module 23 is used to perform uncertainty quantification on the prediction results output by the data-driven sub-model to obtain the probability distribution parameters of the prediction results output by the data-driven sub-model.

[0160] Problem construction module 24 is used to construct a model prediction control optimization problem based on the motion trajectory prediction results and the probability distribution parameters;

[0161] The problem-solving module 25 is used to solve the model predictive control optimization problem online to obtain the optimal control command for the control objective.

[0162] Specifically, the functions of each module in the motion trajectory prediction data processing device provided in this embodiment of the invention correspond one-to-one with the operation flow of each step in the above method-like embodiments, and the achieved effects are also the same. For details, please refer to the above embodiments, and this will not be repeated in this embodiment of the invention.

[0163] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3 As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute the motion trajectory prediction data processing method provided in the above embodiments.

[0164] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to related technologies, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0165] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the motion trajectory prediction data processing method provided in the above embodiments.

[0166] In another aspect, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the motion trajectory prediction data processing method provided in the above embodiments. This computer-readable storage medium can be either a non-transitory computer-readable storage medium or a transient computer-readable storage medium, and is not specifically limited herein.

[0167] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0168] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0169] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for processing motion trajectory prediction data, characterized in that, include: Acquire multi-source temporal motion data of the target object at the current moment, and based on the multi-source temporal motion data, identify the dominant motion phase of the target object at the current moment; One or more motion trajectory prediction sub-models corresponding to the dominant motion phase are selected from the pre-built model library, and the motion trajectory prediction sub-models are applied based on the multi-source time-series motion data to generate the motion trajectory prediction results of the target object at future times; the model library includes data-driven sub-models and physics-driven sub-models; Uncertainty quantification is performed on the prediction results output by the data-driven sub-model to obtain the probability distribution parameters of the prediction results output by the data-driven sub-model. Based on the predicted motion trajectory and the probability distribution parameters, a model prediction control optimization problem is constructed. The model predictive control optimization problem is solved online to obtain the optimal control command for the control objective; The step of selecting one or more motion trajectory prediction sub-models corresponding to the dominant motion phase from a pre-built model library includes: If the dominant motion phase is the gait support phase, then the motion trajectory prediction sub-model is determined to be the physical driving sub-model, and the physical parameters of the physical driving sub-model are estimated online based on the historical motion data of the target object; If the dominant motion phase is the gait swing phase, then the motion trajectory prediction sub-model is determined to be the data-driven sub-model, which is a pre-trained Transformer encoder-decoder model.

2. The motion trajectory prediction data processing method according to claim 1, characterized in that, The data-driven sub-model is obtained by end-to-end training based on motion time-series data samples with true trajectory labels and applying a composite loss function. The composite loss function includes prediction error loss, physical consistency loss, and uncertainty calibration loss. The prediction error loss is calculated based on the predicted trajectory data output by the data-driven sub-model and the true trajectory label. The physical consistency loss is calculated based on the predicted acceleration and the theoretical acceleration. The predicted acceleration is determined based on the second derivative of the predicted trajectory data with respect to time, and the theoretical acceleration is determined based on the predicted trajectory data by applying parameterized dynamic equations. The uncertainty calibration loss is determined based on the initial diagonal covariance matrix output by the data-driven sub-model, the predicted trajectory data, and the ground truth trajectory label.

3. The motion trajectory prediction data processing method according to claim 1, characterized in that, The probability distribution parameters include the predicted mean and the predicted diagonal covariance matrix; The uncertainty quantification of the prediction results output by the data-driven sub-model to obtain the probability distribution parameters of the prediction results output by the data-driven sub-model includes: Based on the multi-source temporal motion data, the data-driven sub-model performs multiple random forward propagations to obtain multiple prediction results and the initial diagonal covariance matrix corresponding to each prediction result. The mean of the multiple prediction results is calculated as the prediction mean, and the diagonal covariance matrix of the multiple prediction results is calculated as the divergence diagonal covariance matrix. Based on the divergence diagonal covariance matrix and the initial diagonal covariance matrix corresponding to each prediction result, the prediction diagonal covariance matrix is ​​determined.

4. The motion trajectory prediction data processing method according to claim 1, characterized in that, The cost function of the model predictive control optimization problem includes a tracking error term, a control cost term, and an uncertainty penalty term; the construction of the model predictive control optimization problem based on the motion trajectory prediction result and the probability distribution parameters includes: The tracking error term is calculated based on the difference between the predicted motion trajectory result and the predicted mean in the probability distribution parameters. The control cost term is calculated based on the sequence of control instructions to be optimized; The uncertainty penalty term is determined based on the trace of the predicted diagonal covariance matrix in the probability distribution parameters.

5. The motion trajectory prediction data processing method according to claim 4, characterized in that, The cost function of the model predictive control optimization problem also includes a security risk term; The step of constructing a model prediction control optimization problem based on the motion trajectory prediction result and the probability distribution parameters further includes: Based on the trajectory point position data in the multi-source temporal motion data, the actual distance between the control target and the target object is determined. The safety risk term is determined based on the difference between the actual distance and the minimum absolute safe distance, and the trace of the predicted diagonal covariance matrix in the probability distribution parameters.

6. The motion trajectory prediction data processing method according to any one of claims 1-5, characterized in that, The step of identifying the dominant motion phase of the target object at the current moment based on the multi-source temporal motion data includes: The multi-source temporal motion data is preprocessed to obtain a standardized input sequence; The standardized input sequence is input into the phase recognition model to obtain the dominant motion phase output by the phase recognition model.

7. A motion trajectory prediction data processing device, characterized in that, include: The phase recognition module is used to acquire multi-source temporal motion data of the target object at the current moment, and based on the multi-source temporal motion data, identify the dominant motion phase of the target object at the current moment; The trajectory prediction module is used to select one or more motion trajectory prediction sub-models corresponding to the dominant motion phase from a pre-built model library, and apply the motion trajectory prediction sub-models based on the multi-source time-series motion data to generate the motion trajectory prediction result of the target object at a future time; the model library includes data-driven sub-models and physics-driven sub-models. The uncertainty quantification module is used to quantify the uncertainty of the prediction results output by the data-driven sub-model to obtain the probability distribution parameters of the prediction results output by the data-driven sub-model. The problem construction module is used to construct a model prediction control optimization problem based on the motion trajectory prediction results and the probability distribution parameters. The problem-solving module is used to solve the model predictive control optimization problem online to obtain the optimal control command for the control objective. The trajectory prediction module is specifically used for: If the dominant motion phase is the gait support phase, then the motion trajectory prediction sub-model is determined to be the physical driving sub-model, and the physical parameters of the physical driving sub-model are estimated online based on the historical motion data of the target object; If the dominant motion phase is the gait swing phase, then the motion trajectory prediction sub-model is determined to be the data-driven sub-model, which is a pre-trained Transformer encoder-decoder model.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the motion trajectory prediction data processing method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the motion trajectory prediction data processing method as described in any one of claims 1-6.

Images