Robot virtual-real cooperative training decision optimization system and method based on digital twinning

Abstract

The invention discloses a robot virtual-real cooperative training decision optimization system and method based on digital twinning, and the method comprises the following steps: collecting state data and disturbance data of an entity robot in a real environment, and carrying out the preprocessing of the data to generate standardized input; joint coding and state perturbation mapping are carried out on the standardized data, and a feature vector sequence embedded in a hyperspherical manifold space is generated; inputting to a virtual twin control body based on a hypersurface neural element structure, executing disturbance direction sensitive activation, and outputting an activated state vector sequence; virtual and entity control action prediction sequences are generated respectively, an embedded space difference vector is calculated, and control body parameters are updated based on a consistency optimization criterion; after convergence, the control body executes reasoning to generate a target control action sequence, the entity robot is driven to complete action execution, and control strategy optimization is achieved. According to the method, high-precision migration and rapid convergence of a robot control strategy are realized, and the execution stability in a complex disturbance environment is improved.

Description

Technical Field

[0001] This invention relates to the field of robot intelligent control technology, and in particular to a robot virtual-real collaborative training decision optimization system and method based on digital twins. Background Technology

[0002] In the field of intelligent robot control, the integration of virtual simulation training and physical control deployment has become an important direction for improving the adaptability of complex tasks. Existing technologies typically use reinforcement learning, imitation learning, or self-supervised methods to build control strategies in a virtual environment and then transfer these strategies to the real robot for execution. Some solutions introduce digital twin modeling structures to simulate and reconstruct the state changes of the physical robot during operation to support control strategy optimization. However, current virtual-real fusion control systems generally suffer from insufficient model representation capabilities, inaccurate disturbance response, and excessive differences between virtual and real control. Especially in non-stationary or frequently disturbed environments, the strategy transfer performance deteriorates significantly, seriously affecting the reliability and robustness of robot action execution.

[0003] Most existing methods construct virtual control modules using traditional neural network structures or standard controller combinations, which makes it difficult to capture the coupling relationship between state disturbances and lacks the ability to collaboratively model the relationship between state and disturbance data. In the process of control strategy optimization, most techniques only update the strategy based on the Euclidean space loss function, which cannot accurately reflect the geometric differences between action sequences in the manifold space, resulting in slow training convergence and weak policy generalization ability. Currently, there is also a lack of model structures specifically designed for embedding and decoupling the characteristics of state disturbances, which limits the ability to respond to control deviations induced by disturbances.

[0004] There is an urgent need for an innovative method that possesses strong expressive power, can respond collaboratively to disturbance changes between the virtual and real worlds, and can accurately execute control strategy optimization within a manifold space to improve the motion control accuracy and training transfer effect of robots in complex environments. Therefore, how to provide a robot virtual-real collaborative training decision optimization system and method based on digital twins is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0005] One objective of this invention is to propose a robot virtual-real collaborative training decision optimization system and method based on digital twins. This invention utilizes digital twin modeling and perturbation embedding mechanisms to construct a control model that can perform inference and training in Riemannian manifold space, which has the advantages of high response accuracy, fast policy convergence, and small virtual-real differences.

[0006] The robot virtual-real collaborative training decision optimization method based on digital twin according to an embodiment of the present invention includes the following steps:

[0007] Collect state data and disturbance data of the physical robot in the real operating environment, perform preprocessing operations, and obtain standardized state data and disturbance data;

[0008] Standardized state data and perturbation data are jointly encoded and processed, and a state perturbation mapping operation is performed to generate a sequence of state perturbation embedding feature vectors embedded in the hyperspherical manifold space.

[0009] The state perturbation is embedded into the feature vector sequence input and constructed in a virtual twin control body of a hypersurface neuron structure. Inside the control body, the perturbation direction-sensitive activation function is used to calculate the feature response of each neuron to obtain the activation state vector sequence.

[0010] The activation state vector sequence is input into the virtual action prediction path and the real action prediction path respectively to generate the virtual control action prediction sequence and the entity control action prediction sequence respectively.

[0011] The embedding space difference vector between the virtual control action prediction sequence and the entity control action prediction sequence is calculated. The connection weights and bias parameters between hypersurface neurons are updated based on the consistency optimization criterion. After training convergence, the converged virtual twin control body is obtained.

[0012] The converged virtual twin controller is used as the action policy reasoning structure to perform reasoning operations on standardized state data and standardized disturbance data to generate the target control action sequence.

[0013] The target control action sequence is input into the control instruction generation process of the physical robot, which drives the physical robot to complete the action execution, realizing the control strategy optimization process based on virtual-real collaborative training.

[0014] Optionally, the physical robot's state data and disturbance data in the real operating environment include joint position data, joint velocity data, motor current data, end effector posture data, external force contact sensing data, ambient temperature data, and structural vibration acceleration data.

[0015] Optionally, the preprocessing operation includes: aligning the state data and disturbance data with timestamps, performing data interpolation and completion operations, unifying the numerical range of the dimensions of the state data and disturbance data, filtering out high-frequency noise interference in the signal, and constructing standardized state data and standardized disturbance data arranged by time steps.

[0016] Optionally, the operation of jointly encoding the standardized state data and the standardized perturbation data, performing a state perturbation mapping operation, and generating a sequence of state perturbation embedding feature vectors embedded in the hyperspherical manifold space includes:

[0017] After performing dimension matching processing on the standardized state data vector and standardized perturbation data vector corresponding to each time step, the vectors are concatenated to generate a joint input vector sequence with unified dimensions.

[0018] The joint input vector sequence is input into an encoding structure consisting of several linear feature transformation layers and nonlinear activation functions. Linear mapping, nonlinear response, and feature reconstruction operations are performed sequentially to obtain an intermediate encoded feature sequence.

[0019] Based on the continuous change magnitude of the perturbation data in the time dimension, a perturbation change weighting factor is constructed. The perturbation change weighting factor is used to adjust the response intensity of the state-related dimension in the intermediate coding features to generate a perturbation-enhanced coding feature sequence.

[0020] Perform feature amplitude normalization on each vector in the perturbation-enhanced coding feature sequence. Specifically, this includes: calculating the overall amplitude index of the vector in all feature dimensions, scaling all components of the vector to a unit length range according to the amplitude ratio, and satisfying the uniform norm constraint.

[0021] The normalized vectors are embedded into a unit hyperspherical manifold space with Riemannian geometry. The embedded vectors maintain geodesic consistency and spatial continuity within the unit hyperspherical manifold space, forming a sequence of state perturbation embedded feature vectors.

[0022] Optionally, the embedding operation of the unit hyperspherical manifold space with Riemannian geometry includes:

[0023] Based on the feature dimension of the perturbation-enhanced coding feature sequence, the spatial dimension order of the unit hyperspherical manifold space is set, and an embedding boundary with a Riemannian metric structure is defined in the space. The embedding boundary constitutes a high-dimensional spherical space that satisfies the norm constant constraint.

[0024] Amplitude normalization operation is performed on each feature vector in the perturbation-enhanced coding feature sequence, and the values ​​of each feature component are scaled by a uniform ratio. The embedding points corresponding to all feature vectors are located on the spherical boundary under the unit norm constraint.

[0025] The distance metric between embedding points in the manifold space is set to geodesic distance based on the inner product cosine function, and the angular structure relationship between the original feature vectors is preserved during the construction of the embedding vector sequence.

[0026] All embedding vectors in the perturbation-enhanced coding feature sequence are arranged in chronological order. During the embedding process, the embedding points between adjacent time steps are constrained to form a continuous curve sequence, which is a set of smooth embedding trajectories constructed in the manifold space.

[0027] Optionally, the process of generating the activation state vector sequence includes:

[0028] The state perturbation embedding feature vector sequence arranged in chronological order is input into a neural computation structure composed of several layers of hypersurface neurons. Each layer of the neural computation structure is composed of a fixed number of hypersurface neuron units. All neurons receive the embedding vector from the previous layer and perform feature transformation.

[0029] In each hypersurface neuron, an internal orientation bias mapping unit is constructed to receive the perturbation orientation vector generated during the embedding construction process of the corresponding perturbation enhancement coding feature vector. The perturbation orientation vector is then multiplied by the current input embedding vector to obtain the current perturbation orientation response value.

[0030] The activation threshold of the neuron is dynamically adjusted based on the perturbation direction response value. The activation function performs nonlinear response processing on the input signal based on the adjusted threshold and outputs the neuron activation value after perturbation direction adjustment.

[0031] The activation values ​​of all neurons in the neural computation structure are aggregated to form the activation state vector sequence at the current time step, and arranged in chronological order to form the final output activation state vector sequence.

[0032] Optionally, the operation of calculating the embedding space difference vector between the virtual control action prediction sequence and the entity control action prediction sequence, and updating the connection weights and bias parameters between hypersurface neurons based on the consistency optimization criterion, includes:

[0033] Amplitude normalization is performed on the virtual control action prediction sequence and the entity control action prediction sequence respectively to generate a control action embedding vector sequence that satisfies the unit norm constraint. The action embedding vector sequence is then mapped to the unit hyperspherical manifold space with the same Riemannian geometry used by the state disturbance embedding feature vector sequence to form the virtual action embedding trajectory and the entity action embedding trajectory.

[0034] In this manifold space, the angle between the virtual action embedding vector and the entity action embedding vector at each time step is calculated based on the tensor inner product relation, generating the main difference term of the geodesic distance representation;

[0035] Construct a sequence of angle offsets between the perturbation direction vector and the change direction of the action embedding vector, as the perturbation direction offset term;

[0036] Calculate the rate of change of angle between action embedding vectors in adjacent time steps within the manifold space, and use it as the action prediction continuity offset term;

[0037] Using the main difference term, disturbance direction offset term, and action continuity offset term as inputs, a consistency optimization criterion is constructed. The consistency optimization criterion is a weighted multinomial loss function defined in the unit hyperspherical manifold space of Riemannian geometry, which is used to measure the degree of spatial deviation between virtual control actions and physical control actions.

[0038] For each hypersurface neuron unit constituting the virtual twin control body, the weighted multinomial loss function constructed based on the consistency optimization criterion performs partial derivative calculation operations on the neuron connection weight matrix and the bias parameter vector, respectively. The partial derivative is the derivative result of the weighted multinomial loss function with respect to the parameters.

[0039] Based on the obtained partial derivative values, gradient descent is used to update each parameter according to the preset learning rate. The update direction is the opposite of the partial derivative, and the update step size is determined by the product of the learning rate and the magnitude of the partial derivative. This completes the iterative optimization of the connection weights and bias parameters in the current training round.

[0040] Optionally, the inference operation on the standardized state data and standardized perturbation data includes: aligning and concatenating the standardized state data and standardized perturbation data in the time dimension, inputting them into the converged virtual twin control body, sequentially performing connection weight calculation and activation function processing of the hypersurface neuron, generating an activation state vector sequence, and outputting a target control action sequence based on the activation state vector sequence.

[0041] Optionally, the control strategy optimization process includes:

[0042] The target control action sequence is input into the entity execution structure to execute control instructions, the actual execution state at each time step is collected, and a feedback vector sequence is constructed.

[0043] The degree of difference between virtual and real control is determined by the geodesic distance between the virtual control action sequence and the feedback vector sequence in the unit hyperspherical manifold space of the Riemannian geometry.

[0044] When the geodesic distance exceeds the preset threshold and the mean square rate of change of the disturbance data is higher than the disturbance static threshold ε1 in M ​​consecutive time steps, the virtual-real synchronous retraining strategy is activated. The feedback vector and the disturbance data are jointly input into the virtual twin control body to perform several rounds of disturbance adversarial training to enhance control robustness.

[0045] When the geodesic distance exceeds the preset threshold but the mean square rate of change of the disturbance data within M consecutive time steps is lower than the static threshold ε1 of the disturbance, the direction correction update strategy between the target control action sequence and the feedback vector is executed, and only the output layer connection weights are locally fine-tuned to maintain the stability of the previous strategy.

[0046] When the geodesic distance is lower than the preset threshold, but the geodesic distance between the feedback vector and the virtual control action vector at the corresponding time step is greater than the deviation threshold ε2 within N consecutive time steps, the deviation tolerance strategy module is invoked to smooth the entity feedback by adjusting the control action tolerance range constraint, thereby maintaining the consistency of the overall control process.

[0047] Based on the optimization results under the three scenarios, the parameters of the virtual twin controller are updated to complete the dynamic optimization of the control strategy based on the virtual-real collaborative training mechanism.

[0048] The robot virtual-real collaborative training decision optimization system based on digital twin according to an embodiment of the present invention includes the following modules:

[0049] The status and disturbance acquisition module is used to acquire status data and disturbance data of the physical robot in a real environment;

[0050] The data preprocessing module is used to perform standardization processing on state data and disturbance data to generate standardized state data and disturbance data;

[0051] The joint encoding mapping module is used to jointly encode standardized state data and perturbation data, perform state perturbation mapping, and generate a sequence of state perturbation embedded feature vectors.

[0052] The virtual twin control module is used to receive state perturbation embedding feature vector sequences based on hypersurface neuron structures and generate activation state vector sequences.

[0053] The motion prediction path module is used to receive the activation state vector sequence and output the virtual control motion prediction sequence and the entity control motion prediction sequence, respectively.

[0054] The parameter optimization and update module is used to update the parameters in the virtual twin controller body until convergence based on the embedding space difference vector between the action prediction sequences.

[0055] The action strategy reasoning module is used to input standardized state data and disturbance data into the converged control body and output the target control action sequence.

[0056] The control execution module is used to receive the target control action sequence, generate control instructions, and drive the physical robot to perform the corresponding actions.

[0057] The beneficial effects of this invention are:

[0058] (1) This invention constructs a virtual twin control body based on a hypersurface neuron structure, integrates the perturbation direction-sensitive activation mechanism and the state perturbation embedding feature expression method, improves the model's ability to express control response under complex perturbation conditions, and enhances the adaptability and stability of the control strategy in non-stationary environments.

[0059] (2) This invention introduces a joint coding mapping and embedding space consistency optimization criterion to jointly model and optimize state data and disturbance data in the Riemann geometric structure unit hyperspherical manifold space, so as to ensure the geometric alignment accuracy between virtual control actions and physical feedback actions and reduce control failure problems caused by virtual-real deviation.

[0060] (3) The present invention constructs a control strategy optimization process based on multiple scenario judgment conditions. It realizes a dynamic update mechanism of strategy by combining geodesic distance and disturbance change rate as criteria, so that the virtual control body has the ability to converge adaptive control strategy in response to disturbance intensity and feedback deviation characteristics, thereby improving the execution accuracy and control closed-loop efficiency of physical robots under complex tasks. Attached Figure Description

[0061] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0062] Figure 1 This is an overall flowchart of the robot virtual-real collaborative training decision optimization method based on digital twin proposed in this invention;

[0063] Figure 2 This is a flowchart of the consistency optimization process for the virtual and physical action differences in the robot virtual-real collaborative training decision optimization method based on digital twin proposed in this invention.

[0064] Figure 3 This is a block diagram of the robot virtual-real collaborative training decision optimization system based on digital twin proposed in this invention. Detailed Implementation

[0065] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0066] refer to Figure 1-3 A robot virtual-real collaborative training decision optimization method based on digital twins includes the following steps:

[0067] Collect state data and disturbance data of the physical robot in the real operating environment, perform preprocessing operations, and obtain standardized state data and disturbance data;

[0068] Standardized state data and perturbation data are jointly encoded and processed, and a state perturbation mapping operation is performed to generate a sequence of state perturbation embedding feature vectors embedded in the hyperspherical manifold space.

[0069] The state perturbation is embedded into the feature vector sequence input and constructed in a virtual twin control body of a hypersurface neuron structure. Inside the control body, the perturbation direction-sensitive activation function is used to calculate the feature response of each neuron to obtain the activation state vector sequence.

[0070] The activation state vector sequence is input into the virtual action prediction path and the real action prediction path respectively to generate the virtual control action prediction sequence and the entity control action prediction sequence respectively.

[0071] The embedding space difference vector between the virtual control action prediction sequence and the entity control action prediction sequence is calculated. The connection weights and bias parameters between hypersurface neurons are updated based on the consistency optimization criterion. After training convergence, the converged virtual twin control body is obtained.

[0072] The converged virtual twin controller is used as the action policy reasoning structure to perform reasoning operations on standardized state data and standardized disturbance data to generate the target control action sequence.

[0073] The target control action sequence is input into the control instruction generation process of the physical robot, which drives the physical robot to complete the action execution, realizing the control strategy optimization process based on virtual-real collaborative training.

[0074] This invention constructs a virtual twin control body based on a hypersurface neuron structure. It performs joint modeling and response reasoning on the state data and disturbance data collected by the physical robot to form a complete closed-loop virtual-physical collaborative control optimization process. Compared with the traditional separate modeling and reasoning scheme, this invention can realize unified data flow and end-to-end action strategy optimization, and has stronger robustness, reasoning efficiency and response accuracy under multiple disturbance conditions.

[0075] In this embodiment, the physical robot's state data and disturbance data in the real operating environment include joint position data, joint velocity data, motor current data, end effector posture data, external force contact sensing data, ambient temperature data, and structural vibration acceleration data.

[0076] In this embodiment, the preprocessing operation includes: aligning the state data and disturbance data with timestamps, performing data interpolation and completion operations, unifying the numerical range of the dimensions of the state data and disturbance data, filtering out high-frequency noise interference in the signal, and constructing standardized state data and standardized disturbance data arranged by time steps.

[0077] In this embodiment, the operation of jointly encoding the standardized state data and the standardized perturbation data, performing a state perturbation mapping operation, and generating a state perturbation embedding feature vector sequence embedded in the hyperspherical manifold space includes:

[0078] After performing dimension matching processing on the standardized state data vector and standardized perturbation data vector corresponding to each time step, the vectors are concatenated to generate a joint input vector sequence with unified dimensions.

[0079] The joint input vector sequence is input into an encoding structure consisting of several linear feature transformation layers and nonlinear activation functions. Linear mapping, nonlinear response, and feature reconstruction operations are performed sequentially to obtain an intermediate encoded feature sequence.

[0080] Based on the continuous change magnitude of the perturbation data in the time dimension, a perturbation change weighting factor is constructed. The perturbation change weighting factor is used to adjust the response intensity of the state-related dimension in the intermediate coding features to generate a perturbation-enhanced coding feature sequence.

[0081] Perform feature amplitude normalization on each vector in the perturbation-enhanced coding feature sequence. Specifically, this includes: calculating the overall amplitude index of the vector in all feature dimensions, scaling all components of the vector to a unit length range according to the amplitude ratio, and satisfying the uniform norm constraint.

[0082] The normalized vectors are embedded into a unit hyperspherical manifold space with Riemannian geometry. The embedded vectors maintain geodesic consistency and spatial continuity within the unit hyperspherical manifold space, forming a sequence of state perturbation embedded feature vectors.

[0083] This invention constructs a joint encoding network based on adjustable linear mapping and perturbation coupled activation, which maps standardized data to the unit hyperspherical manifold space of Riemannian geometry, realizing the geometric embedding expression of high-dimensional perturbation-state features. Compared with existing Euclidean space embedding models, this invention retains the local structural information of the perturbation direction on the manifold, improving the model's response consistency and embedding accuracy to dynamic changes in perturbation.

[0084] In this embodiment, the embedding operation of the unit hyperspherical manifold space with Riemannian geometry includes:

[0085] Based on the feature dimension of the perturbation-enhanced coding feature sequence, the spatial dimension order of the unit hyperspherical manifold space is set, and an embedding boundary with a Riemannian metric structure is defined in the space. The embedding boundary constitutes a high-dimensional spherical space that satisfies the norm constant constraint.

[0086] Amplitude normalization operation is performed on each feature vector in the perturbation-enhanced coding feature sequence, and the values ​​of each feature component are scaled by a uniform ratio. The embedding points corresponding to all feature vectors are located on the spherical boundary under the unit norm constraint.

[0087] The distance metric between embedding points in the manifold space is set to geodesic distance based on the inner product cosine function, and the angular structure relationship between the original feature vectors is preserved during the construction of the embedding vector sequence.

[0088] All embedding vectors in the perturbation-enhanced coding feature sequence are arranged in chronological order. During the embedding process, the embedding points between adjacent time steps are constrained to form a continuous curve sequence, which is a set of smooth embedding trajectories constructed in the manifold space.

[0089] This invention introduces a unit hyperspherical manifold as an embedding space structure in robot control for the first time. By constructing spatial constraints with Riemannian geometric properties, it achieves angular stability control and perturbation alignment between features, overcoming problems such as embedding jumps and directional scattering in existing spaces. The introduction of this structure effectively enhances the continuity and directional consistency of the embedding expression, which is a key step in improving the quality of control volume modeling.

[0090] In this embodiment, the process of generating the activation state vector sequence includes:

[0091] The state perturbation embedding feature vector sequence arranged in chronological order is input into a neural computation structure composed of several layers of hypersurface neurons. Each layer of the neural computation structure is composed of a fixed number of hypersurface neuron units. All neurons receive the embedding vector from the previous layer and perform feature transformation.

[0092] In each hypersurface neuron, an internal orientation bias mapping unit is constructed to receive the perturbation orientation vector generated during the embedding construction process of the corresponding perturbation enhancement coding feature vector. The perturbation orientation vector is then multiplied by the current input embedding vector to obtain the current perturbation orientation response value.

[0093] The activation threshold of the neuron is dynamically adjusted based on the perturbation direction response value. The activation function performs nonlinear response processing on the input signal based on the adjusted threshold and outputs the neuron activation value after perturbation direction adjustment.

[0094] The activation values ​​of all neurons in the neural computation structure are aggregated to form the activation state vector sequence at the current time step, and arranged in chronological order to form the final output activation state vector sequence.

[0095] The virtual twin controller proposed in this invention adopts a combination of hypersurface neuron structure and perturbation direction-sensitive activation function to realize nonlinear response calculation to manifold embedding features. Perturbation information is preserved at the neuron level. Compared with traditional neural network models, this invention has stronger direction perception ability and response layer differentiation ability, and improves the policy generalization and response distribution accuracy in complex perturbation environments.

[0096] In this embodiment, the operation of calculating the embedding space difference vector between the virtual control action prediction sequence and the entity control action prediction sequence, and updating the connection weights and bias parameters between hypersurface neurons based on the consistency optimization criterion, includes:

[0097] Amplitude normalization is performed on the virtual control action prediction sequence and the entity control action prediction sequence respectively to generate a control action embedding vector sequence that satisfies the unit norm constraint. The action embedding vector sequence is then mapped to the unit hyperspherical manifold space with the same Riemannian geometry used by the state disturbance embedding feature vector sequence to form the virtual action embedding trajectory and the entity action embedding trajectory.

[0098] In this manifold space, the angle between the virtual action embedding vector and the entity action embedding vector at each time step is calculated based on the tensor inner product relation, generating the main difference term of the geodesic distance representation;

[0099] Construct a sequence of angle offsets between the perturbation direction vector and the change direction of the action embedding vector, as the perturbation direction offset term;

[0100] Calculate the rate of change of angle between action embedding vectors in adjacent time steps within the manifold space, and use it as the action prediction continuity offset term;

[0101] Using the main difference term, disturbance direction offset term, and action continuity offset term as inputs, a consistency optimization criterion is constructed. The consistency optimization criterion is a weighted multinomial loss function defined in the unit hyperspherical manifold space of Riemannian geometry, which is used to measure the degree of spatial deviation between virtual control actions and physical control actions.

[0102] For each hypersurface neuron unit constituting the virtual twin control body, the weighted multinomial loss function constructed based on the consistency optimization criterion performs partial derivative calculation operations on the neuron connection weight matrix and the bias parameter vector, respectively. The partial derivative is the derivative result of the weighted multinomial loss function with respect to the parameters.

[0103] Based on the obtained partial derivative values, gradient descent is used to update each parameter according to the preset learning rate. The update direction is the opposite of the partial derivative, and the update step size is determined by the product of the learning rate and the magnitude of the partial derivative. This completes the iterative optimization of the connection weights and bias parameters in the current training round.

[0104] This invention designs a consistency optimization criterion that combines a triple structure of geodesic distance, direction vector angle offset, and time series angle change rate to achieve accurate gradient updates of virtual control volume parameters in Riemannian manifold space. Compared to traditional optimization methods using Euclidean loss functions, this method effectively maintains the geometric consistency of control action predictions, avoids policy direction drift or misleading updates during optimization, and improves convergence reliability.

[0105] In this embodiment, the inference operation on the standardized state data and standardized disturbance data includes: aligning and concatenating the standardized state data and standardized disturbance data in the time dimension, inputting them into the converged virtual twin control body, sequentially performing connection weight calculation and activation function processing of the hypersurface neuron, generating an activation state vector sequence, and outputting a target control action sequence based on the activation state vector sequence.

[0106] In this embodiment, the control strategy optimization process includes:

[0107] The target control action sequence is input into the entity execution structure to execute control instructions, the actual execution state at each time step is collected, and a feedback vector sequence is constructed.

[0108] The degree of difference between virtual and real control is determined by the geodesic distance between the virtual control action sequence and the feedback vector sequence in the unit hyperspherical manifold space of the Riemannian geometry.

[0109] When the geodesic distance exceeds the preset threshold and the mean square rate of change of the disturbance data is higher than the disturbance static threshold ε1 in M ​​consecutive time steps, the virtual-real synchronous retraining strategy is activated. The feedback vector and the disturbance data are jointly input into the virtual twin control body to perform several rounds of disturbance adversarial training to enhance control robustness.

[0110] When the geodesic distance exceeds the preset threshold but the mean square rate of change of the disturbance data within M consecutive time steps is lower than the static threshold ε1 of the disturbance, the direction correction update strategy between the target control action sequence and the feedback vector is executed, and only the output layer connection weights are locally fine-tuned to maintain the stability of the previous strategy.

[0111] When the geodesic distance is lower than the preset threshold, but the geodesic distance between the feedback vector and the virtual control action vector at the corresponding time step is greater than the deviation threshold ε2 within N consecutive time steps, the deviation tolerance strategy module is invoked to smooth the entity feedback by adjusting the control action tolerance range constraint, thereby maintaining the consistency of the overall control process.

[0112] Based on the optimization results under the three scenarios, the parameters of the virtual twin controller are updated to complete the dynamic optimization of the control strategy based on the virtual-real collaborative training mechanism.

[0113] This invention constructs a control strategy optimization process based on virtual and real feedback collaboration, classifies and processes disturbance fluctuation amplitude and action error type, and forms a control body reconstruction scheme that can be adjusted in real time through an optimization path selection mechanism triggered by three feedback conditions. Unlike the existing static iterative optimization process, this invention can autonomously decide on optimization strategies in multiple scenarios and states, and achieve collaborative optimization of control stability and response sensitivity.

[0114] The robot virtual-real collaborative training decision optimization system based on digital twin according to an embodiment of the present invention includes the following modules:

[0115] The status and disturbance acquisition module is used to acquire status data and disturbance data of the physical robot in a real environment;

[0116] The data preprocessing module is used to perform standardization processing on state data and disturbance data to generate standardized state data and disturbance data;

[0117] The joint encoding mapping module is used to jointly encode standardized state data and perturbation data, perform state perturbation mapping, and generate a sequence of state perturbation embedded feature vectors.

[0118] The virtual twin control module is used to receive state perturbation embedding feature vector sequences based on hypersurface neuron structures and generate activation state vector sequences.

[0119] The motion prediction path module is used to receive the activation state vector sequence and output the virtual control motion prediction sequence and the entity control motion prediction sequence, respectively.

[0120] The parameter optimization and update module is used to update the parameters in the virtual twin controller body until convergence based on the embedding space difference vector between the action prediction sequences.

[0121] The action strategy reasoning module is used to input standardized state data and disturbance data into the converged control body and output the target control action sequence.

[0122] The control execution module is used to receive the target control action sequence, generate control instructions, and drive the physical robot to perform the corresponding actions.

[0123] Example 1:

[0124] To verify the feasibility of this invention in practice, it was applied to a mobile assembly robot platform exhibiting typical dynamic disturbance characteristics. This robot needs to respond in real time to disturbances of different frequencies, directions, and amplitudes during material handling, mechanical component assembly, and other processes, while maintaining the accuracy and continuity of its motion path. In traditional solutions, this type of robot experiences a reduction in motion accuracy to below 82% when facing high-frequency interference (such as robotic arm resonance or rapid environmental changes), with significant control response delays, exhibiting characteristics of strategy lag and difficulty in adjustment.

[0125] In this implementation, the robot operates in a real-world scenario. A state and disturbance acquisition module continuously records its six-dimensional attitude information (displacement and attitude angles), end effector angular velocity, load weight change rate, and joint current fluctuations, forming a set of state and disturbance data inputs. This set is then normalized and noise-filtered to obtain standardized state and disturbance data. Subsequently, a joint encoding mapping module maps this data to a unit hyperspherical manifold space, constructing a disturbance embedding representation and resolving the coupling error problem between different disturbance types.

[0126] The mapped state perturbation embedding feature vector sequence is input into a virtual twin control volume constructed from hypersurface neurons, generating a response value sequence via a perturbation direction-sensitive activation mechanism. After generating predicted action sequences in both the virtual and physical paths, the geodesic loss and angular change rate are calculated using a consistency optimization criterion. Batch updates are then performed on the connection weights and bias parameters within the neuron structure, ensuring that the control volume model maintains convergence even in high perturbation frequency regions.

[0127] Finally, the converged control model is used as the action policy inference structure and deployed on an actual robot platform. The action sequence output by the inference can be directly used to generate real-time control commands to drive the robot to complete the assembly task. After 15 rounds of repeated testing, the robot's action execution accuracy under different perturbation frequencies and directions improved to an average of 96.4%, which is 13.8% higher than that of traditional policy control. In particular, it can maintain path stability even when there are continuous perturbation changes. In addition, in terms of the overall training cycle, the convergence rate of the model constructed in this invention is shortened by an average of 21.5%, and the inference latency is controlled within 15 milliseconds.

[0128] By setting typical dynamic disturbance scenarios such as disturbance signal amplitude levels, high-frequency impact disturbance injection, and low-speed action switching process disturbances, the control performance indicators of the present invention and the traditional DNN (deep neural network) control volume structure under different disturbance conditions are compared, and the following comparison data table is formed:

[0129] Table 1: Performance Comparison Data of Different Control Methods under Dynamic Disturbance Environment

[0130]

[0131] Based on the data in Table 1, "Performance Comparison Data of Different Control Methods under Dynamic Disturbance Environment," it is clear that the method of this invention exhibits superior control performance and stronger environmental adaptability compared to the traditional DNN control method under multiple disturbance scenarios. The following is a detailed analysis based on four core performance indicators:

[0132] In terms of motion accuracy, the method of this invention outperforms the traditional DNN method under all disturbance conditions. In a undisturbed environment, the accuracy reaches 98.7%, an improvement of 2.5 percentage points compared to the traditional DNN. Under high-frequency impact disturbances, the method of this invention maintains an accuracy of 94.9%, while the traditional DNN drops to 81.6%, widening the gap to 13.3 percentage points. In environments with continuous directional changes and complex multi-source disturbances, the method of this invention maintains 93.2% and 91.4% accuracy, respectively, while the traditional DNN achieves only 79.3% and 77.2%. These results demonstrate that the method of this invention can maintain high control accuracy under complex dynamic disturbance conditions and possesses stronger robustness.

[0133] Regarding control response latency, the inference speed of the control volume in this invention is faster. Under all test conditions, the response time is consistently maintained at around 15ms; while the response latency of traditional DNN control methods is above 25ms, and even approaches 30ms under high-frequency and complex perturbations. This invention achieves smooth real-time action response through efficient state perturbation embedding and hypersurface neuron structure.

[0134] Regarding the training convergence cycle, the present invention reduces the number of training rounds required. In high-interference environments, traditional DNN training requires close to 100 rounds or even more to converge, while the method of the present invention can achieve convergence in most cases within 70 rounds, shortening the training cycle by more than 25% and improving the efficiency of model deployment.

[0135] Regarding execution failure rate, this invention also exhibits an extremely low control error rate. In multi-source disturbance scenarios, the failure rate of traditional DNN methods is as high as 12.4%, while the failure rate of the control body of this invention is only 2.7%. Under high-frequency impacts, this invention can still maintain an error probability of less than 2.1%, indicating that its control capability against dynamic disturbances is more reliable.

[0136] In summary, this invention achieves high-precision, low-latency, and low-failure-rate control strategy optimization under various perturbation conditions by introducing joint modeling of state perturbations, manifold embedding representation, perturbation direction-sensitive neural structures, and embedding consistency optimization mechanisms. This outperforms existing DNN control methods and verifies the advanced nature and engineering practicality of our method in the field of dynamic robot control.

[0137] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A robot virtual-real collaborative training decision optimization method based on digital twins, characterized in that, Includes the following steps: Collect state data and disturbance data of the physical robot in the real operating environment, perform preprocessing operations, and obtain standardized state data and disturbance data; Standardized state data and perturbation data are jointly encoded and processed, and a state perturbation mapping operation is performed to generate a sequence of state perturbation embedding feature vectors embedded in the hyperspherical manifold space. The state perturbation is embedded into the feature vector sequence input and constructed in a virtual twin control body of a hypersurface neuron structure. Inside the control body, the perturbation direction-sensitive activation function is used to calculate the feature response of each neuron to obtain the activation state vector sequence. The activation state vector sequence is input into the virtual action prediction path and the real action prediction path respectively to generate the virtual control action prediction sequence and the entity control action prediction sequence respectively. The embedding space difference vector between the virtual control action prediction sequence and the entity control action prediction sequence is calculated. The connection weights and bias parameters between hypersurface neurons are updated based on the consistency optimization criterion. After training convergence, the converged virtual twin control body is obtained. The converged virtual twin controller is used as the action policy reasoning structure to perform reasoning operations on standardized state data and standardized disturbance data to generate the target control action sequence. The target control action sequence is input into the control instruction generation process of the physical robot, which drives the physical robot to complete the action execution, realizing the control strategy optimization process based on virtual-real collaborative training.

2. The robot virtual-real collaborative training decision optimization method based on digital twin as described in claim 1, characterized in that, The physical robot's state and disturbance data in the real operating environment include joint position data, joint velocity data, motor current data, end effector posture data, external force contact sensing data, ambient temperature data, and structural vibration acceleration data.

3. The robot virtual-real collaborative training decision optimization method based on digital twin according to claim 2, characterized in that, The preprocessing operations include: aligning the state data and disturbance data with timestamps, performing data interpolation and completion operations, unifying the numerical range of the dimensions of the state data and disturbance data, filtering out high-frequency noise interference in the signal, and constructing standardized state data and standardized disturbance data arranged by time steps.

4. The robot virtual-real collaborative training decision optimization method based on digital twin as described in claim 3, characterized in that, The operation of jointly encoding standardized state data and standardized perturbation data, performing state perturbation mapping operation, and generating a sequence of state perturbation embedding feature vectors embedded in the hyperspherical manifold space includes: After performing dimension matching processing on the standardized state data vector and standardized perturbation data vector corresponding to each time step, the vectors are concatenated to generate a joint input vector sequence with unified dimensions. The joint input vector sequence is input into an encoding structure consisting of several linear feature transformation layers and nonlinear activation functions. Linear mapping, nonlinear response, and feature reconstruction operations are performed sequentially to obtain an intermediate encoded feature sequence. Based on the continuous change magnitude of the perturbation data in the time dimension, a perturbation change weighting factor is constructed. The perturbation change weighting factor is used to adjust the response intensity of the state-related dimension in the intermediate coding features to generate a perturbation-enhanced coding feature sequence. Perform feature amplitude normalization on each vector in the perturbation-enhanced coding feature sequence. Specifically, this includes: calculating the overall amplitude index of the vector in all feature dimensions, scaling all components of the vector to a unit length range according to the amplitude ratio, and satisfying the uniform norm constraint. The normalized vectors are embedded into a unit hyperspherical manifold space with Riemannian geometry. The embedded vectors maintain geodesic consistency and spatial continuity within the unit hyperspherical manifold space, forming a sequence of state perturbation embedded feature vectors.

5. The robot virtual-real collaborative training decision optimization method based on digital twin according to claim 4, characterized in that, The embedding operation of the unit hyperspherical manifold space with Riemannian geometry includes: Based on the feature dimension of the perturbation-enhanced coding feature sequence, the spatial dimension order of the unit hyperspherical manifold space is set, and an embedding boundary with a Riemannian metric structure is defined in the space. The embedding boundary constitutes a high-dimensional spherical space that satisfies the norm constant constraint. Amplitude normalization operation is performed on each feature vector in the perturbation-enhanced coding feature sequence, and the values ​​of each feature component are scaled by a uniform ratio. The embedding points corresponding to all feature vectors are located on the spherical boundary under the unit norm constraint. The distance metric between embedding points in the manifold space is set to geodesic distance based on the inner product cosine function, and the angular structure relationship between the original feature vectors is preserved during the construction of the embedding vector sequence. All embedding vectors in the perturbation-enhanced coding feature sequence are arranged in chronological order. During the embedding process, the embedding points between adjacent time steps are constrained to form a continuous curve sequence, which is a set of smooth embedding trajectories constructed in the manifold space.

6. The robot virtual-real collaborative training decision optimization method based on digital twin according to claim 5, characterized in that, The process of generating the activation state vector sequence includes: The state perturbation embedding feature vector sequence arranged in chronological order is input into a neural computation structure composed of several layers of hypersurface neurons. Each layer of the neural computation structure is composed of a fixed number of hypersurface neuron units. All neurons receive the embedding vector from the previous layer and perform feature transformation. In each hypersurface neuron, an internal orientation bias mapping unit is constructed to receive the perturbation orientation vector generated during the embedding construction process of the corresponding perturbation enhancement coding feature vector. The perturbation orientation vector is then multiplied by the current input embedding vector to obtain the current perturbation orientation response value. The activation threshold of the neuron is dynamically adjusted based on the perturbation direction response value. The activation function performs nonlinear response processing on the input signal based on the adjusted threshold and outputs the neuron activation value after perturbation direction adjustment. The activation values ​​of all neurons in the neural computation structure are aggregated to form the activation state vector sequence at the current time step, and arranged in chronological order to form the final output activation state vector sequence.

7. The robot virtual-real collaborative training decision optimization method based on digital twin as described in claim 6, characterized in that, The operation of calculating the embedding space difference vector between the virtual control action prediction sequence and the entity control action prediction sequence, and updating the connection weights and bias parameters between hypersurface neurons based on the consistency optimization criterion, includes: Amplitude normalization is performed on the virtual control action prediction sequence and the entity control action prediction sequence respectively to generate a control action embedding vector sequence that satisfies the unit norm constraint. The action embedding vector sequence is then mapped to the unit hyperspherical manifold space with the same Riemannian geometry used by the state disturbance embedding feature vector sequence to form the virtual action embedding trajectory and the entity action embedding trajectory. In this manifold space, the angle between the virtual action embedding vector and the entity action embedding vector at each time step is calculated based on the tensor inner product relation, generating the main difference term of the geodesic distance representation; Construct a sequence of angle offsets between the perturbation direction vector and the change direction of the action embedding vector, as the perturbation direction offset term; Calculate the rate of change of angle between action embedding vectors in adjacent time steps within the manifold space, and use it as the action prediction continuity offset term; Using the main difference term, disturbance direction offset term, and action continuity offset term as inputs, a consistency optimization criterion is constructed. The consistency optimization criterion is a weighted multinomial loss function defined in the unit hyperspherical manifold space of Riemannian geometry, which is used to measure the degree of spatial deviation between virtual control actions and physical control actions. For each hypersurface neuron unit constituting the virtual twin control body, the weighted multinomial loss function constructed based on the consistency optimization criterion performs partial derivative calculation operations on the neuron connection weight matrix and the bias parameter vector, respectively. The partial derivative is the derivative result of the weighted multinomial loss function with respect to the parameters. Based on the obtained partial derivative values, gradient descent is used to update each parameter according to the preset learning rate. The update direction is the opposite of the partial derivative, and the update step size is determined by the product of the learning rate and the magnitude of the partial derivative. This completes the iterative optimization of the connection weights and bias parameters in the current training round.

8. The robot virtual-real collaborative training decision optimization method based on digital twin according to claim 7, characterized in that, The inference operation on the standardized state data and standardized perturbation data includes: aligning and concatenating the standardized state data and standardized perturbation data in the time dimension, inputting them into the converged virtual twin control body, sequentially performing connection weight calculation and activation function processing of the hypersurface neuron, generating an activation state vector sequence, and outputting a target control action sequence based on the activation state vector sequence.

9. The robot virtual-real collaborative training decision optimization method based on digital twin according to claim 8, characterized in that, The control strategy optimization process includes: The target control action sequence is input into the entity execution structure to execute control instructions, the actual execution state at each time step is collected, and a feedback vector sequence is constructed. The degree of difference between virtual and real control is determined by the geodesic distance between the virtual control action sequence and the feedback vector sequence in the unit hyperspherical manifold space of the Riemannian geometry. When the geodesic distance exceeds the preset threshold and the mean square rate of change of the disturbance data is higher than the disturbance static threshold ε1 in M ​​consecutive time steps, the virtual-real synchronous retraining strategy is activated. The feedback vector and the disturbance data are jointly input into the virtual twin control body to perform several rounds of disturbance adversarial training to enhance control robustness. When the geodesic distance exceeds the preset threshold but the mean square rate of change of the disturbance data within M consecutive time steps is lower than the static threshold ε1 of the disturbance, the direction correction update strategy between the target control action sequence and the feedback vector is executed, and only the output layer connection weights are locally fine-tuned to maintain the stability of the previous strategy. When the geodesic distance is lower than the preset threshold, but the geodesic distance between the feedback vector and the virtual control action vector at the corresponding time step is greater than the deviation threshold ε2 within N consecutive time steps, the deviation tolerance strategy module is invoked to smooth the entity feedback by adjusting the control action tolerance range constraint, thereby maintaining the consistency of the overall control process. Based on the optimization results under the three scenarios, the parameters of the virtual twin controller are updated to complete the dynamic optimization of the control strategy based on the virtual-real collaborative training mechanism.

10. A robot virtual-real collaborative training decision optimization system based on digital twins, applied to the robot virtual-real collaborative training decision optimization method based on digital twins as described in any one of claims 1 to 9, characterized in that, Includes the following modules: The status and disturbance acquisition module is used to acquire status data and disturbance data of the physical robot in a real environment; The data preprocessing module is used to perform standardization processing on state data and disturbance data to generate standardized state data and disturbance data; The joint encoding mapping module is used to jointly encode standardized state data and perturbation data, perform state perturbation mapping, and generate a sequence of state perturbation embedded feature vectors. The virtual twin control module is used to receive state perturbation embedding feature vector sequences based on hypersurface neuron structures and generate activation state vector sequences. The motion prediction path module is used to receive the activation state vector sequence and output the virtual control motion prediction sequence and the entity control motion prediction sequence, respectively. The parameter optimization and update module is used to update the parameters in the virtual twin controller body until convergence based on the embedding space difference vector between the action prediction sequences. The action strategy reasoning module is used to input standardized state data and disturbance data into the converged control body and output the target control action sequence. The control execution module is used to receive the target control action sequence, generate control instructions, and drive the physical robot to perform the corresponding actions.

Images