Model training method, device and storage medium for multiple-configuration robot

Abstract

This application provides a model training method, device, and storage medium for robots with various configurations, relating to the field of robot control technology. The method includes: constructing an initial task dynamic representation model; training an initial unified visual-motor network (VLA) based on a dataset of robotic arm operations for robots with various configurations to obtain a target unified visual-motor network (VLA); using the target VLA, extracting features from the robotic arm operation image sequence and the corresponding robotic arm state sequence to obtain a latent space visual motion representation; and using the latent space visual motion representation as an auxiliary supervision signal, training an initial VLA policy network based on the robotic arm operation image sequence, the corresponding robotic arm state sequence, and the corresponding natural language commands to obtain a target VLA policy network. This application enhances the robustness and generality of model policy learning, improving the execution performance of robots with various configurations.

Description

Technical Field

[0001] This application relates to the field of robot control technology, and more specifically, to a model training device and storage medium for robots with various configurations. Background Technology

[0002] In the field of embodied intelligence, one of the core requirements for robots to perform complex tasks is to achieve accurate understanding of human natural language instructions and autonomously complete a series of actions in conjunction with visual perception. With the rapid development of vision-language models, realizing the correlation between visual input, semantic instructions, and low-level motion control of robots has become an important research direction.

[0003] To achieve the mapping from high-dimensional perception to low-dimensional action, existing technologies mainly use techniques such as vector quantization variational autoencoders to discretize and encode continuous action sequences in robot teaching, generating a compact action codebook. In the policy learning stage, the target action code is predicted and decoded into actual control instructions. Alternatively, a pure visual dynamic coding strategy can be used to extract visual dynamic features from massive amounts of unlabeled human or robot operation videos using self-supervised learning.

[0004] However, existing motion sequence encoding strategies rely heavily on large-scale, precisely labeled robot motion data, which is costly to collect and limited by the kinematic characteristics of specific configurations, resulting in poor model versatility. While pure visual dynamic encoding can utilize abundant unlabeled video data, it lacks clear motion grounding, making it difficult to reliably map visual changes to specific motor control signals, thus affecting actual execution performance. Summary of the Invention

[0005] The purpose of this application is to address the shortcomings of the prior art by providing a model training method, device, and storage medium for robots with various configurations, thereby enhancing the robustness of model policy learning and the versatility of the model, and improving the performance of the robots with different configurations.

[0006] To achieve the above objectives, the technical solutions adopted in the embodiments of this application are as follows:

[0007] In a first aspect, one embodiment of this application provides a model training method for robots with various configurations, the method comprising:

[0008] An initial task dynamic representation model is constructed, which includes an initial unified visual action network and an initial visual language action (VLA) policy network.

[0009] Based on the robotic arm operation dataset for various robot configurations, the initial unified visual-motor network is trained to obtain the target unified visual-motor network; wherein, the robotic arm operation dataset includes: robotic arm operation image sequence, corresponding robotic arm state sequence, and corresponding natural language instructions;

[0010] The target unified visual motion network is used to extract features from the robotic arm operation image sequence and the corresponding robotic arm state sequence to obtain the latent space visual motion representation.

[0011] Using the latent space visual motion representation as an auxiliary supervision signal, the initial VLA policy network is trained based on the robotic arm operation image sequence, the corresponding robotic arm state sequence, and the corresponding natural language instructions to obtain the target VLA policy network.

[0012] Optionally, the initial unified visual-motion network includes: an initial visual encoder, an initial visual decoder, an initial motion encoder, and an initial motion decoder; the step of training the initial unified visual-motion network based on a robotic arm operation dataset for various robot configurations to obtain the target unified visual-motion network includes:

[0013] The initial visual encoder is used to jointly encode the current visual observations and the real visual observations at future moments in the sequence of robot arm operation images to obtain a continuous visual latent vector.

[0014] The initial motion encoder is used to jointly encode the robotic arm state information at multiple consecutive moments in the robotic arm state sequence to obtain a continuous motion potential vector.

[0015] The continuous visual latent vector and the continuous action latent vector are quantized respectively to obtain visual quantization information and action quantization information.

[0016] The initial visual decoder is used to decode the visual quantization information to obtain the predicted visual observations for the future time.

[0017] The initial motion decoder is used to decode the motion quantization information to obtain the predicted robotic arm state information for the future time.

[0018] Based on the predicted visual observations and actual visual observations at the future time, as well as the predicted robotic arm state information and actual robotic arm state information at the future time, the initial visual encoder and the initial motion encoder are tuned to obtain the target visual encoder and the target motion encoder. The target unified visual motion network includes the target visual encoder and the target motion encoder.

[0019] Optionally, the step of quantizing the continuous visual latent vector and the continuous action latent vector respectively to obtain visual quantization information and action quantization information includes:

[0020] Using a preset nearest neighbor search method, the continuous visual latent vectors are quantized in a preset unified visual motion code (UVMC) to obtain and determine the encoding terms corresponding to the continuous visual latent vectors as the visual quantization information.

[0021] Using the preset nearest neighbor search method, the continuous action potential vector is quantized in the UVMC to obtain and determine the encoding item corresponding to the continuous action potential vector as the action quantization information;

[0022] The UVMC itself has: multiple preset coding terms, each preset coding term corresponding to a preset visual latent vector and a corresponding action latent vector.

[0023] Optionally, the step of using the target unified visual motion network to extract features from the robotic arm operation image sequence and the corresponding robotic arm state sequence to obtain a latent space visual motion representation includes:

[0024] The target visual encoder in the target unified visual action network is used to encode the current visual observation in the sequence of robotic arm operation images to obtain the target visual feature vector;

[0025] The target motion encoder in the target unified visual motion network is used to encode the current state information of the robotic arm in the robotic arm state sequence to obtain the target motion feature vector.

[0026] The target visual feature vector and the target action feature vector are quantized respectively to obtain target visual quantization information and target action quantization information as the latent space visual motion representation.

[0027] Optionally, the step of tuning the initial visual encoder and the initial motion encoder based on the predicted visual observations and actual visual observations at the future time, as well as the predicted robotic arm state information and actual robotic arm state information at the future time, to obtain the target visual encoder and the target motion encoder, includes:

[0028] Calculate the first reconstruction loss value based on the predicted visual observations and the actual visual observations at the future time.

[0029] Calculate the first quantization loss value based on the visual quantization information and the continuous visual latent vector;

[0030] The second reconstruction loss value is calculated based on the predicted robotic arm state information and the actual robotic arm state information at the future time.

[0031] Calculate the second quantization loss value based on the action quantization information and the continuous action potential vector;

[0032] Based on the first reconstruction loss value and the first quantization loss value, the initial visual encoder is tuned to obtain the target visual encoder;

[0033] Based on the second reconstruction loss value and the second quantization loss value, the initial motion encoder is tuned to obtain the target motion encoder.

[0034] Optionally, calculating the first quantization loss value based on the visual quantization information and the continuous visual latent vector includes:

[0035] Based on the visual quantization information and the continuous visual latent vector, calculate the first codebook loss and the first commitment loss value respectively;

[0036] Calculate the first quantization loss value based on the first codebook loss and the first commitment loss value;

[0037] The step of calculating the second quantization loss value based on the action quantization information and the continuous action latent vector includes:

[0038] Based on the action quantization information and the continuous action potential vector, calculate the second codebook loss and the second commitment loss value respectively;

[0039] The second quantization loss value is calculated based on the second codebook loss and the second commitment loss value.

[0040] Optionally, before tuning the initial visual encoder and the initial motion encoder based on the predicted visual observations and actual visual observations at the future time, as well as the predicted robotic arm state information and actual robotic arm state information at the future time, to obtain the target visual encoder and the target motion encoder, the method further includes:

[0041] The alignment loss function value is calculated based on the matching probability of the visual quantization information and the matching probability of the action quantization information;

[0042] Based on the alignment loss function value, the initial visual encoder and the initial motion encoder are tuned.

[0043] Optionally, the step of using the latent space visual motion representation as an auxiliary supervision signal to train the initial VLA policy network based on the robotic arm operation image sequence, the corresponding robotic arm state sequence, and the corresponding natural language instructions to obtain the target VLA policy network includes:

[0044] Using the initial VLA policy network, the robotic arm operation image sequence, the corresponding robotic arm state sequence, and the corresponding natural language command are predicted to obtain the predicted latent space features and the predicted action code.

[0045] Using the latent space visual motion representation as an auxiliary supervision signal, the latent space loss function of the predicted latent space features is calculated;

[0046] The cloning loss is calculated based on the predicted action code and the actual action code.

[0047] The initial VLA policy network is tuned based on the latent space loss function and the cloning loss to obtain the target VLA policy network.

[0048] Secondly, another embodiment of this application provides a model training device for robots with various configurations, the device comprising:

[0049] A construction module is used to build an initial task dynamic representation model, which includes an initial unified visual action network and an initial visual language action (VLA) policy network.

[0050] The first training module is used to train the initial unified visual-motor network based on the robotic arm operation dataset for robots of various configurations to obtain the target unified visual-motor network; wherein, the robotic arm operation dataset includes: robotic arm operation image sequence, corresponding robotic arm state sequence and corresponding natural language instructions.

[0051] The extraction module is used to extract features from the robotic arm operation image sequence and the corresponding robotic arm state sequence using the target unified visual motion network to obtain the latent space visual motion representation.

[0052] The second training module is used to train the initial VLA policy network using the latent space visual motion representation as an auxiliary supervision signal, based on the robotic arm operation image sequence, the corresponding robotic arm state sequence, and the corresponding natural language instructions, to obtain the target VLA policy network.

[0053] Thirdly, another embodiment of this application provides an electronic device, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and the processor executes the machine-readable instructions to perform the steps of the model training method for multi-configuration robots as described in any of the first aspects above.

[0054] Fourthly, another embodiment of this application provides a storage medium storing a computer program, which, when executed by a processor, performs the steps of the model training method for multi-configuration robots as described in any of the first aspects above.

[0055] The beneficial effects of this application are:

[0056] This application provides a model training method, device, and storage medium for robots with various configurations. It constructs an initial task dynamic representation model. Based on a dataset of robotic arm operations for robots with various configurations, an initial unified visual-motor network (VLA) is trained to obtain a target unified visual-motor network (VLA). Using the target VLA, features are extracted from the robotic arm operation image sequence and the corresponding robotic arm state sequence to obtain a latent space visual motion representation. Using this latent space visual motion representation as an auxiliary supervision signal, an initial VLA policy network is trained based on the robotic arm operation image sequence, the corresponding robotic arm state sequence, and the corresponding natural language commands to obtain a target VLA policy network. This application achieves the unification of vision and action by extracting the robotic arm operation image sequence and the corresponding robotic arm state sequence through the target unified visual-motor network. Simultaneously, the target VLA policy network improves the model's configuration transfer capability by predicting based on visual and language inputs, enabling configuration-independent policy learning. Through the target unified visual-motor network and the target VLA policy network, the robustness and generality of the model's policy learning are enhanced, thereby improving the execution performance of the robot with different configurations. Attached Figure Description

[0057] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 A flowchart illustrating a model training method for robots with multiple configurations provided in this application embodiment;

[0059] Figure 2A schematic diagram illustrating the process of obtaining a target unified visual-action network in a model training method for robots with multiple configurations provided in this application embodiment;

[0060] Figure 3 This is a schematic diagram illustrating the process of determining quantization information in a model training method for robots with multiple configurations provided in an embodiment of this application.

[0061] Figure 4 A schematic diagram illustrating the process of obtaining latent space visual motion representation in a model training method for robots with multiple configurations provided in this application embodiment;

[0062] Figure 5 A schematic diagram illustrating the process of obtaining the target encoder in a model training method for robots with multiple configurations provided in this application;

[0063] Figure 6 This is a schematic diagram illustrating the process of determining the first quantization loss value in a model training method for robots with multiple configurations, as provided in an embodiment of this application.

[0064] Figure 7 This is a schematic diagram illustrating the process of determining the second quantization loss value in a model training method for robots with multiple configurations, as provided in an embodiment of this application.

[0065] Figure 8 A flowchart illustrating the process of determining the alignment loss function in a model training method for robots with multiple configurations provided in this application embodiment;

[0066] Figure 9 A schematic diagram illustrating the process of determining the target VLA policy network in a model training method for robots with multiple configurations provided in this application embodiment;

[0067] Figure 10 A schematic diagram of a model training device for robots with multiple configurations provided in this application embodiment;

[0068] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0069] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0070] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0071] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0072] The model training method for robots with various configurations provided in this application can be applied to electronic devices with various robot configurations, as well as other electronic devices; this application does not limit its application to this. Robots with various configurations refer to robots with different geometries and postures, such as four-axis robotic arms, six-axis robotic arms, and seven-axis robotic arms. The configuration parameters of a six-axis robotic arm include the angles of the base rotation joint, the shoulder joint, the elbow joint, and the three wrist joints. To clearly describe the model training method for robots with various configurations provided in this application, the method will be described below with reference to several accompanying drawings. Figure 1 A flowchart illustrating a model training method for robots with various configurations provided in this application is shown below. Figure 1 As shown, the method includes:

[0073] Step 101: Construct the initial task dynamic representation model.

[0074] The initial task dynamic representation model includes an initial unified visual-action network and an initial visual-language-action (VLA) policy network. The initial unified visual-action network is a two-branch autoencoder network used to extract the same visual motion representation in the discrete latent space. The initial visual-language-action (VLA) policy network is the Vision-Language-Action (VLA) model.

[0075] Optionally, an initial task dynamic representation model is constructed based on a dual-branch autoencoder network and a visual-language-action (VLA) policy network.

[0076] Step 102: Based on the robotic arm operation dataset for various robot configurations, train the initial unified visual motion network to obtain the target unified visual motion network.

[0077] The robotic arm operation dataset includes: a sequence of robotic arm operation images, a corresponding sequence of robotic arm states, and corresponding natural language commands. The sequence of robotic arm operation images includes a sequence of consecutive frames of third-person view images of the robotic arm. The corresponding sequence of robotic arm states is the sequence of states of the robotic arm within that frame of the robotic arm operation image. For example, it may include: joint space states: the angle, velocity, and torque of each joint of the robotic arm; the position and attitude of the end effector; and, when the robotic arm has a gripper, the gripper opening degree and gripper force sensor readings, etc. This embodiment of the application does not impose limitations on these aspects.

[0078] Optionally, the robotic arm operation datasets of various robot configurations are input into the initial unified visual motion network, and the initial unified visual motion network is trained to obtain the target unified visual motion network.

[0079] Step 103: Using a target-unified visual motion network, feature extraction is performed on the sequence of robotic arm operation images and the corresponding sequence of robotic arm states to obtain the latent space visual motion representation.

[0080] The latent space visual motion representation is the result of splicing latent space visual representation and latent space motion representation. The latent space visual representation maps high-dimensional, redundant original image data to a low-dimensional, dense, semantically rich vector space, including low-level visual features, mid-level visual features, high-level semantic features, structured and relational information, style and texture information. The latent space motion representation is used to map motion sequences to a low-dimensional, dense word vector space, thereby obtaining the essence, pattern, and intent of the motion sequence, including: kinematic and dynamic information, temporal and rhythmic information, semantic and target information, style and personalization information, and interaction and contextual information, etc., which are not limited in this embodiment. Natural language instructions may include action instructions, task instructions, constraint instructions, etc., which are not limited in this embodiment.

[0081] Step 104: Using latent space visual motion representation as an auxiliary supervision signal, train the initial VLA policy network based on the robotic arm operation image sequence, the corresponding robotic arm state sequence, and the corresponding natural language instructions to obtain the target VLA policy network.

[0082] Optionally, the robotic arm operation image sequence, the corresponding robotic arm state sequence, and the corresponding natural language instructions are input into the initial VLA policy network to obtain the corresponding predicted features. The latent space visual motion representation is used as an auxiliary supervision signal to supervise the predicted features. The initial VLA policy network is trained based on the supervision results to obtain the target VLA policy network.

[0083] In this embodiment, an initial task dynamic representation model is constructed. Based on a dataset of robotic arm operations for various robot configurations, an initial unified visual-motor network (VLA) is trained to obtain a target unified visual-motor network (VLA). Using the target VLA, features are extracted from the robotic arm operation image sequence and the corresponding robotic arm state sequence to obtain a latent space visual motion representation. Using this latent space visual motion representation as an auxiliary supervision signal, an initial VLA policy network is trained based on the robotic arm operation image sequence, the corresponding robotic arm state sequence, and the corresponding natural language commands to obtain a target VLA policy network. This application achieves the unification of vision and action by extracting the robotic arm operation image sequence and the corresponding robotic arm state sequence through the target unified visual-motor network. Simultaneously, the target VLA policy network enhances the model's configuration transfer capability by predicting based on visual and language inputs, enabling configuration-independent policy learning. The target unified visual-motor network and the target VLA policy network strengthen the robustness of the model's policy learning and improve the execution performance of the configured robot.

[0084] Based on the above embodiments, the initial unified visual action network includes: an initial visual encoder, an initial visual decoder, an initial action encoder, and an initial action decoder. This application also provides a process for obtaining the target unified visual action network in a model training method for robots with various configurations. Figure 2 This application provides a schematic diagram illustrating the process of obtaining a target unified visual action network in a model training method for robots with multiple configurations, as shown in the embodiments of this application. Figure 2 As shown, in step 103 above, the initial unified visual motion network is trained based on the robotic arm operation dataset for various robot configurations to obtain the target unified visual motion network, including:

[0085] Step 201: Using an initial visual encoder, the current visual observations and the real visual observations at future moments in the robotic arm operation image sequence are jointly encoded to obtain a continuous visual latent vector.

[0086] The initial visual encoder is a self-supervised vision encoder, used to encode the robotic arm operation images by combining the robotic arm's state information from multiple consecutive time points, extracting its latent space representation. The current visual observation is defined as the image sequence in the robotic arm operation images at the current time, and the future real visual observation is the image sequence of the robotic arm operation images at future time points different from the current visual observation. The future time is determined according to specific needs, and this embodiment does not impose any limitations on it. The continuous visual latent vector is used to represent the core change vector between the current visual observation and the future real visual observation.

[0087] Optionally, an initial visual encoder is used to process the current visual observations in the sequence of robot arm operation images. Joint encoding with real visual observations of future moments. , to obtain continuous visual latent vectors , This is the initial visual encoder.

[0088] Step 202: Using an initial motion encoder, the state information of the robotic arm at multiple consecutive moments in the robotic arm state sequence is jointly encoded to obtain the potential vector of continuous motion.

[0089] The initial motion encoder is a self-supervised motion encoder, used to encode the robotic arm's state information at multiple consecutive time points in conjunction with images of the robotic arm's operation, extracting its latent space representation. The robotic arm's state information at multiple consecutive time points corresponds to the robotic arm's state information at multiple consecutive time points between the current visual observation and the actual visual observation at future time points. The continuous motion latent vector represents the core change vector from the robotic arm's state information at multiple consecutive time points.

[0090] Optionally, an initial motion encoder is used to process the robotic arm state information at multiple consecutive moments in the robotic arm state sequence. Joint encoding is performed to obtain the latent vector of continuous actions. .in, This is the initial motion encoder.

[0091] Step 203: Perform quantization operations on the continuous visual latent vector and the continuous motion latent vector respectively to obtain visual quantization information and motion quantization information.

[0092] The quantization operation is used to convert a continuous vector into a discrete vector.

[0093] Optionally, quantization operations are performed on the continuous visual latent vectors and the continuous motion latent vectors respectively to obtain visual quantization information and motion quantization information, so that the visual quantization information and motion quantization information are directly related.

[0094] Step 204: Use the initial visual decoder to decode the visual quantization information to obtain the predicted visual observations for future moments.

[0095] The initial visual decoder is a self-supervised visual decoder, which is used to reconstruct the visual quantization information to obtain the predicted visual observations for future moments.

[0096] Optionally, an initial visual decoder is used to decode and reconstruct the visual quantization information to obtain the predicted visual observations for future moments corresponding to the visual quantization information.

[0097] Step 205: Use the initial motion decoder to decode the motion quantization information to obtain the predicted state information of the robotic arm at future moments.

[0098] The initial motion decoder is a self-supervised motion decoder, which is used to reconstruct the motion quantization information to obtain the predicted state information of the robotic arm at future moments.

[0099] Optionally, an initial motion decoder is used to decode and reconstruct the motion quantization information to obtain the predicted robotic arm state information for future moments corresponding to the motion quantization information.

[0100] Step 206: Based on the predicted visual observations and actual visual observations at future times, as well as the predicted robotic arm state information and actual robotic arm state information at future times, adjust the parameters of the initial visual encoder and the initial motion encoder respectively to obtain the target visual encoder and the target motion encoder.

[0101] The unified visual-motion network for the target includes a target visual encoder and a target motion encoder. It also includes a target visual decoder and a target motion decoder. The real visual observations are the future real visual observations in the initial visual encoder. The real robotic arm state information is the robotic arm state information at the final moment among multiple consecutive moments of robotic arm state information.

[0102] Optionally, the loss of the initial visual encoder is obtained based on the predicted visual observations and the actual visual observations at future times, and the parameters of the initial visual encoder are tuned based on the loss of the initial visual encoder to obtain the target visual encoder.

[0103] Optionally, the loss of the initial motion encoder is obtained based on the predicted state information of the robotic arm at future time and the actual state information of the robotic arm. The parameters of the initial motion encoder are then adjusted based on the loss of the initial motion encoder to obtain the target motion encoder.

[0104] In this embodiment, an initial visual encoder is used to jointly encode the current visual observation and the real visual observation at future moments in the robotic arm operation image sequence to obtain a continuous visual latent vector. An initial motion encoder is used to jointly encode the robotic arm state information at multiple consecutive moments in the robotic arm state sequence to obtain a continuous motion latent vector. Quantization operations are performed on the continuous visual latent vector and the continuous motion latent vector to obtain visual quantization information and motion quantization information, respectively. An initial visual decoder is used to decode the visual quantization information to obtain the predicted visual observation at future moments. An initial motion decoder is used to decode the motion quantization information to obtain the predicted robotic arm state information at future moments. Based on the predicted visual observation and the real visual observation at future moments, as well as the predicted robotic arm state information and the real robotic arm state information at future moments, the initial visual encoder and the initial motion encoder are tuned to obtain the target visual encoder and the target motion encoder. This application achieves a unified representation of the robotic arm operation image sequence and the robotic arm state sequence through quantization operations, realizing cross-modal semantic alignment and improving the model's generalization ability and robustness.

[0105] Based on the above embodiments, this application also provides a process for determining quantization information in a model training method for robots with various configurations. Figure 3 This is a flowchart illustrating the process of determining quantization information in a model training method for robots with multiple configurations provided in this application embodiment. Figure 3 As shown, in step 203 above, quantization operations are performed on the continuous visual latent vector and the continuous action latent vector to obtain visual quantization information and action quantization information, including:

[0106] Step 301: Using a preset nearest neighbor search method, the continuous visual latent vectors are quantized in the preset unified visual motion code (UVMC) to obtain and determine the coding terms corresponding to the continuous visual latent vectors as visual quantization information.

[0107] The UVMC itself has multiple preset coding terms, each of which corresponds to a preset visual latent vector and a corresponding action latent vector.

[0108] Optionally, a preset nearest neighbor search method is used to quantize the continuous visual latent vectors in the preset unified visual motion code (UVMC) to obtain and determine the coding terms corresponding to the continuous visual latent vectors as visual quantization information.

[0109]

[0110]

[0111] in, For the i encoded information in the preset unified visual motion code (UVMC), To quantify information visually, For continuous visual latent vectors, For quantization operation functions, The j-th vector in the Unified Visual Motion Code (UVMC) that corresponds to the continuous visual latent vectors.

[0112] Step 302: Using a preset nearest neighbor search method, perform quantization operations on the continuous action latent vector in the UVMC to obtain and determine the coding terms corresponding to the continuous action latent vector as action quantization information.

[0113] The preset nearest neighbor search method is the encoded information obtained by calculating the minimum Euclidean distance.

[0114] Optionally, a preset nearest neighbor search method is used to quantize the latent vectors of continuous actions in the UVMC, and the coding terms corresponding to the latent vectors of continuous actions are obtained and determined as action quantization information.

[0115]

[0116]

[0117] in, For the i encoded information in the preset unified visual motion code (UVMC), To quantify information visually, For continuous visual latent vectors, For quantization operation functions, The j-th vector in the Unified Visual Motion Code (UVMC) that corresponds to the continuous visual latent vectors.

[0118] In this embodiment, a preset nearest neighbor search method is used to quantize continuous visual latent vectors in a preset unified visual motion code (UVMC) template, obtaining and determining the corresponding encoding terms as visual quantization information. Similarly, a preset nearest neighbor search method is used to quantize continuous action latent vectors in the UVMC template, obtaining and determining the corresponding encoding terms as action quantization information. This application quantizes both continuous visual latent vectors and continuous action latent vectors using the same UVMC template, achieving unified discretization of vision and action, and enhancing the model's generalization and reasoning capabilities through cross-modal semantic enhancement.

[0119] Based on the above embodiments, this application also provides a process for obtaining latent space visual motion representations in a model training method for robots with various configurations. Figure 4 This application provides a schematic diagram illustrating the process of obtaining latent space visual motion representations in a model training method for robots with various configurations, as shown in the embodiments of this application. Figure 4 As shown, in step 103 above, a target-unified visual motion network is used to extract features from the robotic arm operation image sequence and the corresponding robotic arm state sequence to obtain a latent space visual motion representation, including:

[0120] Step 401: Use the target visual encoder in the target unified visual action network to encode the current visual observation in the robot arm operation image sequence to obtain the target visual feature vector.

[0121] Among them, the target visual features are distributed representations obtained after the current visual view undergoes nonlinear transformation.

[0122] Optionally, a target visual encoder in a unified visual action network is used to encode the current visual observation in the sequence of robotic arm operation images, and extract the state information, environmental context information, interaction information and implicit information of the robotic arm from the current visual observation to obtain the target visual feature vector.

[0123] Step 402: Use the target action encoder in the target unified vision action network to encode the current state information of the robotic arm in the state sequence of the robotic arm to obtain the target action feature vector.

[0124] The target motion feature vector is a distributed representation of the current state information of the robotic arm after a nonlinear transformation.

[0125] Optionally, a target action encoder in a unified visual action network is used to encode the current state information of the robotic arm in the current state sequence, and extract the semantic category, motion dynamic features, action orientation features and task-related information from the current state information of the robotic arm to the target action feature vector.

[0126] Step 403: Perform quantization operations on the target visual feature vector and the target action feature vector respectively to obtain target visual quantization information and target action quantization information as latent space visual motion representation.

[0127] Optionally, the target visual feature vector and the target action feature vector are quantized separately to obtain target visual quantization information and target action quantization information. The target visual quantization information and target action quantization information are then concatenated to obtain the latent space visual motion representation.

[0128] In this embodiment, a target visual encoder in a unified target visual-motion network is used to encode the current visual observation in the robotic arm operation image sequence to obtain a target visual feature vector; a target motion encoder in a unified target visual-motion network is used to encode the robotic arm state information at the current moment in the robotic arm state sequence to obtain a target motion feature vector; the target visual feature vector and the target motion feature vector are quantized respectively to obtain target visual quantization information and target motion quantization information as latent space visual motion representations. This application obtains latent space visual motion representations based on target visual quantization information and target motion quantization information, enabling joint optimization of visual features and motion features within the same network architecture, promoting cross-modal alignment, and improving the generalization performance of imitation learning and visual servo control tasks.

[0129] Based on the above embodiments, this application also provides a process for obtaining the target encoder in a model training method for robots with various configurations. Figure 5 A flowchart illustrating the process of obtaining the target encoder in a model training method for robots with multiple configurations provided in this application is shown below. Figure 5 As shown, in step 206 above, based on the predicted visual observations and actual visual observations at future times, as well as the predicted robotic arm state information and actual robotic arm state information at future times, the initial visual encoder and initial motion encoder are tuned respectively to obtain the target visual encoder and target motion encoder, including:

[0130] Step 501: Calculate the first reconstruction loss value based on the predicted visual observations and the actual visual observations at future moments.

[0131] Optionally, based on predicted visual observations at future times. and real visual observation The first reconstruction loss value is calculated using the reconstruction loss calculation formula.

[0132]

[0133] in, The first reconstruction loss value, For predictive visual observation of future moments, For a true visual observation of future moments.

[0134] Step 502: Calculate the first quantization loss value based on the visual quantization information and the continuous visual latent vector.

[0135] The first quantization loss value is the part related to the quantization operation, such as a weighted sum of codebook loss and commitment loss.

[0136] Optionally, codebook loss and commitment loss are calculated based on visual quantization information and continuous visual latent vectors, and the first quantization loss value is calculated based on the weighted sum of codebook loss and commitment loss.

[0137] Step 503: Calculate the second reconstruction loss value based on the predicted state information of the robotic arm at future time and the actual state information of the robotic arm.

[0138] Optionally, based on the predicted state information of the robotic arm at future moments. and real robotic arm status information The second reconstruction loss value is calculated using the reconstruction loss calculation formula.

[0139]

[0140] in, This is the second reconstruction loss value. For predicting the state of the robotic arm at future moments, This represents the actual state information of the robotic arm.

[0141] Step 504: Calculate the second quantization loss value based on the action quantization information and the potential vector of continuous actions.

[0142] The second quantization loss value is the part related to the quantization operation, such as a weighted sum of codebook loss and commitment loss.

[0143] Optionally, codebook loss and commitment loss are calculated based on action quantization information and continuous action latent vectors, and a second quantization loss value is calculated based on the weighted sum of codebook loss and commitment loss.

[0144] Step 505: Based on the first reconstruction loss value and the first quantization loss value, the initial visual encoder is tuned to obtain the target visual encoder.

[0145] Optionally, based on the first reconstruction loss value and the first quantization loss value, all parameter gradients of the visual encoder are collected, an optimization algorithm is executed to update the parameters, and the target visual encoder is obtained when the preset number of training rounds is reached or the loss value converges to a preset threshold.

[0146] Optionally, the input to the initial encoder also includes any set of human video data. The initial encoder jointly encodes the current human visual observations and the real human visual observations at future moments in the human video data to obtain continuous human visual latent vectors. The human visual latent vectors are then quantized to obtain human visual quantization information. The initial visual decoder decodes the human visual quantization information to obtain the predicted human visual observations at future moments. Based on the predicted human visual observations and the real human visual observations at future moments, the initial visual encoder is tuned to obtain the target visual encoder.

[0147] Step 506: Based on the second reconstruction loss value and the second quantization loss value, adjust the parameters of the initial motion encoder to obtain the target motion encoder.

[0148] Optionally, based on the second reconstruction loss value and the second quantization loss value, all parameter gradients of the action encoder are collected, an optimization algorithm is executed to update the parameters, and the target action encoder is obtained when the preset number of training rounds is reached or the loss value converges to the preset threshold.

[0149] In this embodiment, a first reconstruction loss value is calculated based on predicted visual observations and actual visual observations at future times; a first quantization loss value is calculated based on visual quantization information and continuous visual latent vectors; a second reconstruction loss value is calculated based on predicted robotic arm state information and actual robotic arm state information at future times; a second quantization loss value is calculated based on motion quantization information and continuous motion latent vectors; the initial visual encoder is tuned based on the first reconstruction loss value and the first quantization loss value to obtain a target visual encoder; and the initial motion encoder is tuned based on the second reconstruction loss value and the second quantization loss value to obtain a target motion encoder. This application constructs a self-supervised forward prediction target using predicted and actual information, achieving bidirectional supervision and thus improving prediction accuracy.

[0150] Based on the above embodiments, this application also provides a process for determining the first quantization loss value in a model training method for robots with various configurations. Figure 6 This application provides a flowchart illustrating the process of determining the first quantization loss value in a model training method for robots with various configurations, as illustrated in the embodiments of this application. Figure 6 As shown, in step 502 above, the first quantization loss value is calculated based on the visual quantization information and the continuous visual latent vector, including:

[0151] Step 601: Calculate the first codebook loss and the first commitment loss value based on the visual quantization information and the continuous visual latent vector.

[0152] Optionally, based on visual quantification information and continuous visual latent vectors The first codebook loss is obtained by using the codebook loss calculation formula.

[0153]

[0154] in, For continuous visual latent vectors, To quantify information visually, To stop the gradient, This is a loss of the first codebook.

[0155] Optionally, based on visual quantification information and continuous visual latent vectors The first commitment loss is obtained by using the commitment loss calculation formula.

[0156]

[0157] in, For continuous visual latent vectors, To quantify information visually, To stop the gradient, Loss due to the first commitment.

[0158] Step 602: Calculate the first quantization loss value based on the first codebook loss and the first commitment loss value.

[0159] Optionally, the first quantization loss value is calculated using the quantization loss calculation formula based on the first codebook loss and the first commitment loss value.

[0160]

[0161] in, For continuous visual latent vectors, To quantify information visually, To stop the gradient. The weights of the first codebook loss and the first commitment loss value, First quantization loss value.

[0162] Based on the above embodiments, this application also provides a process for determining the second quantization loss value in a model training method for robots with various configurations. Figure 7 This application provides a flowchart illustrating the process of determining the second quantization loss value in a model training method for robots with various configurations, as illustrated in the embodiments of this application. Figure 7 As shown, in step 504 above, the second quantization loss value is calculated based on the action quantization information and the latent vector of continuous actions, including:

[0163] Step 701: Calculate the second codebook loss and the second commitment loss value based on the action quantization information and the continuous action potential vector.

[0164] Optionally, based on motion quantification information and the latent vector of continuous actions The second codebook loss is obtained by using the codebook loss calculation formula.

[0165]

[0166] in, For continuous visual latent vectors, To quantify information visually, To stop the gradient, This is a loss for the second codebook.

[0167] Optionally, based on motion quantification information and the latent vector of continuous actions The second commitment loss is obtained by using the commitment loss calculation formula.

[0168]

[0169] in, For the potential vector of continuous actions, To quantify information about actions, To stop the gradient, For the loss of the second commitment.

[0170] Step 702: Calculate the second quantization loss value based on the second codebook loss and the second commitment loss value.

[0171] Optionally, the second quantization loss value is calculated using the quantization loss calculation formula based on the second codebook loss and the second commitment loss value.

[0172]

[0173] in, For the potential vector of continuous actions, To quantify information about actions, To stop the gradient, The weights for the second codebook loss and the second commitment loss value, This is the second quantization loss value.

[0174] In this embodiment, a first codebook loss and a first commitment loss are calculated based on visual quantization information and continuous visual latent vectors, respectively; a first quantization loss is calculated based on the first codebook loss and the first commitment loss. A second codebook loss and a second commitment loss are calculated based on action quantization information and continuous action latent vectors, respectively; a second quantization loss is calculated based on the second codebook loss and the second commitment loss. This application, through codebook loss and quantization loss, improves the stability of model training and enhances the semantic consistency between visual and action features by optimizing both.

[0175] Based on the above embodiments, this application also provides a process for determining the alignment loss function in a model training method for robots with various configurations. Figure 8 This is a flowchart illustrating the determination of the alignment loss function in a model training method for robots with various configurations provided in this application. Figure 8 As shown, before adjusting the parameters of the initial visual encoder and the initial motion encoder in step 206 above, based on the predicted visual observations and actual visual observations at future times, as well as the predicted robotic arm state information and the actual robotic arm state information at future times, to obtain the target visual encoder and the target motion encoder, the method further includes:

[0176] Step 801: Calculate the alignment loss function value based on the matching probability of visual quantization information and the matching probability of action quantization information.

[0177] Among them, the matching probability of visual quantization information is the posterior probability distribution of visual quantization information in the codebook space, and the matching probability of action quantization information is the posterior probability distribution of action quantization information in the codebook space.

[0178] Optionally, the alignment loss function value is calculated using a preset alignment loss function calculation formula based on the matching probability of visual quantization information and the matching probability of action quantization information.

[0179]

[0180] in, To quantify the matching probability of visual information. The matching probability of quantifying action information. Let KL divergence be denoted as KL divergence.

[0181] Step 802: Adjust the parameters of the initial visual encoder and the initial motion encoder according to the alignment loss function value.

[0182] Optionally, based on the alignment loss function value, all parameter gradients of the initial visual encoder and the initial action encoder are collected, and an optimization algorithm is executed to update the parameters until a preset number of training rounds is reached or the loss value converges to a preset threshold.

[0183] In this embodiment, an alignment loss function value is calculated based on the matching probability of visual quantization information and the matching probability of action quantization information. The initial visual encoder and initial action encoder are then tuned based on the alignment loss function value. This application performs alignment based on probability distribution, enhancing robustness and generalization ability, supporting reasonable one-to-many or many-to-one mappings, and reducing the number of training samples.

[0184] Based on the above embodiments, this application also provides a process for determining the target VLA policy network in a model training method for robots with various configurations. Figure 9 This application provides a flowchart illustrating the process of determining the target VLA policy network in a model training method for robots with multiple configurations, as shown in the embodiments of this application. Figure 9 As shown, in step 104 above, latent space visual motion representation is used as an auxiliary supervision signal. Based on the robotic arm operation image sequence, the corresponding robotic arm state sequence, and the corresponding natural language commands, the initial VLA policy network is trained to obtain the target VLA policy network, including:

[0185] Step 901: Using the initial VLA policy network, predict the robotic arm operation image sequence, the corresponding robotic arm state sequence, and the corresponding natural language commands to obtain the predicted latent space features and predicted action codes.

[0186] The predicted action encoding is the underlying execution action of the configured robot. The initial VLA policy network includes: a backbone network. And action prediction heads (Policy Head / Action Head) for network and visual language action models. network.

[0187] Optionally, the sequence of robotic arm operation images, the corresponding sequence of robotic arm states, and the corresponding natural language commands are input into the initial VLA policy network. The predicted latent space features are obtained from the middle layer, i.e. the backbone network, of the initial VLA policy network. The predicted action code is obtained based on the predicted latent space features and the action prediction head of the network and the visual language action model.

[0188] Step 902: Using the latent space visual motion representation as an auxiliary supervision signal, calculate the latent space loss function for predicting latent space features.

[0189] Optionally, based on the latent space visual motion representation and predicted latent space features, the latent space loss function is calculated using the latent space loss function calculation formula.

[0190]

[0191] in, To predict latent space features, This represents the visual motion representation in latent space. This is the latent space loss function.

[0192] Step 903: Calculate the cloning loss based on the predicted action code and the actual action code.

[0193] The actual motion encoding is the motion encoding of the future time step in the robotic arm state sequence input to the VLA model.

[0194] Optionally, based on the predicted action code and actual action coding The cloning loss was calculated. .in, This is a sequence of images showing robotic arm operations. Natural language commands, This is the state sequence of the robotic arm.

[0195] Step 904: Adjust the parameters of the initial VLA policy network according to the latent space loss function and the cloning loss to obtain the target VLA policy network.

[0196] In this embodiment, an initial VLA policy network is used to predict the robotic arm operation image sequence, the corresponding robotic arm state sequence, and the corresponding natural language commands, obtaining predicted latent space features and predicted action codes. Latent space visual motion representation is used as an auxiliary supervision signal to calculate the latent space loss function for the predicted latent space features. Based on the predicted action codes and actual action codes, a cloning loss is calculated. The initial VLA policy network is then tuned based on the latent space loss function and the cloning loss to obtain the target VLA policy network. This application uses latent space visual motion representation as an auxiliary supervision signal to train the VLA policy network, achieving a transition from imitating actions to understanding intentions, providing richer learning signals, and improving training efficiency and performance.

[0197] Based on the same inventive concept, this application also provides a model training device for robots with multiple configurations, corresponding to the model training method for robots with multiple configurations. Since the principle of the device in this application is similar to the model training method for robots with multiple configurations described above, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0198] Figure 10 This application provides a schematic diagram of the structure of a model training device for robots with various configurations, as shown in the embodiments of this application. Figure 10 As shown, the device includes:

[0199] Module 1001 is used to build an initial task dynamic representation model, which includes an initial unified visual action network and an initial visual language action (VLA) policy network.

[0200] The first training module 1002 is used to train the initial unified visual action network based on the robotic arm operation dataset for robots of various configurations to obtain the target unified visual action network; wherein, the robotic arm operation dataset includes: robotic arm operation image sequence, corresponding robotic arm state sequence and corresponding natural language instructions.

[0201] Extraction module 1003 is used to extract features from the robot arm operation image sequence and the corresponding robot arm state sequence using a target unified visual motion network to obtain latent space visual motion representation.

[0202] The second training module 1004 is used to train the initial VLA policy network using latent space visual motion representation as an auxiliary supervision signal, based on the robotic arm operation image sequence, the corresponding robotic arm state sequence, and the corresponding natural language instructions, to obtain the target VLA policy network.

[0203] In one possible implementation, the initial unified visual action network includes: an initial visual encoder, an initial visual decoder, an initial action encoder, and an initial action decoder; the first training module 1002 is specifically used to: use the initial visual encoder to jointly encode the current visual observation and the real visual observation at future moments in the sequence of robot arm operation images to obtain a continuous visual latent vector.

[0204] An initial motion encoder is used to jointly encode the state information of the robotic arm at multiple consecutive moments in the state sequence of the robotic arm to obtain a continuous motion potential vector.

[0205] Quantization operations are performed on continuous visual latent vectors and continuous action latent vectors respectively to obtain visual quantization information and action quantization information;

[0206] An initial visual decoder is used to decode the visual quantization information to obtain the predicted visual observations for future moments;

[0207] An initial motion decoder is used to decode the motion quantization information to obtain the predicted state information of the robotic arm at future moments;

[0208] Based on the predicted visual observations and actual visual observations at future time points, as well as the predicted robotic arm state information and actual robotic arm state information at future time points, the initial visual encoder and initial motion encoder are tuned to obtain the target visual encoder and target motion encoder. The target unified visual motion network includes the target visual encoder and the target motion encoder.

[0209] In one possible implementation, the first training module 1002 is specifically used to: use a preset nearest neighbor search method to quantize the continuous visual latent vectors in a preset unified visual motion code (UVMC) template, and obtain and determine the encoding terms corresponding to the continuous visual latent vectors as visual quantization information.

[0210] Using a preset nearest neighbor search method, the latent vectors of continuous actions are quantized in the UVMC, and the corresponding coding terms of the latent vectors of continuous actions are obtained and determined as action quantization information.

[0211] The UVMC itself contains: multiple preset coding terms, each of which corresponds to a preset visual latent vector and a corresponding action latent vector.

[0212] In one possible implementation, the first training module 1002 is specifically used to: use the target visual encoder in the target unified visual action network to encode the current visual observation in the robotic arm operation image sequence to obtain the target visual feature vector;

[0213] The target action encoder in the target unified visual action network is used to encode the current state information of the robotic arm in the state sequence of the robotic arm to obtain the target action feature vector.

[0214] The target visual feature vector and the target action feature vector are quantized separately to obtain target visual quantization information and target action quantization information as latent space visual motion representations.

[0215] In one possible implementation, the first training module 1002 is specifically used to: calculate a first reconstruction loss value based on the predicted visual observations and the actual visual observations at future times;

[0216] The first quantization loss value is calculated based on the visual quantization information and the continuous visual latent vector;

[0217] The second reconstruction loss value is calculated based on the predicted state information of the robotic arm at future moments and the actual state information of the robotic arm.

[0218] The second quantization loss value is calculated based on the action quantization information and the potential vector of continuous actions;

[0219] Based on the first reconstruction loss value and the first quantization loss value, the initial visual encoder is tuned to obtain the target visual encoder;

[0220] Based on the second reconstruction loss value and the second quantization loss value, the initial motion encoder is tuned to obtain the target motion encoder.

[0221] In one possible implementation, the first training module 1002 is specifically used to: calculate the first codebook loss and the first commitment loss value based on the visual quantization information and the continuous visual latent vector, respectively.

[0222] Calculate the first quantization loss value based on the first codebook loss and the first commitment loss value;

[0223] Based on the action quantization information and the latent vectors of continuous actions, the second quantization loss value is calculated, including:

[0224] Based on the action quantization information and the potential vector of continuous actions, calculate the second codebook loss and the second commitment loss value respectively;

[0225] Calculate the second quantization loss value based on the second codebook loss and the second commitment loss value.

[0226] In one possible implementation, the first training module 1002 is further configured to: calculate the alignment loss function value based on the matching probability of visual quantization information and the matching probability of action quantization information;

[0227] Based on the alignment loss function value, the initial visual encoder and initial motion encoder are tuned.

[0228] In one possible implementation, the second training module 1004 is specifically used to: use an initial VLA policy network to predict the sequence of robotic arm operation images, the corresponding sequence of robotic arm states and the corresponding natural language commands to obtain the predicted latent space features and the predicted action code.

[0229] Using latent space visual motion representation as an auxiliary supervision signal, the latent space loss function for predicting latent space features is calculated.

[0230] The cloning loss is calculated based on the predicted action code and the actual action code.

[0231] The initial VLA policy network is tuned using the latent space loss function and cloning loss to obtain the target VLA policy network.

[0232] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0233] This application also provides an electronic device. Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 11As shown, the electronic device includes a processor 1101 and a memory 1102, and optionally, a bus 1103. The memory 1102 stores machine-readable instructions executable by the processor 1101. When the electronic device is running, the processor 1101 and the memory 1102 communicate via the bus 1103. When the machine-readable instructions are executed by the processor 1101, the steps of the model training method for robots with multiple configurations described above are performed.

[0234] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the above-described model training method for robots with multiple configurations.

[0235] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0236] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.

[0237] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A model training method for a robot oriented to a plurality of configurations, characterized by, The method includes: Construct an initial task dynamic representation model, which includes: an initial unified visual action network and an initial visual language action (VLA) policy network; Based on the robotic arm operation dataset for various robot configurations, the initial unified visual-motor network is trained to obtain the target unified visual-motor network; wherein, the robotic arm operation dataset includes: robotic arm operation image sequence, corresponding robotic arm state sequence, and corresponding natural language instructions; The target unified visual motion network is used to extract features from the robotic arm operation image sequence and the corresponding robotic arm state sequence to obtain the latent space visual motion representation. Using the latent space visual motion representation as an auxiliary supervision signal, the initial VLA policy network is trained based on the robotic arm operation image sequence, the corresponding robotic arm state sequence, and the corresponding natural language instructions to obtain the target VLA policy network. The initial unified visual-motion network includes: an initial visual encoder, an initial visual decoder, an initial motion encoder, and an initial motion decoder; the step of training the initial unified visual-motion network based on a robotic arm operation dataset for various robot configurations to obtain the target unified visual-motion network includes: The initial visual encoder is used to jointly encode the current visual observations and the real visual observations at future moments in the sequence of robot arm operation images to obtain a continuous visual latent vector. The initial motion encoder is used to jointly encode the robotic arm state information at multiple consecutive moments in the robotic arm state sequence to obtain a continuous motion potential vector. The continuous visual latent vector and the continuous action latent vector are quantized respectively to obtain visual quantization information and action quantization information. The initial visual decoder is used to decode the visual quantization information to obtain the predicted visual observations for the future time. The initial motion decoder is used to decode the motion quantization information to obtain the predicted robotic arm state information for the future time. Based on the predicted visual observations and actual visual observations at the future time, as well as the predicted robotic arm state information and actual robotic arm state information at the future time, the initial visual encoder and the initial motion encoder are tuned to obtain the target visual encoder and the target motion encoder. The target unified visual motion network includes the target visual encoder and the target motion encoder.

2. The method of claim 1, wherein, The step of quantizing the continuous visual latent vector and the continuous action latent vector to obtain visual quantization information and action quantization information respectively includes: Using a preset nearest neighbor search method, the continuous visual latent vectors are quantized in a preset unified visual motion code (UVMC) to obtain and determine the encoding terms corresponding to the continuous visual latent vectors as the visual quantization information. Using the preset nearest neighbor search method, the continuous action potential vector is quantized in the UVMC text to obtain and determine the encoding item corresponding to the continuous action potential vector as the action quantization information; The UVMC itself has: multiple preset coding terms, each preset coding term corresponding to a preset visual latent vector and a corresponding action latent vector.

3. The method according to claim 1, characterized in that, The target unified visual motion network is used to extract features from the robotic arm operation image sequence and the corresponding robotic arm state sequence to obtain a latent space visual motion representation, including: The target visual encoder in the target unified visual action network is used to encode the current visual observation in the sequence of robotic arm operation images to obtain the target visual feature vector; The target motion encoder in the target unified visual motion network is used to encode the current state information of the robotic arm in the robotic arm state sequence to obtain the target motion feature vector. The target visual feature vector and the target action feature vector are quantized respectively to obtain target visual quantization information and target action quantization information as the latent space visual motion representation.

4. The method of claim 1, wherein, The step of tuning the initial visual encoder and the initial motion encoder based on the predicted visual observations and actual visual observations at the future time, as well as the predicted robotic arm state information and actual robotic arm state information at the future time, to obtain the target visual encoder and the target motion encoder, includes: Calculate the first reconstruction loss value based on the predicted visual observations and the actual visual observations at the future time. Calculate the first quantization loss value based on the visual quantization information and the continuous visual latent vector; The second reconstruction loss value is calculated based on the predicted robotic arm state information and the actual robotic arm state information at the future time. Calculate the second quantization loss value based on the action quantization information and the continuous action potential vector; Based on the first reconstruction loss value and the first quantization loss value, the initial visual encoder is tuned to obtain the target visual encoder; Based on the second reconstruction loss value and the second quantization loss value, the initial motion encoder is tuned to obtain the target motion encoder.

5. The method of claim 4, wherein, The step of calculating the first quantization loss value based on the visual quantization information and the continuous visual latent vector includes: Based on the visual quantization information and the continuous visual latent vector, calculate the first codebook loss and the first commitment loss value respectively; Calculate the first quantization loss value based on the first codebook loss and the first commitment loss value; The step of calculating the second quantization loss value based on the action quantization information and the continuous action latent vector includes: Based on the action quantization information and the continuous action potential vector, calculate the second codebook loss and the second commitment loss value respectively; The second quantization loss value is calculated based on the second codebook loss and the second commitment loss value.

6. The method of claim 1, wherein, Before the step of tuning the initial visual encoder and the initial motion encoder based on the predicted visual observations and actual visual observations at the future time, as well as the predicted robotic arm state information and the actual robotic arm state information at the future time, to obtain the target visual encoder and the target motion encoder, the method further includes: The alignment loss function value is calculated based on the matching probability of the visual quantization information and the matching probability of the action quantization information; Based on the alignment loss function value, the initial visual encoder and the initial motion encoder are tuned.

7. The method of claim 1, wherein, The process of using the latent space visual motion representation as an auxiliary supervision signal, and training the initial VLA policy network based on the robotic arm operation image sequence, the corresponding robotic arm state sequence, and the corresponding natural language instructions to obtain the target VLA policy network includes: Using the initial VLA policy network, the robotic arm operation image sequence, the corresponding robotic arm state sequence, and the corresponding natural language command are predicted to obtain the predicted latent space features and the predicted action code. Using the latent space visual motion representation as an auxiliary supervision signal, the latent space loss function of the predicted latent space features is calculated; The cloning loss is calculated based on the predicted action code and the actual action code. The initial VLA policy network is tuned based on the latent space loss function and the cloning loss to obtain the target VLA policy network.

8. An electronic device, comprising: include: A processor and a memory, the memory storing machine-readable instructions executable by the processor, which, when the electronic device is running, are executed by the processor to perform the steps of the model training method for multi-configuration robots as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the model training method for multi-configuration robots as described in any one of claims 1 to 7.

Images