A model training method and device, and an electronic device

Abstract

The application discloses a model training method and device and electronic equipment. The method comprises the following steps: acquiring space availability data, embodied visual question answering data and robot demonstration data; constructing a multi-modal training data set based on the space availability data, the embodied visual question answering data and the robot demonstration data; pre-training an initial visual language model using the multi-modal training data set to obtain a target visual language model; adding a motion head to the target visual language model to obtain an initial visual language motion model; screening a robot motion trajectory data set from the robot demonstration data; pre-training the initial visual language motion model using the motion trajectory data set to obtain a target visual language motion model; and the target visual language model in the target visual language motion model is used to obtain a fused semantic representation according to input data, and the motion head in the target visual language motion model is used to output a motion control parameter of a robot according to the fused semantic representation.

Description

Technical Field

[0001] This application relates to the field of robot operation technology, and in particular to a model training method, apparatus and electronic device. Background Technology

[0002] In the field of robotics, end-to-end visual-language action models have become the mainstream technical approach to enable robots to autonomously perform operations based on visual observation and natural language commands. Related technologies typically rely on high-density 3D point cloud data to provide spatial structure information. While this offers high accuracy, it is computationally burdensome, and its dedicated networks are difficult to integrate into general visual-language pre-training frameworks based on large-scale image-text pairs, resulting in low training efficiency and limited scalability. Another method based on RGB images, while more easily scalable, struggles to achieve accurate and generalizable action output in the absence of structured spatial supervision. Therefore, how to efficiently train a general model capable of understanding complex commands and generating precise actions has become a pressing issue. Summary of the Invention

[0003] This application aims to propose a model training method, apparatus, and electronic device that can efficiently train a general model that can understand complex instructions and generate precise actions.

[0004] In a first aspect, embodiments of this application provide a model training method, including: Acquire spatial availability data, embodied visual question-answering data, and robot demonstration data; Based on the spatial availability data, the embodied visual question-answering data, and the robot demonstration data, a multimodal training dataset is constructed; The initial visual language model is pre-trained using the multimodal training dataset to obtain the target visual language model; An action head is added to the target visual language model to obtain an initial visual language action model; The robot motion trajectory dataset was obtained by filtering from the robot demonstration data; The initial visual language action model is pre-trained using the motion trajectory dataset to obtain a target visual language action model; wherein, the target visual language model in the target visual language action model is used to obtain a fused semantic representation based on the input data, and the action head in the target visual language action model is used to output the robot's motion control parameters based on the fused semantic representation.

[0005] According to some embodiments of this application, constructing a multimodal training dataset based on the spatial availability data, the embodied visual question-answering data, and the robot demonstration data includes: Perform data type tagging on the spatial availability data, the embodied visual question answering data, and the robot demonstration data respectively; Encode the tagged spatial availability data, embodied visual question answering data, and robot demonstration data into a sequence of tokens to obtain the training data set.

[0006] According to some embodiments of the present application, the robot demonstration data is encoded into a sequence of tokens through the following steps: Use a visual encoder to convert the image data in the robot demonstration data into visual tokens; Use a tokenizer to convert the language data in the robot demonstration data into text tokens; Based on the FAST method, convert the action data in the robot demonstration data into action tokens; Concatenate the visual tokens, text tokens, and action tokens to obtain the sequence of tokens.

[0007] According to some embodiments of the present application, the pre-training of the initial vision-language model using the multi-modal training data set includes: Input the sequence of tokens into the initial vision-language model; Use the first k' action tokens in the sequence of tokens as prompt conditions to predict the (k'+1)-th to k-th action tokens to train the initial vision-language model, where 0 ≤ k' < k and k is the total number of action tokens; During the training iteration process, for multiple data subsets of the multi-modal training data set, monitor the prediction accuracy of the action tokens in the multiple data subsets, and remove the data subsets with the prediction accuracy lower than the preset accuracy threshold from the subsequent training iterations.

[0008] According to some embodiments of the present application, the screening of the robot action trajectory data set from the robot demonstration data includes: Based on the prediction accuracy of the action tokens by the target vision-language model, screen the robot action trajectory data set from the robot demonstration data.

[0009] According to some embodiments of the present application, the data type tagging of the spatial availability data is performed through the following steps: According to the spatial annotation type of the spatial availability data, add corresponding start and end tags to the spatial availability data; Wherein, the spatial annotation type includes at least one of point-based representation, bounding box-based representation, and trajectory flow-based representation.

[0010] According to some embodiments of the present application, it further includes: Acquire interactive data of the robot performing the target task in the target application scenario; Based on the interaction data, a robot target task dataset is generated; The target visual language action model is fine-tuned using the robot target task dataset.

[0011] According to some embodiments of this application, the pre-training of the initial visual language model using the multimodal training dataset includes: Training samples are obtained from the multimodal training dataset using a hierarchical sampling strategy: The initial visual language model is pre-trained using the training samples.

[0012] Secondly, embodiments of this application provide a model training apparatus, including: The acquisition module is used to acquire spatial availability data, embodied visual question-and-answer data, and robot demonstration data; The construction module is used to construct a multimodal training dataset based on the spatial availability data, the embodied visual question-answering data, and the robot demonstration data; The first pre-training module is used to pre-train the initial visual language model using the multimodal training dataset to obtain the target visual language model. An addition module is used to add an action head to the target visual language model to obtain an initial visual language action model. A filtering module is used to filter out robot motion trajectory datasets from the robot demonstration data; The second pre-training module is used to pre-train the initial visual language action model using the motion trajectory dataset to obtain a target visual language action model; wherein, the target visual language model in the target visual language action model is used to obtain a fused semantic representation based on the input data, and the action head in the target visual language action model is used to output the robot's motion control parameters based on the fused semantic representation.

[0013] Thirdly, embodiments of this application provide an electronic device, including: At least one processor; At least one memory for storing at least one program; The model training method described above is implemented when at least one of the programs is executed by at least one of the processors.

[0014] Fourthly, embodiments of this application provide a computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to implement the model training method described above.

[0015] In this embodiment, a multimodal training dataset is constructed that integrates spatial availability data, embodied visual question-answering data, and robot demonstration data. This dataset is then used to pre-train an initial visual language model, enabling the model to establish deep connections between visual, linguistic, and action semantics within a unified semantic space. This allows the model to gain an understanding of scene structure and task intent. Furthermore, a motion trajectory dataset selected from the robot demonstration data is used to pre-train an initial visual language action model with an expanded motion head. This allows the model to focus on learning the precise mapping from semantic representation to continuous action parameters. Through phased pre-training, large-scale, diverse non-action-annotated data can be effectively used to improve the model's semantic generalization ability, while high-quality motion trajectory data can be used to optimize action generation accuracy. This significantly improves the overall model performance and training efficiency while avoiding the difficulties of training with complex heterogeneous data. The model can then directly output precise action control parameters based on visual and linguistic inputs, achieving accurate operation execution in diverse robot tasks.

[0016] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0017] The present application will be further described below with reference to the accompanying drawings and embodiments, wherein: Figure 1 A flowchart illustrating an embodiment of the model training method provided in this application; Figure 2 A schematic diagram of an embodiment of the electronic device provided in this application.

[0018] Figure label: Electronic device 100, processor 110, memory 120. Detailed Implementation

[0019] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0020] In the description of this application, it should be understood that the orientation descriptions, such as up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0021] In the description of this application, "multiple" refers to two or more. The use of "first" and "second" is for the purpose of distinguishing technical features only and should not be construed as indicating or implying relative importance, or implicitly indicating the number of technical features indicated, or the order in which the technical features are indicated.

[0022] In the description of this application, unless otherwise expressly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0023] The following is based on Figures 1-2 This application describes a model training method, apparatus, and electronic device provided in its embodiments.

[0024] This application provides a model training method, such as... Figure 1 As shown, it includes: Step S100: Acquire spatial availability data, embodied visual question-answering data, and robot demonstration data; Step S200: Construct a multimodal training dataset based on spatial availability data, embodied visual question-answering data, and robot demonstration data; Step S300: Pre-train the initial visual language model using a multimodal training dataset to obtain the target visual language model; Step S400: Add an action head to the target visual language model to obtain the initial visual language action model; Step S500: Filter the robot demonstration data to obtain the robot motion trajectory dataset; Step S600: Pre-train the initial visual language action model using the action trajectory dataset to obtain the target visual language action model; wherein, the target visual language model in the target visual language action model is used to obtain the fused semantic representation based on the input data, and the action head in the target visual language action model is used to output the robot's motion control parameters based on the fused semantic representation.

[0025] In this embodiment, a multimodal training dataset is constructed that integrates spatial availability data, embodied visual question-answering data, and robot demonstration data. This dataset is then used to pre-train an initial visual language model, enabling the model to establish deep connections between visual, linguistic, and action semantics within a unified semantic space. This allows the model to gain an understanding of scene structure and task intent. Furthermore, a motion trajectory dataset selected from the robot demonstration data is used to pre-train an initial visual language action model with an expanded motion head. This allows the model to focus on learning the precise mapping from semantic representation to continuous action parameters. Through phased pre-training, large-scale, diverse non-action-annotated data can be effectively used to improve the model's semantic generalization ability, while high-quality motion trajectory data can be used to optimize action generation accuracy. This significantly improves the overall model performance and training efficiency while avoiding the difficulties of training with complex heterogeneous data. The model can then directly output precise action control parameters based on visual and linguistic inputs, achieving accurate operation execution in diverse robot tasks.

[0026] In step S100 above, the spatial availability data includes spatial information about the operable areas within an object or scene. Embodied visual question-and-answer data includes visual scenes related to physical interaction and corresponding questions and answers. Robot demonstration data includes visual observations, states, and action sequences of the robot during task execution.

[0027] In step S200 above, spatial availability data, embodied visual question-answering data, and robot demonstration data are integrated to obtain a multimodal training dataset. For example, spatial availability data, embodied visual question-answering data entries, and robot demonstration data fragments (which may include video clips, image frames, text commands, and action sequences) from different sources are collected and processed into multiple training samples that can be used as model input. Each sample may contain one or more types of data.

[0028] In step S300 above, the initial visual language model (e.g., a model based on the Transformer architecture) is trained using a multimodal training dataset. During training, information such as images and text from a sample is input into the model, and optimization is performed with the goal of predicting the next lexical unit through autoregression. The goal of this stage is to enable the model to learn to associate visual content, language description, and task semantics. For example, the model needs to learn to predict the semantic representation related to the "grab" action when it sees the instruction "pick up the cup" and related images. Iterative training continues until the model's loss function converges, resulting in the target visual language model.

[0029] In step S400 above, a new network module is connected to the target visual language model as an action head, thereby forming an initial visual language action model. The function of this action head is to map the abstract semantic representation output by the visual language model to the specific control parameters of the robot actuator (such as joint angles, end effector pose, etc.).

[0030] In step S500 above, a portion of the acquired robot demonstration data is filtered out according to certain criteria to obtain a robot motion trajectory dataset for subsequent motion head training. For example, filtering can be based on intuitive criteria such as the reliability of the data source, the success rate of task completion, or the completeness of data records. The purpose of filtering is to obtain a higher-quality robot motion trajectory dataset that is more suitable for learning motion mapping.

[0031] In step S600 above, the initial visual language action model is pre-trained using the robot motion trajectory dataset. This pre-training process uses the visual observations and language commands contained in the robot motion trajectory dataset as model inputs, and their corresponding robot actions as supervision targets. By optimizing the model parameters (including the visual language model part and the action head part), the entire model learns the overall mapping from multimodal perceptual inputs to motion control parameters. This process aims to improve the accuracy and generalization of the model's output actions. Training continues until the model performance converges, resulting in the target visual language action model.

[0032] In some embodiments of this application, the step S200 of "constructing a multimodal training dataset based on spatial availability data, embodied visual question-answering data, and robot demonstration data" is further described. Step S200 includes: Step S210: Label the spatial availability data, embodied visual question-and-answer data, and robot demonstration data with data types respectively; Step S220: Encode the labeled spatial availability data, embodied visual question answering data, and robot demonstration data into word sequences to obtain the training dataset.

[0033] In this implementation, by applying data type labels to spatial availability data, embodied visual question-answering data, and robot demonstration data, and then uniformly encoding them into a lexical sequence, the model can clearly identify and distinguish the different semantic information contained in the input (such as spatial location, task description, and action demonstration), thereby more effectively learning the inherent relationships between these heterogeneous data during training. By transforming diverse raw data into a unified lexical sequence representation, not only is the complexity of model processing and fusing multimodal information reduced, but a direct and efficient input foundation is also provided for subsequent deep pre-training based on sequence models, significantly improving the robustness and effectiveness of the overall training process from a data perspective.

[0034] In step S210 above, spatial availability data is identified using different dedicated tags based on its specific annotation format. For example, for "point-based annotation" data, tags are added to the beginning and end of its text description. <points>Start tag and< / points> End marker; for "bounding box based annotation" data, add... <box>With< / box> Labels; for "track flow-based annotation" data, add... <flow>With< / flow> Tagging. For embodied visual question-and-answer data, tags such as [tags] can be added to the beginning and end of the question and answer text. <vqa>With< / vqa> The tag indicates that it belongs to a question-and-answer task. For robot demonstration data, a tag such as [example tag] can be added to the text describing the task instructions or scenario. <demo>With< / demo> The label indicates that it originated from a robot operation demonstration.

[0035] In step S220 above, all text data marked in step S210 is input into a pre-trained word segmenter for encoding. This word segmenter (e.g., based on the BPE algorithm) segments the continuous text string and maps it into a string of discrete token IDs. For example, the sentence " <points>Picking up a cup< / points> "It may be encoded as a sequence of token IDs in the form [101, 305, 102], where 101 corresponds to..." <points>, 305 corresponds to the "picking up a cup" semantics, 102 corresponds to< / points> Ultimately, all the labeled and encoded data entries together constitute the multimodal training dataset, with each sample represented in a uniform word sequence format for subsequent pre-training.

[0036] In some embodiments of this application, robot demonstration data is encoded into a word sequence through the following steps: Step S221: Using a visual encoder, convert the image data in the robot demonstration data into visual words; Step S222: Use a word segmenter to convert the language data in the robot demonstration data into text words; Step S223: Based on the FAST method, convert the motion data in the robot demonstration data into action lexical units; Step S224: Concatenate visual lexical units, textual lexical units, and action lexical units to obtain a lexical sequence.

[0037] In this embodiment, by converting the image data, language data, and action data in the robot demonstration data into visual lexical units, text lexical units, and action lexical units respectively, and jointly encoding them under a unified sequence framework, a tightly aligned multimodal semantic representation space is established for the model. This enables the model to synchronously learn the intrinsic relationship between visual features, language instructions, and action sequences, thereby significantly enhancing the model's generalization ability to perform joint reasoning and action prediction in multimodal contexts in complex robot tasks.

[0038] In step S221 above, the image data contained in the robot demonstration data is processed using a pre-trained visual encoder (e.g., a model based on the VisionTransformer architecture). This visual encoder segments the input image into several image patches and converts them into a series of visual feature vectors, which serve as visual lexical units.

[0039] In step S222 above, the language data (such as task instructions or scene description text) contained in the robot demonstration data is processed using a pre-trained word segmenter (e.g., a byte-pair-based word segmenter). This word segmenter divides the text string into sub-word units and maps each sub-word to a discrete identifier, thereby generating text lexical units.

[0040] In step S223 above, the motion data contained in the robot demonstration data, such as continuous motion sequences (e.g., time series of joint angles or end effector poses), is processed based on the FAST method. Specifically, the continuous motion sequence is first subjected to discrete cosine transform to obtain its transform coefficient sequence in the frequency domain; then, the coefficient sequence is encoded byte-to-byte, discretized, and mapped into a string of discrete identifiers to generate action lexics.

[0041] In step S224 above, visual lexical units, textual lexical units, and action lexical units are spliced ​​and combined according to their temporal correspondence or a specific order to obtain a single and coherent multimodal lexical sequence.

[0042] In some embodiments of this application, the step S300 of "pre-training the initial visual language model using a multimodal training dataset" is further described. Step S300 includes: Step S310: Obtain training samples from the multimodal training dataset using a hierarchical sampling strategy: Step S320: Pre-train the initial visual language model using training samples.

[0043] In this embodiment, by assigning sampling weights to samples in the multimodal training dataset, the problem of unbalanced training sample distribution caused by heterogeneous data sources and scale differences is effectively overcome. This ensures that the model can learn data from different semantic categories and their internal subcategories, such as spatial availability, embodied visual question answering, and robot demonstration, in a controllable proportion during pre-training. This avoids the model from being overly biased towards a certain type of data, thereby weakening the learning of information from other important modalities or tasks.

[0044] In step S310 above, first, define the major task categories and sub-categories involved in sampling. The major task categories at least include: spatial affordance data, embodied visual question-answering data, and robot demonstration data. Under each major task category, according to the specific source or task characteristics of the data, multiple sub-categories are further divided. For example, under the major category of "robot demonstration data", multiple different open-source data sets such as "Agibot", "Droid", "Jacoplay", etc. can be included as sub-categories. Subsequently, calculate and allocate two-level weights. At the first level, calculate an initial sampling weight for each major task category. This weight can be set based on a preset balance ratio or the relative data scale of each major category. For example, weights of 0.2, 0.3, and 0.5 are assigned to the three types of data respectively to ensure that the model learns evenly from different semantic categories. At the second level, within each major task category, calculate an internal sampling weight for each of its sub-categories. This weight can be adjusted according to the data quality, scale, or diversity of the sub-categories to balance the information coverage within each category. In each round of training iteration, first randomly select a major task category according to the first-level weight, then randomly select a specific data source from its sub-categories according to the second-level weight under this major category, and finally extract one or more specific training samples from the selected data source.

[0045] In some embodiments of the present application, the "pre-training the initial vision-language model using the multi-modal training data set" in step S300 is further described. Step S300 includes: Step S321: Input the token sequence into the initial vision-language model; Step S322: Use the first k' action tokens in the token sequence as prompt conditions to predict the (k'+1)-th to k-th action tokens to train the initial vision-language model, where 0 ≤ k' < k, and k is the total number of action tokens; Step S323: During the training iteration process, for multiple data subsets of the multi-modal training data set, monitor the prediction accuracy of the action tokens in the multiple data subsets, and remove the data subsets with prediction accuracy lower than the preset accuracy threshold from the subsequent training iterations.

[0046] In this embodiment, by setting the training task of "using the first k' action tokens in the token sequence as prompt conditions to predict the (k'+1)-th to k-th action tokens", the model can directly learn the semantic mapping from perception to action intention during pre-training. At the same time, dynamically screen the training data according to the real-time performance of the model on this training task, and automatically exclude the inefficient data subsets in the multi-modal training data set, thereby significantly improving the efficiency of action semantic learning and the model generalization ability.

[0047] In step S321 above, the lexical sequence is input into the initial visual language model for training. This lexical sequence is a multimodal sequence composed of visual lexical units, text lexical units, and action lexical units concatenated in sequence.

[0048] In step S322 above, a dedicated action semantic learning task is performed. The first k' action words are used as known cues, which, together with all visual and text words in the word sequence, constitute the model's input context. The training objective of the model is to predict the subsequent k'+1 to kth words in an autoregressive manner. Through this task, the model learns multimodal associations and establishes causal reasoning relationships between current environmental observations, task instructions, and future executable action sequences.

[0049] In step S323 above, an adaptive data filtering mechanism is implemented during the training iteration process. First, the multimodal training dataset is divided into multiple data subsets according to its source or task category (e.g., divided into different robot demonstration dataset sources). In each training iteration cycle or every few iteration cycles, the average prediction accuracy of the initial visual language model for action words in the training samples contained in each data subset is monitored and statistically analyzed. Subsequently, the prediction accuracy of each data subset is compared with a preset accuracy threshold. Data subsets with prediction accuracy lower than the preset accuracy threshold are temporarily removed or permanently excluded from the sampling pool of subsequent training iterations to ensure that the training process continuously focuses on high-quality, highly learnable data.

[0050] In some embodiments of this application, the step S500 of "filtering out the robot motion trajectory dataset from the robot demonstration data" is further described. Step S500 includes: Step S510: Based on the prediction accuracy of action lexical units by the target visual language model, obtain the robot motion trajectory dataset from the robot demonstration data.

[0051] In this embodiment, the robot motion trajectory dataset is filtered based on the prediction accuracy of action lexical units by the trained target visual language model. This automatically identifies and filters high-quality interaction segments in the robot demonstration data that match the model's current semantic understanding ability and have a moderate learning difficulty. By using the model's own prediction performance as a filtering criterion, low-quality data containing noise, ambiguity, or inconsistency with the model's representation space is effectively filtered out. This ensures that the robot motion trajectory dataset used for subsequent motion head pre-training has a higher learning signal-to-noise ratio and intrinsic consistency, significantly improving the training efficiency and stability of the motion learning stage.

[0052] In step S510 above, the screening process is implemented as follows: Using the target visual language model, the action lexical units of the encoded samples in the robot demonstration data are predicted, and their prediction accuracy is calculated. Based on a preset accuracy threshold, the robot demonstration data is screened, retaining only samples whose prediction accuracy reaches or exceeds the threshold. The retained samples are used to construct a robot motion trajectory dataset for subsequent model pre-training.

[0053] In some embodiments of this application, spatial availability data is labeled with data type using the following steps: Step S211: Add corresponding start and end markers to the spatial availability data according to the spatial annotation type of the spatial availability data; Spatial annotation types include at least one of point-based representation, bounding box-based representation, and trajectory flow-based representation.

[0054] In this embodiment, by adding corresponding dedicated start and end markers to spatial availability data with different geometric representations (e.g., points, bounding boxes, trajectory flows), various spatial annotations with significant differences in form are uniformly encapsulated into serialized data with clear type identifiers. This enables the model to clearly identify and distinguish different types of spatial prior information (e.g., discrete operation points, regional operation ranges, or continuous motion trajectories). This not only allows the model to more accurately understand and utilize the geometric and operational semantics contained in various spatial availability data, but also significantly improves the regularity and modelability of spatial components in multimodal training data.

[0055] In step S211 above, the data type labeling of spatial availability data is achieved through the following steps: First, the labeling type of each piece of spatial availability data is identified. Based on its geometric representation, it is mainly divided into three categories: point-based representation, bounding box-based representation, and trajectory flow-based representation. Then, according to the identified type, corresponding dedicated label pairs are added to the data for encapsulation. For example, for data describing a single operation point in coordinate form, start tags are added to the beginning and end of its text description. <points>End tag< / points> For data describing a rectangular operation area using the coordinates of two diagonal points, add... <box>With< / box> Mark; for data describing a motion path as a series of continuous coordinate points, add a marker. <flow>With< / flow> Labeling. After this processing, all spatial availability data is converted into a structured and clearly defined format, enabling the model to clearly identify and distinguish the different spatial operation semantics contained in the input.

[0056] In some embodiments of this application, it also includes: Step S700: Acquire the interaction data of the robot performing the target task collected in the target application scenario; Step S800: Generate a robot target task dataset based on the interaction data; Step S900: Fine-tune the target visual language action model using the robot target task dataset.

[0057] In this embodiment, by acquiring robot interaction data collected in the target application scenario and fine-tuning the pre-trained target visual language action model, precise adaptation of the general model to specific tasks and robot platforms is achieved. This allows the model, pre-trained on large-scale general data, to quickly optimize its parameters using relatively small-scale but highly relevant scenario-based data to adapt to the specific dynamic characteristics, sensor configuration, and environmental constraints of the current robot and task. This targeted fine-tuning not only significantly improves the model's action generation accuracy, task success rate, and operational robustness in the target application scenario, but also overcomes the performance loss problem that general models may suffer when facing new hardware or subtle task changes, thereby enabling reliable deployment and efficient application of the model in actual robot systems.

[0058] In step S700 above, firstly, interaction data of the robot performing the task is collected between the target application scenario (e.g., a specific desktop operation task) and the designated robot platform (e.g., the Franka robotic arm). During the collection, image sequences captured by the robot's head and wrist cameras, corresponding natural language task commands, and the robot's state (e.g., joint angles) and motion data are recorded simultaneously.

[0059] In step S800 above, the collected interaction data is uniformly converted into a predefined structured format (e.g., Lerobot format), and the continuous action and state values ​​are normalized to eliminate dimensional differences and improve training stability. Finally, a robot target task dataset is constructed based on the processed data. Each sample in this dataset contains a scene image, task instructions, and a standardized sequence of robot actions, forming a data pair for supervised fine-tuning.

[0060] In step S900 above, the scene images and task instructions in the robot target task dataset are used as model input, and the corresponding standardized robot action sequences are used as training targets. The model parameters are optimized through backpropagation. For fine-tuning, a specialized action head structure can be used, such as an action head based on the principle of flow matching, to improve the quality and smoothness of continuous action generation. During training, the model's prediction loss on the validation set is monitored in real time. When the loss value decreases and stabilizes near a preset threshold (e.g., 0.01), training is terminated, resulting in a visual-language action model adapted to the specific task and robot. Through this process, the model's general capabilities are finely adjusted for the target scene, significantly improving its execution accuracy and robustness in that scene.

[0061] In some embodiments of this application, the action head includes a multilayer sensor.

[0062] In this embodiment, the multilayer perceptron takes fused semantic representation as input and, through its hierarchical fully connected layers and nonlinear activation functions, can progressively fit and output accurate and smooth continuous action control parameters. The multilayer perceptron has advantages such as high parameter efficiency, fast forward inference speed, and ease of combination with different loss functions (such as L1 regression loss) for stable optimization, thus providing the model with reliable and low-latency action generation capabilities.

[0063] A multilayer perceptron (MLP) can be a four-layer MLP. Each fully connected layer can be followed by a nonlinear activation function (such as ReLU). The dimension of the final linear output layer matches the dimension of the robot's motion control parameters (e.g., for a 7-DOF robot arm, the output dimension is 7, corresponding to the target angles or angular velocities of each joint). The MLP decodes the input abstract semantic vector into specific robot motion parameters through a combination of linear transformations and nonlinear activations. During training, an L1 regression loss function (mean absolute error) is used to calculate the difference between the predicted and actual motion parameters. By minimizing this L1 regression loss, the parameters of the MLP are optimized, driving the model to learn and generate accurate and robust motion outputs.

[0064] In addition, embodiments of this application provide a model training apparatus, including: The acquisition module is used to acquire spatial availability data, embodied visual question-and-answer data, and robot demonstration data. The building module is used to construct a multimodal training dataset based on spatial availability data, embodied visual question answering data, and robot demonstration data. The first pre-training module is used to pre-train the initial visual language model using a multimodal training dataset to obtain the target visual language model. Add a module, which is used to add an action head to the target visual language model to obtain an initial visual language action model; The filtering module is used to filter the robot motion trajectory dataset from the robot demonstration data; The second pre-training module is used to pre-train the initial visual language action model using the action trajectory dataset to obtain the target visual language action model. The target visual language model in the target visual language action model is used to obtain the fused semantic representation based on the input data, and the action head in the target visual language action model is used to output the robot's motion control parameters based on the fused semantic representation.

[0065] The model training apparatus provided in this application embodiment can implement the various processes implemented in the above method embodiments and achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0066] In addition, one embodiment of this application also discloses an electronic device 100, such as... Figure 2 As shown, it includes: At least one processor 110; At least one memory 120 is used to store at least one program; The model training method described above is implemented when at least one program is executed by at least one processor 110.

[0067] The electronic device 100 provided in this application embodiment can implement the various processes implemented in the above method embodiments and achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0068] In addition, embodiments of this application provide a computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to implement the model training method described above.

[0069] The computer-readable storage medium provided in this application embodiment can implement the various processes implemented in the above method embodiments and achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0070] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0071] The embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of this application.

Claims

1. A model training method, characterized in that, Comprising: Obtaining spatial availability data, embodied visual question-answering data, and robot demonstration data; Constructing a multi-modal training dataset based on the spatial availability data, the embodied visual question-answering data, and the robot demonstration data; Pre-training an initial vision-language model using the multi-modal training dataset to obtain a target vision-language model; Adding an action head to the target vision-language model to obtain an initial vision-language action model; Filtering from the robot demonstration data to obtain a robot action trajectory dataset; Pre-training the initial vision-language action model using the action trajectory dataset to obtain a target vision-language action model; wherein, the target vision-language model in the target vision-language action model is used to obtain a fused semantic representation according to input data, and the action head in the target vision-language action model is used to output action control parameters of the robot according to the fused semantic representation.

2. The model training method according to claim 1, characterized in that, The constructing a multi-modal training dataset based on the spatial availability data, the embodied visual question-answering data, and the robot demonstration data includes: Respectively performing data type marking on the spatial availability data, the embodied visual question-answering data, and the robot demonstration data; Encoding the marked spatial availability data, the embodied visual question-answering data, and the robot demonstration data into token sequences to obtain the training dataset.

3. The model training method according to claim 2, characterized in that, The robot demonstration data is encoded into a token sequence through the following steps: Using a visual encoder to convert the image data in the robot demonstration data into visual tokens; Using a tokenizer to convert the language data in the robot demonstration data into text tokens; Based on the FAST method, converting the action data in the robot demonstration data into action tokens; Concatenating the visual tokens, text tokens, and action tokens to obtain the token sequence.

4. The model training method according to claim 3, characterized in that, The pre-training the initial vision-language model using the multi-modal training dataset includes: Inputting the token sequence into the initial vision-language model; Using the first k' action tokens in the token sequence as prompt conditions to predict the (k'+1)-th to k-th action tokens to train the initial vision-language model, where 0 ≤ k' < k and k is the total number of action tokens; During the training iteration process, for multiple data subsets of the multi-modal training dataset, monitoring the prediction accuracy of the action tokens in the multiple data subsets, and removing the data subsets with the prediction accuracy lower than a preset accuracy threshold from subsequent training iterations.

5. The model training method according to claim 4, characterized in that, The filtering from the robot demonstration data to obtain a robot action trajectory dataset includes: Filtering from the robot demonstration data to obtain the robot action trajectory dataset based on the prediction accuracy of the action tokens by the target vision-language model.

6. The model training method according to claim 2, characterized in that, The data type marking of the spatial availability data is performed through the following steps: According to the spatial annotation type of the spatial availability data, adding corresponding start markers and end markers to the spatial availability data; The spatial annotation types include at least one of point-based representation, bounding box-based representation, and trajectory flow-based representation.

7. The model training method according to claim 1, characterized in that, Also includes: Acquire interactive data of the robot performing the target task in the target application scenario; Based on the interaction data, a robot target task dataset is generated; The target visual language action model is fine-tuned using the robot target task dataset.

8. The model training method according to claim 1, characterized in that, The pre-training of the initial visual language model using the multimodal training dataset includes: Training samples are obtained from the multimodal training dataset using a hierarchical sampling strategy: The initial visual language model is pre-trained using the training samples.

9. A model training device, characterized in that, include: The acquisition module is used to acquire spatial availability data, embodied visual question-and-answer data, and robot demonstration data; The construction module is used to construct a multimodal training dataset based on the spatial availability data, the embodied visual question-answering data, and the robot demonstration data; The first pre-training module is used to pre-train the initial visual language model using the multimodal training dataset to obtain the target visual language model. An addition module is used to add an action head to the target visual language model to obtain an initial visual language action model. A filtering module is used to filter out robot motion trajectory datasets from the robot demonstration data; The second pre-training module is used to pre-train the initial visual language action model using the motion trajectory dataset to obtain a target visual language action model; wherein, the target visual language model in the target visual language action model is used to obtain a fused semantic representation based on the input data, and the action head in the target visual language action model is used to output the robot's motion control parameters based on the fused semantic representation.

10. An electronic device, characterized in that, include: At least one processor; At least one memory for storing at least one program; The model training method as described in any one of claims 1 to 8 is implemented when at least one of the programs is executed by at least one of the processors.

Images