Model training method and related device

Abstract

The embodiment of the invention discloses a model training method and a related device, and the method comprises the steps: obtaining the motion information of a robot executing a target motion, and the motion information comprises a motion sequence of the target motion; generating an overall text description of the target action according to the action information; splitting the action sequence into a plurality of sub-actions, and generating sub-action text descriptions of the plurality of sub-actions; determining training information of each moment in the action sequence; and selecting the overall text description or the sub-action text description according to a preset probability, combining the training information as model input, and training a target model. Therefore, the data utilization rate of model imitation learning is improved, and the generalization ability of the model to the text information is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a model training method and related apparatus. Background Technology

[0002] With the rapid development of artificial intelligence and robotics, the automated learning and execution of robot operational skills has become a key research focus. Currently, data acquisition for robot imitation learning largely utilizes overall action descriptions as textual annotations. During training, these textual annotations are input into the model as part of the robot's observational information for encoding, enabling the model to execute corresponding actions upon receiving instructions. Ultimately, this allows the model to accept and follow human commands during reasoning.

[0003] Currently, when using overall action descriptions as text annotations for data, the text information obtained by the model is very limited due to the efficiency and cost of data collection, which greatly restricts the model's ability to generalize text semantic information. Summary of the Invention

[0004] In view of this, embodiments of this application provide a model training method and related apparatus, which aims to improve the model's ability to generalize text information by dividing the action into multiple sub-actions while using the overall action description, and selecting the overall text or sub-action text to be input into the model for training with a preset probability. This allows the model to obtain more text description data during the training process, improves the data utilization rate of imitation learning, and enhances the model's ability to generalize text information.

[0005] In a first aspect, embodiments of this application provide a model training method, including: Obtain motion information of the robot performing a target action, wherein the motion information includes the motion sequence of the target action; Based on the action information, generate a complete text description of the target action; The action sequence is split into multiple sub-actions, and text descriptions of the sub-actions are generated. Determine the training information at each moment in the action sequence; The overall text description or the sub-action text description is selected with a preset probability and combined with the training information as model input to train the target model.

[0006] In one possible embodiment, the robot is equipped with an image sensor; determining the training information at each moment in the action sequence includes: determining the robot's body state information at the current moment based on the action sequence, the body state being used to characterize the robot's joint bending angle, the training information including the body state information and sensor observation information, the sensor observation information being used to characterize the information collected by the image sensor; determining the robot's sensor observation information at the current moment based on the action sequence, the observation information including: color data, spatial distance data, and three-dimensional point cloud data.

[0007] In one possible embodiment, selecting the overall text description or the sub-action text description with a preset probability includes: determining a first preset probability; selecting the overall text description with the first preset probability and selecting the sub-action text description with a second preset probability, wherein the sum of the first preset probability and the second preset probability is 1.

[0008] In one possible embodiment, the action information refers to video image information of the robot performing the target action; the step of splitting the action sequence into multiple sub-actions includes: splitting the action sequence according to the frame range or timestamp in the action information to obtain the multiple sub-actions; wherein, the frame range refers to the continuous image frame interval corresponding to a certain sub-action in the action information, and the timestamp refers to the continuous time interval corresponding to a certain sub-action in the action information.

[0009] In one possible embodiment, generating the sub-action text descriptions of the plurality of sub-actions includes: inputting the overall text description of the target action into a visual language model to output an action flow containing the plurality of sub-actions; and generating corresponding sub-action text descriptions based on the definitions of each sub-action in the action flow.

[0010] In one possible embodiment, after training the target model, the method further includes: determining the training time of the target model; obtaining the output result of the target model, the output result referring to the action sequence predicted by the target model within a preset time period after the training time; comparing the output result with a real action sequence, the real action sequence referring to the sequence of the target action within the preset time period after the training time; calculating and determining the loss value of the output result and the real action sequence using a loss function; and adjusting the target model using a gradient descent algorithm to reduce the loss value.

[0011] In one possible embodiment, the robot includes: a single-arm robot, a dual-arm robot, a humanoid robot, or a dexterous hand, wherein the dexterous hand is a humanoid bionic end effector.

[0012] Secondly, embodiments of this application provide a model training apparatus, comprising: an acquisition unit, a first generation unit, a second generation unit, a determination unit, and a training unit; wherein, the acquisition unit is used to acquire action information of a robot performing a target action, the action information including an action sequence of the target action; the first generation unit is used to generate an overall text description of the target action based on the action information; the second generation unit is used to split the action sequence into multiple sub-actions and generate sub-action text descriptions of the multiple sub-actions; the determination unit is used to determine training information at each moment in the action sequence; the training unit is used to select the overall text description or the sub-action text description with a preset probability, and combine it with the training information as model input to train a target model.

[0013] Thirdly, embodiments of this application provide an electronic device, including a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the processor, and the programs include instructions for performing the steps in the first aspect of embodiments of this application.

[0014] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program for electronic data interchange, wherein the computer program causes a computer to perform some or all of the steps described in the first aspect of embodiments of this application.

[0015] Fifthly, embodiments of this application provide a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps described in the first aspect of embodiments of this application. The computer program product may be a software installation package.

[0016] As can be seen, the model training method and related apparatus provided in this embodiment first acquire the action information of the robot performing the target action, including the action sequence of the target action; secondly, based on the action information, a complete text description of the target action is generated; next, the action sequence is broken down into multiple sub-actions, generating sub-action text descriptions for each sub-action; then, the training information at each moment in the action sequence is determined; finally, the complete text description or sub-action text description is selected with a preset probability and combined with the training information as model input to train the target model. Thus, by dividing the complete target action into multiple sub-actions for description, and then selecting the complete text or sub-action text to input into the model for training with a preset probability, the data utilization rate of the model's imitation learning is improved, and the model's generalization ability to text information is enhanced. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the architecture of a robot system provided in an embodiment of this application; Figure 2 This is a schematic flowchart of a model training method provided in an embodiment of this application; Figure 3 This is a schematic diagram of the architecture of a model provided in an embodiment of this application; Figure 4 This is an application diagram of an action scenario provided in an embodiment of this application; Figure 5 This is a schematic diagram of a process for outputting text descriptions of sub-actions provided in an embodiment of this application; Figure 6 This is a block diagram of the functional units of a model training device provided in an embodiment of this application; Figure 7 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0020] The terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0021] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article indicates that the preceding and following related objects have an "or" relationship.

[0022] In this application's embodiments, "multiple" refers to two or more. In this application's embodiments, "connection" refers to various connection methods, such as direct or indirect connections, to achieve communication between devices; this application's embodiments do not impose any limitations on this.

[0023] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0024] The following describes the relevant content, concepts, meanings, technical issues, technical solutions, and beneficial effects involved in the embodiments of this application.

[0025] VLM (Visual Language Model) is a multimodal large model that integrates computer vision and natural language processing capabilities. Its core function is to automatically generate action flows, replacing the manual sub-task of breaking down target actions.

[0026] With the rapid development of artificial intelligence and robotics, the automated learning and execution of robot operational skills has become a key research focus. Currently, most robot imitation learning data acquisition uses holistic action descriptions as text annotations. During training, these text annotations are input into the model as part of the robot's observation information for encoding, enabling the model to execute corresponding actions upon receiving instructions. Ultimately, this allows the model to accept and follow human instructions during reasoning. However, currently, using holistic action descriptions as text annotations limits the efficiency and cost of data acquisition, resulting in very limited textual information obtained by the model, significantly restricting its ability to generalize from semantic information.

[0027] To address the aforementioned issues, this application provides a model training method and related apparatus. By using a holistic action description while dividing the action into multiple sub-actions for description, and selecting either the holistic text or the sub-action text with a preset probability to input into the model for training, the model can obtain more text description data during training, thereby improving the data utilization rate of imitation learning and enhancing the model's generalization ability for text information.

[0028] First, the method in the embodiments of this application is applied to a server, combined with Figure 1 The model training method in the embodiments of this application will be described. Figure 1 This is a schematic diagram of the architecture of a robot system provided in an embodiment of this application, such as... Figure 1 As shown, the robot system 100 includes a server 110 and a robot body 120. The server 110 can communicate with the robot body 120 or be located inside the robot body 120.

[0029] First, server 110 acquires motion information of the robot performing the target action, including the motion sequence of the target action. Then, server 110 generates a complete text description of the target action based on the motion information. Next, server 110 breaks down the motion sequence into multiple sub-actions, generating text descriptions for each sub-action. Then, server 110 determines the training information for each moment in the motion sequence. Finally, server 110 selects either the complete text description or the sub-action text descriptions with a preset probability, and combines them with the training information as model input to train the target model. Ultimately, this achieves the goal of controlling the robot body 120 to perform the target action using the target model.

[0030] Specifically, the robot body 120 can include: a single-arm robot, a dual-arm robot, a humanoid robot, or a dexterous hand, where the dexterous hand is a humanoid bionic end effector. A single-arm robot refers to a robot with only one multi-joint robotic arm, typically paired with an end effector, achieving spatial movement through rotational or translational joints; a dual-arm robot refers to a robot with two independent robotic arms; a humanoid robot refers to a robot designed to mimic the structure of the human body, including a torso, head, and limbs, possessing multi-joint coordinated movement capabilities, and adaptable to unstructured environments; a dexterous hand is a high-degree-of-freedom end effector designed to mimic the structure of the human hand, typically containing five independently movable fingers, partially integrating tactile sensors, enabling precise grasping and manipulation, and can serve as the end component of the first three types of robots or as a standalone execution unit.

[0031] The following is combined Figure 2 This application describes a model training method in one of its embodiments. Figure 2 This is a flowchart illustrating a model training method provided in an embodiment of this application. The method is applied to, for example... Figure 1The server 110 shown includes the following steps: S210, Obtain motion information of the robot performing the target action.

[0032] The action information includes the action sequence of the target action.

[0033] Specifically, motion information is the complete data collection during the robot's execution of a target action. It serves as the fundamental data source for generating text descriptions, breaking down sub-actions, and training models. Motion information includes, but is not limited to, video information, robot motion data such as changes in the joint angles of the robotic arm, the torque of the finger joints of the dexterous hand, the displacement data of the mobile chassis, and timestamp data. For example, a motion sequence can be divided by frame: the target action of putting a pen into a storage box contains 450 frames, corresponding to consecutive frame images from the initial state, opening the lid, picking up the pen, putting it into the box, and closing the lid; or by timestamp: the continuous time interval of opening the lid (0-2s), picking up the pen (2-4s), and putting it into the box (4-6s).

[0034] S220: Generate a complete text description of the target action based on the action information.

[0035] The overall text description is a complete semantic summary of the target action, specifically including the action object, action target, and scene information, such as putting a red marker into the storage box in the left drawer, or using dexterous hands to grab a mineral water bottle and unscrew the cap.

[0036] S230, split the action sequence into multiple sub-actions and generate text descriptions of the sub-actions.

[0037] Sub-actions are components of the target action and have independent execution objectives. Specifically, they can be broken down based on video frame ranges. For example, putting a pen into a storage box can be broken down into frames 1-150: opening the storage box lid, frames 150-300 picking up the pen, and frames 300-450 placing the pen into the box. The sub-action text description associates the execution logic of the sub-action with the text instructions. For example, the sub-action of opening the storage box lid mentioned above is described as: "Use the robotic arm gripper to grasp the right edge of the storage box and lift the lid up to 90°." The sub-action of picking up the pen mentioned above is described as: "The robotic arm moves above the pen, the gripper opens to a diameter of 1cm, lowers and closes the gripper to pick up the middle of the pen."

[0038] S240, determine the training information at each moment in the action sequence.

[0039] The training information may include environmental and robot state data related to the execution of the action sequence at each moment, such as the robot's body state and environmental observation information. The robot's body state includes things like the joint angles of the robotic arm and the opening and closing degree of the dexterous hand gripper; environmental observation information includes things like color data, spatial distance data, and 3D point cloud data.

[0040] S250 selects the overall text description or sub-action text description with a preset probability, and combines it with training information as model input to train the target model.

[0041] Among them, the learning weights of the model for the overall task and local steps are used to balance the learning weights of the model, and the sum of the preset probability of selecting the overall text description and the preset probability of selecting the sub-action text description is 1.

[0042] Specifically, the model structure includes: a fusion text encoding layer for processing text descriptions, a visual encoding layer for processing observation information, and a state encoding layer for processing ontology states; the subsequent model output is: the action sequence from time t+1 to t+10.

[0043] As can be seen, the model training method provided in this application first obtains the action information of the robot performing the target action, including the action sequence of the target action; secondly, based on the action information, a complete text description of the target action is generated; next, the action sequence is broken down into multiple sub-actions, generating sub-action text descriptions for each sub-action; then, the training information at each moment in the action sequence is determined; finally, either the complete text description or the sub-action text description is selected with a preset probability and combined with the training information as model input to train the target model. Thus, by dividing the complete target action into multiple sub-actions for description, and then selecting either the complete text or the sub-action text to be input into the model for training with a preset probability, the data utilization rate of the model's imitation learning is improved, and the model's generalization ability to text information is enhanced.

[0044] In one possible embodiment, the robot is equipped with an image sensor; determining training information at each moment in the action sequence includes: determining the robot's body state information at the current moment based on the action sequence, the body state being used to characterize the robot's joint bending angle, the training information including body state information and sensor observation information, the sensor observation information being used to characterize the information collected by the image sensor; determining the robot's sensor observation information at the current moment based on the action sequence, the observation information including: color data, spatial distance data, and three-dimensional point cloud data.

[0045] Among them, the color data image sensor records the object color information in the RGB channel, which is used to distinguish different objects and identify the object state; the spatial distance data is the straight-line distance information from the object to the sensor, which is used to quantify the spatial position of the object and avoid collisions or excessive distances when the robot moves; the three-dimensional point cloud data is a set of three-dimensional coordinate points generated by fusing color data and spatial distance data, which is used to characterize the three-dimensional shape, size and spatial layout of the object, and is the core data to help the model understand the three-dimensional structure of the environment.

[0046] Please refer to the details. Figure 3 , Figure 3 This is a schematic diagram of the architecture of a model provided in an embodiment of this application, such as... Figure 3 As shown, after inputting sensor observation information 310, robot body state information 320, and action description 330 into model 300, model 300 outputs action sequence 1...action sequence N after training. Sensor observation information 310 includes color data 311, spatial distance data 312, and three-dimensional point cloud data 313.

[0047] As can be seen, in this embodiment, the robot's physical state information at the current moment is determined based on the action sequence; and the robot's sensor observation information at the current moment is also determined based on the action sequence. This allows the model to learn action logic by combining the robot's own state with details of the environment, improving the robot's generalization ability to similar text commands, enhancing its adaptability to the environment, and enabling it to be applicable to different environments.

[0048] In one possible embodiment, selecting the overall text description or the sub-action text description with a preset probability includes: determining a first preset probability; selecting the overall text description with the first preset probability and selecting the sub-action text description with a second preset probability, wherein the sum of the first preset probability and the second preset probability is 1.

[0049] The first and second preset probabilities range from 0 to 1. Specifically, the preset probabilities can be set according to the complexity of the task. For example, for simple tasks, such as picking up a cup, the first preset probability can be adjusted to 0.3 to increase the probability of learning sub-actions. For complex tasks, such as assembling toys, it can be set to 0.5.

[0050] Specifically, in other possible embodiments, the training process can be divided into three stages: initial, intermediate, and late. Each model training stage has a pre-defined probability adjustment rule, causing the first and second pre-defined probabilities to change in a stepwise manner with the iteration progress, matching the model's needs at different learning stages. For example, in the initial stage, the model needs to prioritize learning the basic logic of sub-actions, setting the first pre-defined probability to 0.1 and the second pre-defined probability to 0.9; in the intermediate stage, the model needs to establish the association between the whole and sub-actions, increasing the first pre-defined probability by 0.2 every 1000 iterations while simultaneously decreasing the second pre-defined probability; in the late stage, the model needs to strengthen the generalization of the overall task, fixing the first pre-defined probability at 0.8 and the second pre-defined probability at 0.2 to ensure the model prioritizes understanding the semantics of the complete instruction.

[0051] It can also monitor the model's learning effect on sub-actions in real time, such as sub-action prediction accuracy and loss value. When the sub-action is not learned sufficiently, the second preset probability is automatically increased; when the sub-action is mastered, the first preset probability is increased, thus achieving adaptive adjustment of the preset probabilities. For example, the preset sub-action prediction accuracy threshold is 85%, and the sub-action loss value threshold is 0.05. If the sub-action prediction accuracy is <85% in 100 consecutive iterations, the second preset probability is increased by 0.1, up to a maximum of 0.9, while the first preset probability is decreased simultaneously. If the sub-action loss value is <0.05 in 200 consecutive iterations, the first preset probability is increased by 0.1, up to a maximum of 0.9, while the second preset probability is decreased simultaneously.

[0052] As can be seen, in this embodiment, by complementary selection of the first preset probability and the second preset probability, the model learns both the overall action target and the sub-action steps during training. This avoids the model focusing on the overall action and ignoring details due to a single input, or focusing on sub-actions and missing the overall action logic. This improves the model's learning effect and understanding depth of text instructions, and enhances the generalization ability of text processing.

[0053] In one possible embodiment, motion information refers to video image information of the robot performing a target motion; the motion sequence is divided into multiple sub-motions, including: splitting the motion sequence according to the frame range or timestamp in the motion information to obtain multiple sub-motions; wherein, the frame range refers to the continuous image frame interval corresponding to a certain sub-motion in the motion information, and the timestamp refers to the continuous time interval corresponding to a certain sub-motion in the motion information.

[0054] Among them, video image information refers to the collection of continuous video images that record the entire process of a robot performing a target action, captured by image sensors such as RGB cameras and depth cameras.

[0055] Specifically, the action sequence is broken down by frame range. For example, for the sub-action of grabbing the remote control, frames 30-60 are defined from 180 frames of video. Frame 30 is when the gripper begins to approach the remote control, and frame 60 is when the gripper fully closes to grab the remote control. Each frame within the range records the continuous visual changes as the gripper moves to the closing position. The action sequence is also broken down by timestamp. For example, for the sub-action of grabbing the remote control, if the total video duration is 6 seconds, the corresponding timestamp is 1-2 seconds. At 1 second, the gripper begins to approach the remote control, and at 2 seconds, the grabbing is completed, which corresponds to the frame range of 30-60 frames.

[0056] Specifically, the multiple sub-actions can be broken down as follows: For the overall action sequence of using a robotic arm to organize the table (frames 1-180, or 0-6 seconds), the following sub-actions can be obtained: Sub-action 1: Move to the remote control, 1-30 frames (or 0-1 second); Sub-action 2: Grab the remote control, 31-60 frames (or 1-2 seconds); Sub-action 3: Place in the storage box, 61-120 frames (or 2-4 seconds); Sub-action 4: Return to the initial position, 121-180 frames (or 4-6 seconds).

[0057] As can be seen, in this embodiment, by splitting the action sequence according to the frame range or timestamp in the action information, multiple sub-actions are obtained, ensuring that each sub-action is completely aligned with the continuous visual segments in the video. This provides an accurate visual semantic association basis for subsequent text annotation and model training, helps the model learn the mapping relationship between text semantics and action visual features more intuitively, and improves the accuracy of action prediction.

[0058] In one possible embodiment, generating sub-action text descriptions of multiple sub-actions includes: inputting an overall text description of the target action into a visual language model to output an action flow containing multiple sub-actions; and generating corresponding sub-action text descriptions based on the definitions of each sub-action in the action flow.

[0059] Among them, the cross-modal understanding capability of the Visual Language Model (VLM) is used to transform the abstract overall task into an executable sequence of sub-actions, ensuring that the sub-action decomposition conforms to the objective logic of the target action, such as the logical order of gripping the bottle body and turning the cap before twisting the bottle cap.

[0060] Please refer to the details. Figure 4 , Figure 4 This is an application diagram illustrating an action scenario provided in an embodiment of this application, such as... Figure 4 As shown, the following items are placed on the desktop 400: a storage box 410, a tissue box 420, an orange 431, an apple 432, a black box 440, and a marker 450. The action description for storing these items is divided into multiple action flow descriptions; please refer to [link to relevant documentation]. Figure 5 , Figure 5 This is a schematic diagram of a process for outputting text descriptions of sub-actions provided in an embodiment of this application, including... S51: Tissue paper packaging 420; After VLM model conversion, the output is S511: The robot's left arm grabs the tissue package 420 and puts it into the storage box 410.

[0061] S52: Organizing marker 450; After VLM model conversion, the output is S521: the robot's left arm grasps the marker 450, puts the pen cap on, and puts it into the storage box 410.

[0062] S53: Arrange the fruit; After VLM model conversion, the output is S531: the robot's right arm grabs orange 431 and places it on the lower right corner of the table, and the robot's left arm grabs apple 432 and places it on the lower right corner of the table so that it is close to orange 431.

[0063] S54: Adjust the position of black box 440; After VLM model conversion, the output is S541: the robot's right arm grasps the black box 440 and places it in the upper right corner of the table.

[0064] S55: Adjust the position of storage box 410; After VLM model conversion, the output is S551: The robot's two arms adjust the position of the storage box 410 to ensure that it is centered on the table.

[0065] As can be seen, in this embodiment, by inputting the overall text description of the target action into the visual language model, an action flow of multiple sub-actions is obtained. Then, based on the definition of each sub-action in the action flow, the corresponding sub-action text description is generated. This improves the efficiency of sub-action annotation and reduces manual costs. Manual annotation is prone to inconsistencies in sub-action decomposition logic due to differences in subjective understanding. Action annotation based on a large model strictly follows the objective laws of the target action, improving the model's responsiveness to similar text instructions that it has not yet learned.

[0066] In one possible embodiment, after training the target model, the method further includes: determining the training time of the target model; obtaining the output result of the target model, the output result being the action sequence predicted by the target model within a preset time period after the training time; comparing the output result with the real action sequence, the real action sequence being the sequence of the target action within the preset time period after the training time; calculating the loss value of the output result and the real action sequence using a loss function; and adjusting the target model using a gradient descent algorithm to reduce the loss value.

[0067] The loss function is a mathematical function used to quantify the deviation between the predicted action sequence and the actual action sequence. It is the core tool for evaluating the model's prediction error, and the loss value directly reflects the model's current learning performance. For example, for action temporal logic, such as the order of sub-actions, cross-entropy loss is used to quantify the difference in probability distribution between the predicted sub-action type and the actual sub-action type. The gradient descent algorithm is an optimization algorithm that iteratively adjusts model parameters, such as neural network weights and biases, based on the gradient direction of the loss function to reduce the loss value.

[0068] Specifically, training time can be in units of frames, such as selecting the 200th frame from a 450-frame action sequence of putting a pen into a storage box as the training time, and the model predicting subsequent actions based on the training information of the 200th frame; or it can be in units of timestamps, such as selecting the 3rd second from an action sequence of 0-6 seconds as the training time, and the model predicting subsequent actions based on the state information of the 3rd second.

[0069] As can be seen, in this embodiment, after training the target model, the predicted action sequence and the actual action sequence of the target model are compared. A loss algorithm is used to evaluate the model's learning and training effect, and a gradient descent algorithm is used to adjust the target model to improve prediction accuracy. In this way, the action prediction deviation is quantified through the loss function, avoiding subjective evaluation errors, reducing the error in the target model's action prediction, and ensuring the accuracy of the robot's subsequent actions executed according to the model's output.

[0070] This application embodiment can divide the electronic device into functional units according to the above method example. For example, each function can be divided into a separate functional unit, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0071] and Figure 2 The implementation is consistent with the previous one; please refer to [link / reference]. Figure 6 , Figure 6 This is a functional unit block diagram of a model training device provided in an embodiment of this application, such as... Figure 6As shown, the model training device 600 includes: an acquisition unit 610, a first generation unit 620, a second generation unit 630, a determination unit 640, and a training unit 650; wherein, the acquisition unit 610 is used to acquire action information of the robot performing a target action, the action information including the action sequence of the target action; the first generation unit 620 is used to generate an overall text description of the target action based on the action information; the second generation unit 630 is used to split the action sequence into multiple sub-actions and generate sub-action text descriptions of the multiple sub-actions; the determination unit 640 is used to determine the training information at each moment in the action sequence; the training unit 650 is used to select the overall text description or the sub-action text description with a preset probability, and combine it with the training information as model input to train the target model.

[0072] In one possible embodiment, the robot is equipped with an image sensor; in determining the training information at each moment in the action sequence, the determining unit 640 is specifically configured to: determine the robot's body state information at the current moment based on the action sequence, the body state being used to characterize the robot's joint bending angle, the training information including body state information and sensor observation information, the sensor observation information being used to characterize the information acquired by the image sensor; and determine the robot's sensor observation information at the current moment based on the action sequence, the observation information including: color data, spatial distance data, and three-dimensional point cloud data.

[0073] In one possible embodiment, in terms of selecting the overall text description or the sub-action text with a preset probability, the training unit 650 is specifically configured to: determine a first preset probability; select the overall text description with the first preset probability, select the sub-action text description with a second preset probability, and the sum of the first preset probability and the second preset probability is 1.

[0074] In one possible embodiment, the action information refers to video image information of the robot performing a target action; in terms of splitting the action sequence into multiple sub-actions, the second generation unit 630 is specifically used to: split the action sequence according to the frame range or timestamp in the action information to obtain multiple sub-actions; wherein, the frame range refers to the continuous image frame interval corresponding to a certain sub-action in the action information, and the timestamp refers to the continuous time interval corresponding to a certain sub-action in the action information.

[0075] In one possible embodiment, in generating text descriptions of multiple sub-actions, the second generation unit 630 is specifically configured to: input an overall text description of the target action into a visual language model to output an action flow containing multiple sub-actions; and generate corresponding text descriptions of the sub-actions based on the definitions of each sub-action in the action flow.

[0076] In one possible embodiment, after training the target model, the model training device 600 is further configured to: determine the training time of the target model; obtain the output result of the target model, the output result being the action sequence predicted by the target model within a preset time period after the training time; compare the output result with the real action sequence, the real action sequence being the sequence of the target action within the preset time period after the training time; calculate the loss value of the output result and the real action sequence using a loss function; and adjust the target model using a gradient descent algorithm to reduce the loss value.

[0077] In one possible embodiment, the robot includes: a single-arm robot, a dual-arm robot, a humanoid robot, or a dexterous hand, wherein the dexterous hand is a humanoid bionic end effector.

[0078] It is understood that since the method embodiments and the device embodiments are different presentations of the same technical concept, the content of the method embodiment section in this application should be adapted to the device embodiment section in a synchronous manner, and will not be repeated here.

[0079] Figure 7 This is a structural block diagram of an electronic device provided in an embodiment of this application. For example... Figure 7 As shown, electronic device 700 may include one or more components: a processor 701 and a memory 702 coupled to the processor 701, wherein the memory 702 may store one or more computer programs, which may be configured to implement the methods described in the examples above when executed by one or more processors 701. Electronic device 700 may be as follows: Figure 1 The server shown is 110.

[0080] Processor 701 may include one or more processing cores. Processor 701 connects to various parts within the electronic device 700 using various interfaces and lines, and performs various functions and processes data of the electronic device 700 by running or executing instructions, programs, code sets, or instruction sets stored in memory 702, and by calling data stored in memory 702. Optionally, processor 701 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 701 may integrate one or more of a Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. It is understood that the aforementioned modem may also not be integrated into processor 701, but may be implemented separately through a communication chip.

[0081] The memory 702 may include random access memory (RAM) or read-only memory (ROM). The memory 702 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 702 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method examples described above. The data storage area may also store data created during the use of the electronic device 700.

[0082] It is understood that the electronic device 700 may include more or fewer structural elements than those shown in the above block diagram, such as a power module, physical buttons, WiFi (Wireless Fidelity) module, speaker, Bluetooth module, sensor, etc., without limitation.

[0083] This application also provides a computer storage medium storing a computer program / instructions thereon, which, when executed by a processor, implements some or all of the steps of any of the methods described in the above method embodiments.

[0084] This application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program operable to cause a computer to perform some or all of the steps of any of the methods described in the above method embodiments.

[0085] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0086] In the several embodiments provided in this application, it should be understood that the disclosed methods, apparatuses, and systems can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for example, the division of units is merely a logical functional division, and there may be other division methods in actual implementation; for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0087] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0088] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can be physically comprised separately, or two or more units can be integrated into one unit. The integrated unit described above can be implemented in hardware or in the form of hardware plus software functional units.

[0089] The integrated units implemented as software functional units described above can be stored in a computer-readable storage medium. These software functional units, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute partial steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes: a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, volatile memory, or non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM), etc., which are various media capable of storing program code.

[0090] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can easily conceive of variations or substitutions without departing from the spirit and scope of the present invention, and various modifications and alterations can be made, including combinations of the different functions and implementation steps described above, as well as software and hardware implementation methods, all of which are within the protection scope of the present invention.

Claims

1. A model training method, characterized in that, The method comprises the following steps: obtaining action information of a robot performing a target action, wherein the action information comprises an action sequence of the target action; generating an overall text description of the target action according to the action information; splitting the action sequence into a plurality of sub-actions, and generating sub-action text descriptions of the plurality of sub-actions; determining training information at each time in the action sequence; selecting the overall text description or the sub-action text description with a preset probability, and combining the training information as model input to train a target model.

2. The method of claim 1, wherein, The robot is provided with an image sensor; the determination of the training information at each time in the action sequence comprises: determining the body state information of the robot at the current time according to the action sequence, wherein the body state information is used to represent the joint bending angle of the robot, and the training information comprises the body state information and sensor observation information, wherein the sensor observation information is used to represent the information collected by the image sensor; determining the sensor observation information of the robot at the current time according to the action sequence, wherein the observation information comprises color data, spatial distance data and three-dimensional point cloud data.

3. The method of claim 2, wherein, The selection of the overall text description or the sub-action text description with a preset probability comprises: determining a first preset probability; selecting the overall text description with the first preset probability and selecting the sub-action text description with a second preset probability, wherein the sum of the first preset probability and the second preset probability is 1.

4. The method of claim 3, wherein, The action information refers to video image information of the robot performing the target action; the splitting of the action sequence into a plurality of sub-actions comprises: splitting the action sequence according to the frame range or the timestamp in the action information to obtain the plurality of sub-actions; wherein the frame range refers to the continuous image frame interval corresponding to a sub-action in the action information, and the timestamp refers to the continuous time interval corresponding to a sub-action in the action information.

5. The method according to any one of claims 1 to 4, characterized in that, The generation of the sub-action text descriptions of the plurality of sub-actions comprises: inputting the overall text description of the target action into a visual language model to output an action flow containing the plurality of sub-actions; generating corresponding sub-action text descriptions according to the definition of each sub-action in the action flow.

6. The method of claim 1, wherein, After the training of the target model, the method further comprises: determining the training time of the target model; obtaining the output result of the target model, wherein the output result refers to the action sequence predicted by the target model within a preset time period after the training time; comparing the output result with a real action sequence, wherein the real action sequence refers to the sequence in the action sequence of the target action within the preset time period after the training time; calculating the loss value of the output result and the real action sequence through a loss function; adjusting the target model through a gradient descent algorithm to reduce the loss value.

7. The method according to any one of claims 1 to 6, characterized in that, The robot comprises a single-arm robot, a double-arm robot, a humanoid robot or a dexterous hand, and the dexterous hand is a humanoid end effector.

8. A model training apparatus, comprising: The model training apparatus comprises an acquisition unit, a first generation unit, a second generation unit, a determination unit and a training unit, wherein The acquisition unit is configured to acquire action information of a robot performing a target action, the action information comprising an action sequence of the target action; The first generation unit is configured to generate an overall text description of the target action according to the action information; The second generation unit is configured to split the action sequence into a plurality of sub-actions and generate sub-action text descriptions of the plurality of sub-actions; The determination unit is configured to determine training information at each time in the action sequence; The training unit is configured to select the overall text description or the sub-action text description with a preset probability, combine the training information as model input, and train a target model.

9. An electronic device, comprising: A computer program product comprising a processor and a memory configured to store one or more programs for execution by the processor, the programs comprising instructions for performing the steps of the method of any of claims 1-7.

10. A computer-readable storage medium, characterized in that, A computer program for electronic data interchange, wherein the computer program causes a computer to perform the method of any of claims 1-7.

Images