Humanoid robot task planning model training method and task planning method
By using multimodal data fusion and neural network training optimization, the adaptability and generalization capabilities of humanoid robots in multi-task scenarios were solved, achieving more efficient task planning and resource conservation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- 广州里工实业有限公司
- Filing Date
- 2025-11-17
- Publication Date
- 2026-05-12
AI Technical Summary
Existing technologies make it difficult for humanoid robots to adapt to dynamic changes in multi-task scenarios. They have poor scenario adaptability, weak work generalization ability, and insufficient multimodal data fusion, resulting in poor task planning performance.
By acquiring multimodal training data, using neural networks for model training and parameter optimization, integrating visual, linguistic, force-sensing, and gait temporal data, and combining reinforcement learning to optimize the action output layer, cross-scenario task planning model training is achieved.
It improves the adaptability and task planning effectiveness of humanoid robots in multi-task scenarios, and reduces the computational resource requirements for model updates.
Smart Images
Figure CN121340271B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of humanoid robot technology, and in particular to a training method and task planning method for a humanoid robot task planning model. Background Technology
[0002] With the development of robotics technology, humanoid robots, possessing human-like form and mobility, have become an important development direction in the fields of industrial automation and home services.
[0003] Currently, related technologies typically construct task models for single task scenarios (such as industrial assembly scenarios or household cleaning scenarios only), and drive humanoid robots to perform corresponding task actions through the constructed task models. This approach is difficult to adapt to dynamic changes in task scenarios, has poor scenario adaptability, and weak generalization ability of humanoid robots.
[0004] Therefore, the problems with the relevant technologies still need to be solved and optimized. Summary of the Invention
[0005] The purpose of this invention is to at least partially solve one of the technical problems existing in the related art.
[0006] Therefore, one objective of this invention is to provide a training method and a task planning method for a humanoid robot task planning model. The training method can provide a humanoid robot task planning model that can better adapt to the dynamic changes of the task scenario, which is beneficial to improving the humanoid robot's scenario adaptability and work generalization ability.
[0007] To achieve the above-mentioned technical objectives, the technical solutions adopted in the embodiments of this application include:
[0008] In a first aspect, embodiments of this application provide a method for training a humanoid robot task planning model, comprising:
[0009] Acquire first multimodal training data, second multimodal training data, and third multimodal training data. The first and second multimodal training data have the same task scenario, while the first and third multimodal training data have different task scenarios.
[0010] The first multimodal training data is input into the first neural network for model training to obtain the trained first neural network;
[0011] The second multimodal training data is input into the trained first neural network to obtain the first prediction task action;
[0012] Based on the task trajectory data, the parameters of the action output layer in the trained first neural network are optimized to obtain the second neural network; the task trajectory data is the trajectory data formed by the humanoid robot performing the first predicted task action.
[0013] The task trajectory data and the third multimodal training data are input into the second neural network for model training to obtain the trained humanoid robot task planning model.
[0014] In addition, the method according to the above embodiments of this application may also have the following additional technical features:
[0015] Furthermore, in one embodiment of this application, the target training data is input into the target neural network for model training to obtain a trained target neural network, including:
[0016] The target training data is input into the multimodal feature extraction layer in the target neural network for feature extraction to obtain a multimodal feature vector;
[0017] The multimodal feature vector is input into the multimodal fusion layer in the target neural network for attention fusion to obtain multimodal fusion features;
[0018] The multimodal fusion features are input into the action output layer of the target neural network to obtain the trained target neural network;
[0019] Wherein, if the target neural network is a first neural network, then the target training data is the first multimodal training data; or, if the target neural network is a second neural network, then the target training data is the task trajectory data and the third multimodal training data.
[0020] Furthermore, in one embodiment of this application, the multimodal feature vector includes a visual feature vector, a language feature vector, and a temporal feature vector. The step of inputting the target training data into the multimodal feature extraction layer of the target neural network for feature extraction to obtain the multimodal feature vector includes:
[0021] Based on the target training data, visual modal data, language modal data, and temporal modal data are acquired, wherein the temporal modal data includes force temporal data and gait temporal data;
[0022] Visual feature vectors are obtained by extracting visual features from the visual modality data.
[0023] Language feature extraction is performed on the language modality data to obtain the language feature vector;
[0024] Temporal features are extracted from the temporal modal data to obtain the temporal feature vector.
[0025] Furthermore, in one embodiment of this application, the step of inputting the multimodal fusion features into the action output layer of the target neural network to obtain the trained target neural network includes:
[0026] Obtain the real action labels corresponding to the multimodal fusion features;
[0027] The multimodal fusion features are input into the action output layer to obtain the second predicted task action;
[0028] Based on the actual action labels, the loss value of the second predicted task action is calculated to obtain the first target loss value;
[0029] Based on the first target loss value, the parameters of the target neural network are updated to obtain the trained target neural network.
[0030] Furthermore, in one embodiment of this application, the step of optimizing the parameters of the action output layer in the trained first neural network based on task trajectory data to obtain a second neural network includes:
[0031] Obtain the target loss function, and obtain trajectory sample data based on the task trajectory data;
[0032] Perform sample dominance analysis on the trajectory sample data to obtain sample dominance data;
[0033] Based on the target loss function, loss analysis is performed on the sample advantage data to obtain a second target loss value;
[0034] Based on the second target loss value, the parameters of the action output layer in the trained first neural network are optimized to obtain the second neural network.
[0035] Furthermore, in one embodiment of this application, the method further includes:
[0036] Obtain the fourth multimodal training data;
[0037] Based on the fourth multimodal training data, the parameters of the multimodal fusion layer in the trained humanoid robot task planning model are fine-tuned to obtain the fine-tuned humanoid robot task planning model.
[0038] The first multimodal training data is updated based on the fourth multimodal training data, and the first neural network is updated based on the fine-tuned humanoid robot task planning model. Then, the process returns to the step of inputting the first multimodal training data into the first neural network for model training to obtain the trained first neural network.
[0039] Secondly, embodiments of this application provide a task planning method, including:
[0040] Acquire multimodal scene data collected by humanoid robots;
[0041] The multimodal scene data is input into the pre-trained humanoid robot task planning model to perform task planning, thereby obtaining the humanoid robot's task planning data.
[0042] Thirdly, embodiments of this application provide a training system for a humanoid robot task planning model, comprising:
[0043] The first processing unit is used to acquire first multimodal training data, second multimodal training data, and third multimodal training data. The first multimodal training data and the second multimodal training data have the same task scenario, while the first multimodal training data and the third multimodal training data have different task scenarios.
[0044] The second processing unit is used to input the first multimodal training data into the first neural network for model training, so as to obtain the trained first neural network.
[0045] The third processing unit is used to input the second multimodal training data into the trained first neural network to obtain the first prediction task action;
[0046] The fourth processing unit is used to optimize the parameters of the action output layer in the trained first neural network based on the task trajectory data to obtain the second neural network; the task trajectory data is the trajectory data formed by the humanoid robot performing the first predicted task action;
[0047] The fifth processing unit is used to input the task trajectory data and the third multimodal training data into the second neural network for model training, so as to obtain the trained humanoid robot task planning model.
[0048] Fourthly, embodiments of this application also provide an electronic device, including:
[0049] At least one processor;
[0050] At least one memory for storing at least one program;
[0051] When the at least one program is executed by the at least one processor, the at least one processor performs the method described above.
[0052] Fifthly, embodiments of this application also provide a computer-readable storage medium storing a processor-executable program, which, when executed by the processor, is used to implement the above-described method.
[0053] The advantages and beneficial effects of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application:
[0054] This application discloses a training method and task planning method for a humanoid robot task planning model. The method involves acquiring first multimodal training data, second multimodal training data, and third multimodal training data. The first and second multimodal training data have the same task scenario, while the task scenarios of the first and third multimodal training data are different. The first multimodal training data is input into a first neural network for model training to obtain a trained first neural network. The second multimodal training data is input into the trained first neural network to obtain a first predicted task action. Based on task trajectory data, the parameters of the action output layer in the trained first neural network are optimized to obtain a second neural network. The task trajectory data is the trajectory data formed by the humanoid robot performing the first predicted task action. The task trajectory data and the third multimodal training data are input into the second neural network for model training to obtain the trained humanoid robot task planning model. This method trains a second neural network by merging third multimodal data from a specific task scenario with task trajectory data obtained from a humanoid robot's successful execution in another task scenario. This improves the model's adaptability to dynamic changes in task scenarios and enhances the humanoid robot's scenario adaptability and work generalization ability. Attached Figure Description
[0055] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the following description is provided with accompanying drawings of the relevant technical solutions in the embodiments of this application or the prior art. It should be understood that the accompanying drawings described below are only for the purpose of clearly illustrating some embodiments of the technical solutions in this application. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.
[0056] Figure 1 A flowchart illustrating a training method for a humanoid robot task planning model provided in an embodiment of this application;
[0057] Figure 2 A schematic diagram of the framework of a training system for a humanoid robot task planning model provided in an embodiment of this application;
[0058] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0059] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.
[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.
[0061] Currently, related technologies typically build task models for single task scenarios (such as industrial assembly scenarios or household cleaning scenarios only), and drive humanoid robots to perform corresponding task actions through the constructed task models. This approach often requires manual reprogramming or full parameter training of the task model when switching scenarios, making it difficult to adapt to dynamic scenario changes such as changes in part models in industry, adjustments in furniture layout in households, and education and care. The scenario adaptability is poor and the generalization ability of humanoid robots is weak.
[0062] Furthermore, related technologies typically rely on visual and linguistic data for task planning, neglecting the core force feedback of humanoid robots (such as force control when grasping parts and contact pressure perception when assisting users) and gait stability data (such as center adjustment during walking and adaptation to ground flatness). This means that this approach suffers from insufficient multimodal data fusion, resulting in poor task planning performance. Specifically, in industrial scenarios, the lack of force data can easily lead to parts being assembled too tightly (damaging components) or too loosely (assembly failure). In home scenarios, the lack of gait data can easily cause the robot to fall when switching between carpet and floor, and the lack of force data may result in excessive pressure when assisting users, potentially causing injury.
[0063] Furthermore, related technologies typically train task models based on static expert datasets, and then deploy the models to humanoid robots after training. Subsequent updates to the task models are achieved through full parameter fine-tuning. Since humanoid robot task models usually contain complex multimodal networks, this approach struggles to adjust strategies based on the actual performance of the humanoid robot's tasks. It also requires high-performance GPUs, resulting in significant computational resource demands for updates. Consequently, its applicability is limited in grassroots factory or home settings where computing power and equipment costs are relatively scarce.
[0064] It should be noted that the aforementioned related technologies are only used to assist in understanding the technical solutions of this application and do not mean that they belong to the publicly disclosed prior art.
[0065] In view of this, embodiments of this application provide a training method and a task planning method for a humanoid robot task planning model. The training method trains a second neural network by merging third multimodal data in a certain task scenario with task trajectory data obtained by the humanoid robot in another task scenario. This can improve the model's adaptability to dynamic changes in task scenarios and is beneficial to improving the humanoid robot's scenario adaptability and work generalization ability.
[0066] Furthermore, this method extracts multimodal data containing force perception time-series data and gait time-series data, and fuses them with visual feature vectors and language feature vectors. This can effectively improve the multimodal data fusion effect, thereby improving the task planning effect of the subsequent model output.
[0067] Furthermore, in practical applications, this method fine-tunes the parameters of the multimodal fusion layer in the task planning model and updates the model based on the fine-tuned task planning model, thereby enabling dynamic data training of the task planning model and reducing the computational resources required for updating the task planning model after deployment.
[0068] Reference Figure 1 In this embodiment of the application, a method for training a humanoid robot task planning model includes:
[0069] Step 110: Obtain first multimodal training data, second multimodal training data, and third multimodal training data. The first and second multimodal training data have the same task scenario, while the first and third multimodal training data have different task scenarios.
[0070] In this embodiment, the first multimodal training data can be multimodal training data collected offline or online by a humanoid robot in industrial scenarios (such as mechanical assembly) or household scenarios (such as housework and companionship). The multimodal training data includes scene visual frames, natural language task instructions, force sensing time-series data, and gait time-series data. The scene visual frames can be collected by a camera device mounted on the humanoid robot, the natural language task instructions can be collected by an audio acquisition device mounted on the humanoid robot, the force sensing time-series data can be collected by a force sensor mounted on the humanoid robot, and the gait time-series data can be collected by an IMU sensor mounted on the humanoid robot.
[0071] It is understood that, in another optional implementation, the first multimodal training data also includes real action labels; specifically, if the first multimodal training data is multimodal training data in an industrial scenario (such as mechanical assembly), the corresponding real action labels may include the spatial pose (coordinates, rotation angle), joint torque distribution, and action timing (such as the speed of tightening screws, pause intervals), etc. of the humanoid robot end effector; or, if the first multimodal training data is multimodal training data in a home scenario (such as helping a user get up under companionship and care), the corresponding real action labels may include parameters that can directly guide the robot's execution, such as the magnitude of the humanoid robot's arm support force on the user, the stride length / step frequency of the humanoid robot's lower limb gait, and the body forward lean angle.
[0072] For example, for multimodal training data in industrial scenarios, one example of such multimodal training data is scene visual frames: video of the part assembly process; natural language task instructions: "assemble bearing to bushing", "tighten M8 bolt", etc.; real motion labels: robotic arm joint angles (14 axes), gripping force (50-200N); force data: contact force (0-500N); gait data: stance coordinates (x / y / z), center of gravity offset (±5cm); where force data and gait data are data of the humanoid robot at a certain time point in the time series.
[0073] For multimodal training data in home scenarios, one example of such multimodal training data is scene visual frames: tidying up the table, video of a user falling down; natural language task instructions: "hand the water glass to the coffee table" "help the user get up", etc.; real action labels: hand grasping trajectory (x / y / z axis coordinates), assistance force (50-80N); force data: object contact force (5-30N), assistance pressure (50-80N); gait data: walking stride length (30-50cm), cadence (0.8-2 steps / second, 0.8-1 steps / second when the user is assisting).
[0074] It should be noted that the content of the second and third multimodal training data is similar to that of the first multimodal training data mentioned above. The difference is that when the first and second multimodal training data are multimodal training data in an industrial scenario, the corresponding third multimodal training data is multimodal training data in a home scenario; or, when the first and second multimodal training data are multimodal training data in a home scenario, the corresponding third multimodal training data is multimodal training data in an industrial scenario.
[0075] Step 120: Input the first multimodal training data into the first neural network for model training to obtain the trained first neural network;
[0076] In this embodiment, the first multimodal training data can be input into the first neural network for model training. Various specific model training methods exist; for example, the parameters of the first neural network can be updated based on the backpropagation algorithm, and the first neural network can be obtained after several iterations. The specific number of iterations can be preset, or training can be considered complete when the test set reaches the required accuracy.
[0077] In some embodiments, target training data is input into a target neural network for model training to obtain a trained target neural network, including:
[0078] The target training data is input into the multimodal feature extraction layer in the target neural network for feature extraction to obtain a multimodal feature vector;
[0079] Further, the multimodal feature vector includes visual feature vector, language feature vector, and temporal feature vector. The step of inputting the target training data into the multimodal feature extraction layer of the target neural network for feature extraction to obtain the multimodal feature vector includes:
[0080] Based on the target training data, visual modal data, language modal data, and temporal modal data are acquired, wherein the temporal modal data includes force temporal data and gait temporal data;
[0081] Visual feature vectors are obtained by extracting visual features from the visual modality data.
[0082] Language feature extraction is performed on the language modality data to obtain the language feature vector;
[0083] Temporal features are extracted from the temporal modal data to obtain the temporal feature vector.
[0084] In this embodiment, if the target neural network is a first neural network, the first neural network includes a multimodal feature extraction layer, a multimodal fusion layer, and an action output layer. The multimodal feature extraction layer includes a visual feature extraction sublayer, a language feature extraction sublayer, and a temporal feature extraction sublayer. The visual feature extraction sublayer can be a ResNet-50 network, the language feature extraction sublayer can be a BERT model, and the temporal feature extraction sublayer can be a bidirectional long short-term memory network (LSTM) model. The multimodal fusion layer can be a Transformer encoder.
[0085] It is understandable that visual modality data can be the aforementioned scene visual frames, and visual feature extraction can be performed by inputting the visual modality data into a visual feature extraction sublayer to extract visual features, thereby obtaining a visual feature vector; language modality data can be the aforementioned natural language task instructions, and language feature extraction can be performed by inputting the language modality data into a language feature extraction sublayer to extract language features, thereby obtaining a language feature vector; while temporal modality data can be the aforementioned gait data and force perception data, and temporal feature extraction can be performed by inputting the temporal modality data into a temporal feature extraction sublayer to extract temporal features, thereby obtaining a temporal feature vector.
[0086] The multimodal feature vector is input into the multimodal fusion layer in the target neural network for attention fusion to obtain multimodal fusion features;
[0087] In this embodiment, a multimodal feature vector containing temporal feature vectors, language feature vectors, and visual feature vectors can be input into the multimodal fusion layer of the first neural network for multimodal fusion. The feature vectors of each modality are fused through the multimodal fusion layer to obtain multimodal fused features.
[0088] The multimodal fusion features are input into the action output layer of the target neural network to obtain the trained target neural network;
[0089] Further, the step of inputting the multimodal fusion features into the action output layer of the target neural network to obtain the trained target neural network includes:
[0090] Obtain the real action labels corresponding to the multimodal fusion features;
[0091] The multimodal fusion features are input into the action output layer to obtain the second predicted task action;
[0092] Based on the actual action labels, the loss value of the second predicted task action is calculated to obtain the first target loss value;
[0093] Based on the first target loss value, the parameters of the target neural network are updated to obtain the trained target neural network.
[0094] In this embodiment, the motion output layer may include an adaptive force adjustment module (including a fully connected layer and a ReLU activation function layer) and a gait planning module (including a connected layer and a linear activation function layer).
[0095] The multimodal fusion features are mapped to 1D gripping force parameters through a fully connected layer in the adaptive force adjustment module, and then the gripping force / contact force adjustment parameters (such as force control values of 0-200N) are output through a ReLU activation function layer. Additionally, the gait features and visual features in the gait planning module are mapped to 3D gait parameters through a fully connected layer, and then the bipedal gait adjustment parameters for the humanoid robot (such as stride length, stride frequency, and center of gravity offset) are output through a linear activation function layer. Finally, all the output parameters are combined with the motion trajectory features in the multimodal fusion features (which integrate spatial trajectory information from the scene's visual frames, the action intent of the language commands, and the temporal features of force and gait) to generate the second predicted task action.
[0096] Understandably, the loss value calculation can measure the degree of difference between the humanoid robot's actual action label and the second predicted task action. Specifically, it can be calculated using a loss function to determine the first target loss value between the actual action label and the second predicted task action. There are various types of loss functions, including any combination of one or more loss functions such as squared loss function, absolute loss function, logarithmic loss function, cross-entropy loss function, and mean squared error loss function. This application does not impose any restrictions on this. After obtaining the first target loss value, the parameters of the first neural network can be updated based on this first target loss value, thereby obtaining the trained target neural network.
[0097] It should be noted that if the target neural network is the second neural network, then the conclusion is similar to the previous conclusion that the target neural network is the first neural network, and can be easily deduced by analogy. This application will not elaborate further here.
[0098] Step 130: Input the second multimodal training data into the trained first neural network to obtain the first prediction task action;
[0099] In this embodiment, second multimodal training data can be input into a trained first neural network. The task scenario of the second multimodal training data is the same as that of the first multimodal training data. The trained first neural network predicts the task actions that the humanoid robot needs to perform under the second multimodal training data, thereby obtaining a first predicted task action. The content of the first predicted task action is similar to the content of the aforementioned second predicted task action. The system provides the humanoid robot with the ability to perform the first predicted task action and records the humanoid robot's motion trajectory. After the humanoid robot completes the first predicted task action, the collected motion trajectory is determined as task trajectory data.
[0100] Step 140: Based on the task trajectory data, optimize the parameters of the action output layer in the trained first neural network to obtain the second neural network; the task trajectory data is the trajectory data formed by the humanoid robot performing the first predicted task action;
[0101] In this embodiment of the application, the parameters of the action output layer in the trained first neural network can be optimized based on reinforcement learning (RL) technology and combined with task trajectory data to obtain the second neural network.
[0102] In some embodiments, optimizing the parameters of the action output layer in the trained first neural network based on task trajectory data to obtain a second neural network includes:
[0103] Obtain the target loss function, and obtain trajectory sample data based on the task trajectory data;
[0104] Perform sample dominance analysis on the trajectory sample data to obtain sample dominance data;
[0105] Based on the target loss function, loss analysis is performed on the sample advantage data to obtain a second target loss value;
[0106] Based on the second target loss value, the parameters of the action output layer in the trained first neural network are optimized to obtain the second neural network.
[0107] In this embodiment, the parameters of the action output layer in the trained first neural network can be optimized based on the Proximal Policy Optimization (PPO) algorithm. During the parameter optimization process, the parameters of the multimodal feature extraction layer and the multimodal fusion layer in the trained first neural network are frozen, and only the parameters of the action output layer are updated.
[0108] Understandably, trajectory sample data can be constructed based on task trajectory data, as well as the corresponding first predicted task action and second multimodal training data. This trajectory sample data includes state s (second multimodal training data), action a (first predicted task action), and next state s', where the next state s' can be determined based on the task trajectory data. Sample advantage analysis can be performed based on Generalized Advantage Estimation (GAE) to analyze the trajectory sample data and obtain sample advantage data, which can be expressed as:
[0109]
[0110] in, The advantage function is the representation of the generalized advantage estimation, which is the sample advantage data at the current time step t; T is the total time steps of the trajectory. As a variable; The discount factor represents the decay coefficient of future rewards; the specific value in this application example is 0.95. The time-series difference error can be calculated from the state s' at time step t+1 and the corresponding value function. There are various ways to calculate this time-series difference error in the generalized advantage estimation (GAE) technique, which will not be elaborated here.
[0111] It should be noted that loss analysis can be performed by substituting the sample's advantage function into the target loss function to obtain a second target loss value, which can be expressed as:
[0112]
[0113] in, This is the functional representation of the target loss function, also known as the second target loss value; The current strategy, i.e., the optimized parameters of the action output layer. ; The old strategy refers to the parameters of the action output layer before optimization. ; It is a minimum value function; The sample dominance data at a certain time step; It is a range function; The pruning factor is used to limit the policy update magnitude. In this embodiment, the pruning factor is 0.2. This value can prevent the policy update magnitude from being too large, which would cause model oscillation, while ensuring update efficiency.
[0114] It is worth mentioning that the parameter optimization of the action output layer can be achieved by minimizing the target loss function through gradient descent, thereby iteratively optimizing the parameters of the action output layer until the loss function converges to obtain the second neural network.
[0115] Step 150: Input the task trajectory data and the third multimodal training data into the second neural network for model training to obtain the trained humanoid robot task planning model.
[0116] In the embodiments of this application, in the first implementation, all data in the task trajectory data can be input into the second neural network along with the third multimodal training data. The task scenario of the third multimodal training data is different from that of the first multimodal training data (that is, the task scenario of the third multimodal training data is different from that of the task trajectory data). The second neural network performs multi-scenario fusion training on the cross-scenario task trajectory data and the third multimodal training data to obtain a well-trained humanoid robot task planning model.
[0117] In the second implementation, the trajectory data corresponding to the humanoid robot's successful execution of the first predicted task action can be input into the second neural network along with the third multimodal training data. The second neural network then performs multi-scene fusion training on the cross-scene task trajectory data and the third multimodal training data to obtain a trained humanoid robot task planning model. The specific model training content is similar to that in step 120 above and can be easily deduced.
[0118] In some embodiments, the method further includes:
[0119] Obtain the fourth multimodal training data;
[0120] Based on the fourth multimodal training data, the parameters of the multimodal fusion layer in the trained humanoid robot task planning model are fine-tuned to obtain the fine-tuned humanoid robot task planning model.
[0121] The first multimodal training data is updated based on the fourth multimodal training data, and the first neural network is updated based on the fine-tuned humanoid robot task planning model. Then, the process returns to the step of inputting the first multimodal training data into the first neural network for model training to obtain the trained first neural network.
[0122] In this embodiment, when the humanoid robot task planning model needs to be updated after deployment, the model can be updated by parameter fine-tuning. This parameter fine-tuning can be based on the low-rank adaptation (LORA) fine-tuning algorithm, which inputs the fourth multimodal training data into the trained humanoid robot task planning model to fine-tune the parameters of the multimodal fusion layer, thereby obtaining the fine-tuned humanoid robot task planning model.
[0123] It is understandable that the fourth multimodal training data can be multimodal training data subsequently collected by the manufacturer after the humanoid robot task planning model is deployed to the humanoid robot; or multimodal training data collected by the humanoid robot in the user's scenario after obtaining user authorization. This fourth multimodal training data is similar to the aforementioned first multimodal training data, and can be easily deduced by analogy. Then, the fourth multimodal training data is added to the first multimodal training data, and the fine-tuned humanoid robot task planning model is determined as the first neural network, and the process returns to step 120.
[0124] In this embodiment of the application, a task planning method includes:
[0125] Acquire multimodal scene data collected by humanoid robots;
[0126] The multimodal scene data is input into the pre-trained humanoid robot task planning model to perform task planning, thereby obtaining the humanoid robot's task planning data.
[0127] In this embodiment, a humanoid robot can collect multimodal scene data of the user's location and input the multimodal scene data into a trained humanoid robot task planning model. The model outputs corresponding task planning data for the humanoid robot to execute. The content of the task planning data is similar to the content of the aforementioned first predicted task action and / or second predicted task action, and can be easily deduced by analogy.
[0128] Reference Figure 2 The present application proposes a training system for a humanoid robot task planning model, including...
[0129] The first processing unit 101 is used to acquire first multimodal training data, second multimodal training data and third multimodal training data. The first multimodal training data and the second multimodal training data have the same task scenario, while the first multimodal training data and the third multimodal training data have different task scenarios.
[0130] The second processing unit 102 is used to input the first multimodal training data into the first neural network for model training, so as to obtain the trained first neural network.
[0131] The third processing unit 103 is used to input the second multimodal training data into the trained first neural network to obtain the first prediction task action;
[0132] The fourth processing unit 104 is used to optimize the parameters of the action output layer in the trained first neural network based on the task trajectory data to obtain a second neural network; the task trajectory data is the trajectory data formed by the humanoid robot performing the first predicted task action;
[0133] The fifth processing unit 105 is used to input the task trajectory data and the third multimodal training data into the second neural network for model training to obtain the trained humanoid robot task planning model.
[0134] Reference Figure 3 This application also provides an electronic device, including:
[0135] At least one processor 201;
[0136] At least one memory 202 is used to store at least one program;
[0137] When the at least one program is executed by the at least one processor 201, the at least one processor 201 implements the method embodiment described above.
[0138] Similarly, it can be understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0139] This application also provides a computer-readable storage medium storing a program executable by a processor 201, which, when executed by the processor 201, is used to implement the above-described method embodiments.
[0140] Similarly, the content of the above method embodiments is applicable to the present computer-readable storage medium embodiments. The specific functions implemented by the present computer-readable storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.
[0141] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.
[0142] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods.
[0143] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this application are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.
[0144] Furthermore, although this application is described in the context of functional modules, it should be understood that, unless otherwise stated to the contrary, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding this application. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional technology for an engineer. Therefore, those skilled in the art can implement the application set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of this application, which is determined by the full scope of the appended claims and their equivalents.
[0145] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods in the embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0146] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.
[0147] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.
[0148] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.
[0149] In the foregoing description of this specification, the references to terms such as "one embodiment," "another embodiment," or "some embodiments," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.
[0150] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.
[0151] The above is a detailed description of the preferred embodiments of this application, but this application is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application, and these equivalent modifications or substitutions are all included within the scope defined by the claims of this application.
Claims
1. A training method for a humanoid robot task planning model, characterized in that, include: Acquire first multimodal training data, second multimodal training data, and third multimodal training data. The first and second multimodal training data have the same task scenario, while the first and third multimodal training data have different task scenarios. The first multimodal training data is input into the first neural network for model training to obtain the trained first neural network; The second multimodal training data is input into the trained first neural network to obtain the first prediction task action; Based on the task trajectory data, the parameters of the action output layer in the trained first neural network are optimized to obtain the second neural network; the task trajectory data is the trajectory data formed by the humanoid robot performing the first predicted task action. The task trajectory data and the third multimodal training data are input into the second neural network for model training to obtain a trained humanoid robot task planning model; The step of optimizing the parameters of the action output layer in the trained first neural network based on the task trajectory data to obtain the second neural network includes: Obtain the target loss function, and obtain trajectory sample data based on the task trajectory data; Perform sample dominance analysis on the trajectory sample data to obtain sample dominance data; Based on the target loss function, loss analysis is performed on the sample advantage data to obtain a second target loss value; Based on the second target loss value, the parameters of the action output layer in the trained first neural network are optimized to obtain the second neural network.
2. The method according to claim 1, characterized in that, The target training data is input into the target neural network for model training, resulting in a trained target neural network, including: The target training data is input into the multimodal feature extraction layer in the target neural network for feature extraction to obtain a multimodal feature vector; The multimodal feature vector is input into the multimodal fusion layer in the target neural network for attention fusion to obtain multimodal fusion features; The multimodal fusion features are input into the action output layer of the target neural network to obtain the trained target neural network; Wherein, if the target neural network is a first neural network, then the target training data is the first multimodal training data; or, if the target neural network is a second neural network, then the target training data is the task trajectory data and the third multimodal training data.
3. The method according to claim 2, characterized in that, The multimodal feature vector includes visual feature vectors, language feature vectors, and temporal feature vectors. The step of inputting the target training data into the multimodal feature extraction layer of the target neural network for feature extraction to obtain the multimodal feature vector includes: Based on the target training data, visual modal data, language modal data, and temporal modal data are acquired, wherein the temporal modal data includes force temporal data and gait temporal data; Visual feature vectors are obtained by extracting visual features from the visual modality data. Language feature extraction is performed on the language modality data to obtain the language feature vector; Temporal features are extracted from the temporal modal data to obtain the temporal feature vector.
4. The method according to claim 2, characterized in that, The step of inputting the multimodal fusion features into the action output layer of the target neural network to obtain the trained target neural network includes: Obtain the real action labels corresponding to the multimodal fusion features; The multimodal fusion features are input into the action output layer to obtain the second predicted task action; Based on the actual action labels, the loss value of the second predicted task action is calculated to obtain the first target loss value; Based on the first target loss value, the parameters of the target neural network are updated to obtain the trained target neural network.
5. The method according to claim 1, characterized in that, The method further includes: Obtain the fourth multimodal training data; Based on the fourth multimodal training data, the parameters of the multimodal fusion layer in the trained humanoid robot task planning model are fine-tuned to obtain the fine-tuned humanoid robot task planning model. The first multimodal training data is updated based on the fourth multimodal training data, and the first neural network is updated based on the fine-tuned humanoid robot task planning model. Then, the process returns to the step of inputting the first multimodal training data into the first neural network for model training to obtain the trained first neural network.
6. A task planning method, characterized in that, include: Acquire multimodal scene data collected by humanoid robots; The multimodal scene data is input into the trained humanoid robot task planning model for task planning, thereby obtaining the humanoid robot's task planning data; The trained humanoid robot task planning model is obtained by training the model as described in any one of claims 1-5.
7. A training system for a humanoid robot task planning model, characterized in that, include: The first processing unit is used to acquire first multimodal training data, second multimodal training data, and third multimodal training data. The first multimodal training data and the second multimodal training data have the same task scenario, while the first multimodal training data and the third multimodal training data have different task scenarios. The second processing unit is used to input the first multimodal training data into the first neural network for model training, so as to obtain the trained first neural network. The third processing unit is used to input the second multimodal training data into the trained first neural network to obtain the first prediction task action; The fourth processing unit is used to optimize the parameters of the action output layer in the trained first neural network based on the task trajectory data to obtain the second neural network; the task trajectory data is the trajectory data formed by the humanoid robot performing the first predicted task action; The fifth processing unit is used to input the task trajectory data and the third multimodal training data into the second neural network for model training, so as to obtain a trained humanoid robot task planning model. The step of optimizing the parameters of the action output layer in the trained first neural network based on the task trajectory data to obtain the second neural network includes: Obtain the target loss function, and obtain trajectory sample data based on the task trajectory data; Perform sample dominance analysis on the trajectory sample data to obtain sample dominance data; Based on the target loss function, loss analysis is performed on the sample advantage data to obtain a second target loss value; Based on the second target loss value, the parameters of the action output layer in the trained first neural network are optimized to obtain the second neural network.
8. An electronic device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor performs the method as described in any one of claims 1-6.
9. A computer-readable storage medium storing a processor-executable program, characterized in that, The processor-executable program, when executed by the processor, is used to implement the method as described in any one of claims 1-6.