Model training method, mobile device, electronic device, and medium

Abstract

The application provides a model training method, a mobile device, an electronic device and a medium. The model training method comprises: executing a test task by using a task execution model, and obtaining test data corresponding to a target period in the process of executing the test task; inputting the test data into an initial state judgment model to obtain an initial state judgment result; in a case where a test result corresponding to the test data meets a preset requirement and the initial state judgment result is inconsistent with the test result, storing the test data as positive sample data in a positive sample database; in a case where the test result does not meet the preset requirement and the initial state judgment result is inconsistent with the test result, storing the test data as negative sample data in a negative sample database; and training the initial state judgment model by using the positive sample data in the positive sample database and the negative sample data in the negative sample database. The technical solution of the application can reduce the entire training period of the model and improve the training efficiency.

Description

Technical Field

[0001] This application relates to the field of model training technology, specifically to a model training method, mobile device, electronic device, and medium. Background Technology

[0002] In the field of artificial intelligence, models are a core component, serving as the mathematical and computational framework for realizing intelligent behavior. Specifically, a model can be an abstract framework used to describe a system, structure, or process, learning patterns from data and establishing input-output mappings. For example, training an image recognition model with sample data enables the model to identify target objects in images. Currently, the process of training a model to obtain a final, satisfactory model suffers from excessively long timelines. Summary of the Invention

[0003] In view of this, embodiments of this application provide a model training method, mobile device, electronic device, and medium that can reduce the overall training cycle of the model and improve training efficiency.

[0004] In a first aspect, embodiments of this application provide a model training method, comprising: executing a test task using a task execution model to obtain test data corresponding to a target time period during the execution of the test task; correcting the test data to obtain corrected data when the test results corresponding to the test data do not meet preset requirements; and training the task execution model using the corrected data to obtain a trained task execution model.

[0005] Secondly, embodiments of this application provide a model training method, comprising: executing a test task using a task execution model to obtain test data corresponding to a target time period during the execution of the test task; inputting the test data into an initial state judgment model to obtain an initial state judgment result, wherein the initial state judgment result is used to characterize whether the execution process of the test task during the target time period, as determined by the initial state judgment model, has encountered an error; if the test result corresponding to the test data meets a preset requirement and the initial state judgment result is inconsistent with the test result, storing the test data as positive sample data in a positive sample database, wherein the test result is used to characterize whether the execution process of the test task during the target time period has actually encountered an error; if the test result does not meet the preset requirement and the initial state judgment result is inconsistent with the test result, storing the test data as negative sample data in a negative sample database; training the initial state judgment model using the positive sample data in the positive sample database and the negative sample data in the negative sample database to obtain a state judgment model, wherein the state judgment model is used to obtain a state judgment result based on the test data, and the state judgment result is used as the test result to characterize whether an error has occurred during the execution of the test task.

[0006] Thirdly, embodiments of this application provide a task execution method, including: executing a target task using a task execution model, obtaining execution data corresponding to a target time period during the execution of the target task; inputting the execution data into a state judgment model to obtain a state judgment result, wherein the state judgment result is used to characterize whether the execution result corresponding to the execution data meets preset requirements; and issuing a reminder message when the state judgment result characterizes that the execution result corresponding to the execution data does not meet preset requirements.

[0007] Fourthly, embodiments of this application provide a model training apparatus, comprising: an execution module for executing a test task using a task execution model and obtaining test data corresponding to a target time period during the execution of the test task; a correction module for correcting the test data to obtain corrected data when the test results corresponding to the test data do not meet preset requirements; and a training module for training the task execution model using the corrected data to obtain a trained task execution model.

[0008] Fifthly, embodiments of this application provide a model training apparatus, comprising: an execution module, configured to execute a test task using a task execution model and acquire test data corresponding to a target time period during the execution of the test task; an input module, configured to input the test data into an initial state judgment model to obtain an initial state judgment result, wherein the initial state judgment result is used to characterize whether the execution process of the test task during the target time period, as determined by the initial state judgment model, has encountered an error; a storage module, configured to: store the test data as positive sample data in a positive sample database when the test result corresponding to the test data meets preset requirements and the initial state judgment result is inconsistent with the test result, wherein the test result is used to characterize whether the execution process of the test task during the target time period has actually encountered an error; and store the test data as negative sample data in a negative sample database when the test result does not meet preset requirements and the initial state judgment result is inconsistent with the test result; and a training module, configured to train the initial state judgment model using positive sample data in the positive sample database and negative sample data in the negative sample database to obtain a state judgment model, wherein the state judgment model is used to obtain a state judgment result based on the test data, and the state judgment result is used as the test result to characterize whether an error has occurred during the execution of the test task.

[0009] Sixthly, embodiments of this application provide a task execution device, comprising: an execution module, configured to execute a target task using a task execution model and obtain execution data corresponding to a target time period during the execution of the target task; an input module, configured to input the execution data into a state judgment model to obtain a state judgment result, wherein the state judgment result is used to characterize whether the execution result corresponding to the execution data meets preset requirements; and an output module, configured to issue a reminder message when the state judgment result characterizes that the execution result corresponding to the execution data does not meet preset requirements.

[0010] In a seventh aspect, embodiments of this application provide a mobile device including a control module, which is used to execute the model training method described in the first aspect, the model training method described in the second aspect, or the task execution method described in the third aspect.

[0011] Eighthly, embodiments of this application provide an electronic device, including: a processor; and a memory for storing processor-executable instructions, wherein the processor is configured to execute the model training method described in the first aspect, the model training method described in the second aspect, or the task execution method described in the third aspect.

[0012] Ninthly, embodiments of this application provide a computer-readable storage medium storing a computer program for executing the model training method described in the first aspect, the model training method described in the second aspect, or the task execution method described in the third aspect.

[0013] In a tenth aspect, embodiments of this application provide a computer program product comprising a computer program that, when executed by a processor of a computer device, enables the computer device to perform the model training method described in the first aspect, the model training method described in the second aspect, or the task execution method described in the third aspect.

[0014] Eleventhly, embodiments of this application provide a chip, including: a processor; and a memory for storing processor-executable instructions, wherein the processor is used to execute the model training method described in the first aspect, the model training method described in the second aspect, or the task execution method described in the third aspect.

[0015] This application provides a model training method, mobile device, electronic device, and medium. The method involves testing a task execution model, specifically executing a test task using the task execution model, and obtaining test data corresponding to the target time period during the task execution. If the test results do not meet preset requirements, such as indicating an error during task execution, the test data is corrected to obtain corrected data. This corrected data is then used to train the task execution model, resulting in a trained model. This approach allows for focus on aspects of the task execution model that perform poorly during testing; that is, correcting the test data when it does not meet preset requirements. Further training the model using the corrected data can rapidly improve model convergence, reduce the overall training cycle, and increase training efficiency. Attached Figure Description

[0016] Figure 1 The diagram shown is a schematic representation of the system architecture of a model training system provided in an exemplary embodiment of this application.

[0017] Figure 2 The diagram shown is a flowchart illustrating a model training method provided in an exemplary embodiment of this application.

[0018] Figure 3a The diagram shown is a schematic representation of a manual takeover process provided in an exemplary embodiment of this application.

[0019] Figure 3b The diagram shown is a schematic representation of a manual takeover process provided in another exemplary embodiment of this application.

[0020] Figure 4a The diagram shown is a flowchart of a model training method provided in another exemplary embodiment of this application.

[0021] Figure 4b The diagram shown is a schematic representation of a data acquisition process provided in an exemplary embodiment of this application.

[0022] Figure 5 The diagram shown is a flowchart of a model training method provided in another exemplary embodiment of this application.

[0023] Figure 6 The diagram shown is a flowchart illustrating a task execution method provided in an exemplary embodiment of this application.

[0024] Figure 7 The diagram shown is a flowchart illustrating a task execution method provided in another exemplary embodiment of this application.

[0025] Figure 8The diagram shown is a schematic flowchart of a model training apparatus provided in an exemplary embodiment of this application.

[0026] Figure 9 The diagram shown is a schematic flowchart of a model training apparatus provided in another exemplary embodiment of this application.

[0027] Figure 10 The diagram shown is a flowchart of a task execution device provided in an exemplary embodiment of this application.

[0028] Figure 11 The diagram shown is a flowchart of a task execution apparatus provided in another exemplary embodiment of this application.

[0029] Figure 12 The diagram shown is a block diagram of an electronic device for performing a model training method or a task execution method according to an exemplary embodiment of this application. Detailed Implementation

[0030] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] Application Overview

[0032] Before practical use, a model is typically trained using sample data. The trained model is then tested. If the test results show that the model's accuracy does not meet the requirements, training continues, and the model is tested again. This process is repeated until the trained model achieves the required accuracy. A model that meets the accuracy requirements can then be deployed in practical applications.

[0033] Specifically, for scenarios where robots use models deployed on the robots to perform target tasks, sample data collection is required before training the model. Generally, factories that need to use robots to perform target tasks (such as grasping target objects) can propose sample data collection tasks. The data collection factory can break down the collection task and collect data according to certain standards. For example, a data collector can control the robot to perform the target task through a remote control, collecting sample data, such as image data, during the task execution. The sample data is then uploaded to the cloud and used to train the model. After downloading the trained model, the factory can test it. For example, the trained model can be deployed on a robot to perform the target task, and the performance of the target task can be verified to obtain the test results. If the performance is not good, the data collection, training, and testing processes need to be repeated, or the collection task can be replanned and the data collection, training, and testing processes can be repeated until a model with good test results is obtained.

[0034] This shows that the current model training process, especially the training process for models used by robots to perform tasks, has the problem of being too long.

[0035] To address the aforementioned technical problems, this application provides a model training method. This method involves testing a task execution model by executing a test task using the model, thereby obtaining test data corresponding to the target time period during the task execution. If the test results do not meet preset requirements, such as indicating an error during task execution, the test data is corrected to obtain corrected data. This corrected data is then used to train the task execution model, resulting in a trained model. This approach allows for focus on aspects of the task execution model that perform poorly during testing; for example, correcting test data when results do not meet preset requirements. Further training the model using the corrected data can rapidly improve convergence, reduce the overall training cycle, and increase training efficiency.

[0036] Exemplary System

[0037] Figure 1 The diagram shown is a schematic representation of the system architecture of a model training system provided in an exemplary embodiment of this application. Figure 1 As shown, the model training system 100 may include a computing device 110 and a task execution model 120 deployed on the computing device 110.

[0038] In one example, computing device 110 can be a server or an electronic device. The server can be a physical server, a cloud server, a virtual server, or other types of server. The electronic device can include computers, mobile phones, tablets, or mobile devices, and mobile devices can include robots or vehicles. The robot can be a humanoid robot, a wheeled robot, a tracked robot, or other types of robot.

[0039] In one example, computing device 110 can use task execution model 120 to execute a test task and obtain test data corresponding to the target time period during the execution of the test task. If the test results corresponding to the test data do not meet the preset requirements, the test data can be corrected by computing device 110, by technicians, or by other devices to obtain corrected data. Computing device 110 can use the corrected data to train task execution model 120 to obtain a trained task execution model.

[0040] Furthermore, the computing device 110 can repeatedly execute the process of testing the task execution model 120. That is, the computing device 110 can use the task execution model 120 to execute the test task again and obtain the test data corresponding to the target time period during the execution of the test task. If the test results corresponding to the test data do not meet the preset requirements, the test data can be corrected to obtain corrected data. The computing device 110 can use the corrected data to train the task execution model again to obtain the trained task execution model. This process is repeated until the trained task execution model meets the training cutoff requirements, such as meeting the accuracy requirements.

[0041] In one example, the test task in each test process can be the same, such as grabbing the target object. However, the target object corresponding to the test task in each test process can be different. The target time period corresponding to the test result that does not meet the preset requirements in each test process can be the same or different. Similarly, the test data corresponding to the target time period in each test process can also be the same or different.

[0042] Optionally, when the test task has not been fully executed, after correcting the test data corresponding to the target time period, the task execution model can continue to execute subsequent stages of the test task. During the execution of subsequent stages, test data corresponding to those stages can also be obtained, and it can be determined whether the test results of the test data for those stages meet preset requirements. If the test results of the test data for those stages do not meet the preset requirements, the test data can be corrected to obtain corrected data, which can then be used to train the task execution model.

[0043] In one example, once the trained task execution model meets the training deadline, it can be applied to the actual job process.

[0044] It should be understood that the above application scenario examples are only shown to facilitate understanding of the spirit and principles of this application, and the embodiments of this application are not limited thereto. Rather, the embodiments of this application can be applied to any applicable scenario.

[0045] Exemplary methods

[0046] Figure 2 The diagram shown is a flowchart illustrating a model training method provided in an exemplary embodiment of this application. Figure 2 The method can be derived from Figure 1 The computing device 110 in the middle performs the operation. For example... Figure 2 As shown, the model training method may include the following.

[0047] 210: Execute test tasks using the task execution model to obtain test data corresponding to the target time period during the execution of the test tasks.

[0048] Specifically, the task execution model may include at least one of the following: convolutional neural network model, recurrent neural network model, feedforward neural network model, generative adversarial network model, Transformer model, or other network model.

[0049] In one example, at the initial stage, the task execution model can be a model trained on a large amount of sample data; this model can be called a mature model, a warm model, or a hot model. Alternatively, at the initial stage, the task execution model can be an initial model that has not been fully trained or has only undergone basic pre-training; this initial model can be called a cold model, or a cold-start model. Alternatively, at the initial stage, the task execution model can be an intermediate model that has undergone preliminary training or fine-tuning with limited data; this intermediate model can be called a mild transition model, or a mild transition model.

[0050] It should be understood that the type of test task refers to the type of task that the task execution model is expected to perform. A test task is a task performed using the task execution model during the testing process. In one example, the test task, or the task that the task execution model is expected to perform, can be a static task such as image recognition or speech recognition, or a dynamic task such as grasping a target object or cleaning a target area. When the test task is dynamic, the computing device can be a mobile device.

[0051] In one example, test data can be acquired during the execution of a test task. This test data may include input and output data of the task execution model within a target time period. For example, input data may include image data, and output data may include recognition results. Alternatively, test data can be acquired at a certain frequency during the execution of the test task, with each acquisition covering a specific time period. In one example, the target time period may be the entire duration of the test task execution, a specific time period within the test task, or the current time period within the test task.

[0052] 220: When the test results corresponding to the test data do not meet the preset requirements, the test data is corrected to obtain corrected data.

[0053] In one example, the preset requirements can be set according to actual needs. These preset requirements characterize the performance of the task execution model expected by the user, or the execution effect of the task, or the absence of errors in the execution process of the test task during the target time period. Correspondingly, the test results characterize the execution effect of the task execution model on the test task during the target time period, or whether errors occurred during the execution process of the test task during the target time period. In one example, the preset requirements may include at least one of the following: the target action in the test task is successfully completed during the target time period; the execution time range of at least one action in the test task during the target time period; no unexpected events occur during the execution process of the test task during the target time period, etc. The test results can characterize at least one of the following: whether the target action in the test task is successfully completed during the target time period; the execution time length of at least one action in the test task during the target time period; whether unexpected events occur during the execution process of the test task during the target time period, etc. When the test results meet the preset requirements, it indicates that no errors occurred during the execution process of the test task during the target time period.

[0054] In one example, the test result characterizes whether the task execution model encountered any errors while performing the test task within the target time period. For instance, if the above identification result is "no error," meaning correct or successful, the test result meets the preset requirements; otherwise, it does not. In one example, whether the above identification result indicates an error can be determined by a technical personnel.

[0055] If the test results do not meet the preset requirements, the test data is corrected to obtain corrected data. In one example, the output data in the test data can be corrected to obtain corrected output data, which can be used as annotations for the input data. The input data plus the corrected output data can be used as corrected data.

[0056] In one example, when the test task is an image recognition task, the image recognition task may include multiple stages, each corresponding to specific input and output data. During the execution of the image recognition task, if the test results corresponding to the test data for the target time period (such as the input and output data for the target time period) do not meet the preset requirements, the test data can be corrected.

[0057] 230: Use the corrected data to train the task execution model to obtain the trained task execution model.

[0058] Specifically, the corrected data can be used as training data to train the task execution model, resulting in the trained task execution model.

[0059] In one example, steps 210 and 220 can be repeated to obtain a certain amount of corrected data. This corrected data can then be used to train the task execution model, resulting in a trained task execution model. The trained task execution model can achieve high accuracy in subsequent task executions.

[0060] This application provides a model training method. It involves testing a task execution model by executing a test task using the model, obtaining test data corresponding to the target time period during the task execution. If the test results do not meet preset requirements, such as indicating an error during task execution, the test data is corrected to obtain corrected data. This corrected data is then used to train the task execution model, resulting in a trained model. This approach allows for focus on aspects of the task execution model that perform poorly during testing; by correcting test data when results do not meet preset requirements, and further training the model using the corrected data, the model's convergence performance can be rapidly improved, the overall training cycle reduced, and training efficiency increased.

[0061] According to one embodiment of this application, a task execution model is deployed on a robot, and the test data includes real-time perception data collected by the robot during the execution of a test task based on the task execution model. Step 210 may include: controlling the robot to collect perception data in real time, using the task execution model to obtain motion control commands based on the perception data at the current moment, and controlling the robot to move based on the motion control commands to execute the test task.

[0062] Specifically, the computing device can be a robot, which performs tasks through a task execution model deployed on it. In one example, the robot can acquire perception data in real time, and the perception data at the current moment can be used as input data for the task execution model. In one example, the perception data may include image data and / or data representing the pose of some joints on the robot. For example, the robot can acquire image data through image acquisition sensors installed on it. The image data can reflect the robot's surrounding environment at the current moment, such as the robot's current pose and the poses of objects around the robot. Furthermore, the robot can acquire data representing the pose of some joints on it, such as joint positions and joint angles, through an inertial measurement unit installed on it.

[0063] In one example, the task execution model can output motion control commands based on perception data. These commands may include data representing the pose of the robot's upper joints at future moments, such as joint position and angle. In some scenarios, the motion control commands may also include parameters such as the opening and closing degree of the robot arm and the robot's pitch and rise parameters.

[0064] In one example, motion control algorithms, such as filtering, collision detection, and admittance compliance algorithms, can be used to smooth the motion control commands, resulting in processed motion control commands. Controlling the robot's movement based on these processed commands can improve the smoothness of the robot's actions.

[0065] Optionally, the task execution model can obtain motion control commands based on perception data, smooth the motion control commands, and then output the smoothed motion control commands.

[0066] In one example, while the robot is controlling its movement based on motion control commands or smoothed motion control commands, it still collects sensory data in real time. This sensory data (such as image data, or image data combined with data representing the poses of the robot's upper joints) can serve as a direct reflection of the motion control commands or smoothed motion control commands in the robot's actions. This sensory data can then be used as indirect output data from the task execution model.

[0067] In this embodiment, the test data may include the input data of the task execution model during the target time period and the corresponding indirect output data.

[0068] In one example, if the test results do not meet preset requirements, the indirect output data in the test data can be corrected to obtain corrected output data, which can then be used as annotations for the input data. Input data plus the corrected output data constitutes the corrected data. For example, input data may include image data, and indirect output data may also include image data. The entire set of input and indirect output data can include multiple frames of images continuously acquired by the robot. In one example, the indirect output data can be corrected using an algorithm to obtain corrected output data. The corrected output data reflects the robot's successful execution of the target stage within the target time period, i.e., actions that were not successfully executed within the target time period, such as picking up the target object.

[0069] In this embodiment, if the test results do not meet the preset requirements, the test data is corrected to obtain corrected data, and the corrected data is used to train the task execution model. This allows attention to the parts of the task execution model that do not perform well during the test, thereby quickly improving the model's convergence effect, reducing the overall training cycle of the model, and improving training efficiency, that is, improving the robot's task learning efficiency.

[0070] According to one embodiment of this application, the target time period includes the current time period, and the preset requirement includes that the robot's execution process for the test task during the current time period is error-free. Step 220 may include: controlling the robot to execute the corresponding target stage in the current time period via teleoperation to obtain test data corresponding to the teleoperation, thereby correcting the test data. The test data corresponding to the teleoperation is corrected data, which includes sensory data collected by the robot during the teleoperation process. Further, the model training method also includes: controlling the robot to continue executing stages in the test task following the target stage based on the task execution model.

[0071] Specifically, teleoperation can include controlling robot movements via a handle, remote control, or other means. During the robot's execution of a test task based on a task execution model, if the test results do not meet preset requirements (i.e., the robot malfunctions), teleoperation can be used to control the robot to continue executing the test task. In one example, teleoperation can control the robot to execute the target segment corresponding to the current time period, i.e., the segment where the error occurred. In other words, teleoperation is used instead of analyzing input data using a task execution model to control the robot to execute the target joint in the test task. While the technician is controlling the robot to execute the target joint using teleoperation, the robot can continue to collect sensory data in real time; this sensory data is the test data corresponding to the teleoperation. Teleoperation can compensate for robot errors; therefore, obtaining the corresponding test data through teleoperation is equivalent to correcting the robot's errors, meaning the test data corresponding to the teleoperation can serve as corrective data.

[0072] Furthermore, after successfully executing the target stage via teleoperation, the task execution model can be used to control the robot to execute subsequent stages in the test task. During the execution of these subsequent stages, test data can be acquired, and the results can be assessed to determine if they meet preset requirements. If the test results for subsequent stages do not meet the preset requirements, the test data can be corrected to obtain corrected data, which can then be used to train the task execution model.

[0073] In one example, a test task may include one or more stages, each stage including one step. For example, if a test task includes multiple steps, the test task may be making a sandwich, which may include: Step 1, picking up a slice of bread from the table and placing it on the work surface; Step 2, picking up a slice of meat from the table and placing it on the slice of bread; Step 3, picking up another slice of bread from the table and placing it on the slice of meat, thus obtaining a sandwich.

[0074] The robot first performs a test task of making sandwiches based on a task execution model. In one scenario, during step 1, the robot picks up a slice of bread, but it drops it on the floor instead of placing it on the worktable. This indicates that step 1 of the test task has failed, or that the robot made an error while performing step 1. In this case, manual intervention is possible. A technician can remotely control the robot to pick up another slice of bread from the table and place it on the worktable, thus completing step 1. In one example, manual intervention can be achieved using a first operation key (such as a button) on the robot. For example, pressing the first operation key stops the task execution model's control process and begins the remote operation control process. After completing step 1, the remote operation can be canceled, and the robot can continue to autonomously execute subsequent steps, namely, steps 2 and 3, based on the task execution model. In one example, canceling the remote operation can be achieved using a second operation key (such as a button) on the robot. For example, pressing the second operation key stops the remote operation control process and begins the task execution model's control process. In one example, the first and second operation keys can be the same or different operation keys.

[0075] like Figure 3aAs shown, the robot autonomously executes the task execution model from time t0 to t1. An error occurs at time t2, at which point step 1 is not completed. At time t3, a technician takes over, and the robot can revert to a certain position, such as from the position corresponding to time t3, returning to the position corresponding to time t2 at time t2', and then returning to the position corresponding to time t1 at time t1'. For example, the position at time t1' can be the same as the position at time t1, and time t1 can be a time before the robot malfunctions. After returning to the position at time t1, the technician remotely controls the robot to continue executing the test task, i.e., through times t1', t2'', t3', and t4, the technician controls the robot to correctly complete step 1. At time t5, the test task is returned to the task execution model for execution until an error occurs, at which point the technician takes over again. Here, the perceived data from time t3' to t5 (excluding time t5) can be used as correction data, or the perceived data from time t1' to t5 (excluding time t5) can be used as correction data, as long as the correction data covers the malfunctioning step (step 1).

[0076] In another scenario, during step 1, the robot fails to pick up the bread slice; it remains on the table and is not placed on the worktable. This indicates that step 1 has failed, or that the robot encountered an error while performing step 1. In this case, manual intervention is possible. A technician can remotely control the robot to pick up a bread slice from the table and place it on the worktable, thus completing step 1. After completing step 1, the remote operation can be canceled, and the robot can continue to perform subsequent steps autonomously.

[0077] like Figure 3b As shown, the robot autonomously executes the task execution model from time t0 to t1. An error occurs at time t2, at which point step 1 is not completed. At time t3, a technician takes over, directly controlling the robot to continue the test task via teleoperation. That is, after times t4-t5, the technician controls the robot to correctly complete step 1. At time t6, the test task is returned to the task execution model until an error occurs again, at which point the technician takes over again. Here, the perceived data from times t3 to t6 (excluding time t6) can be used as correction data to ensure that the correction data covers the erroneous step (step 1).

[0078] In this embodiment, if the test results do not meet the preset requirements, the robot continues to execute the test task through remote control. This allows for the correction of the steps that do not meet the preset requirements, i.e., correct execution, thereby obtaining corrected data. The corrected data can be used to train the task execution model, improving its correct execution of the steps corresponding to the corrected data. This strengthens the weak points of the task execution model, rapidly improving its convergence, reducing the overall training cycle, and increasing training efficiency.

[0079] According to one embodiment of this application, the model training method further includes: inputting test data into a state judgment model to obtain a state judgment result, wherein the state judgment result serves as a test result to characterize whether the robot has made an error during the execution of the test task in the current time period; and issuing a reminder message if the test result does not meet the preset requirements.

[0080] Specifically, a state judgment model can be used to replace technicians in judging the test results corresponding to the test data.

[0081] In one example, test data is input into a state judgment model, which then outputs a state judgment result. This result can be used to characterize whether the robot encounters situations that fail to meet preset requirements during the execution of a test task.

[0082] In one example, the state judgment result can be used to characterize whether the robot made an error during the execution of the test task; that is, it is used to characterize whether an error occurred during the execution of the test task within the target time period. This state judgment result can be used as the test result.

[0083] Furthermore, if the test results do not meet the preset requirements, such as when the test results indicate that the robot has made an error during the execution of the test task, a reminder message will be issued. The reminder message may include at least one of the following: text information, voice information, light information, etc. The reminder message is used to alert technicians that the task execution model has malfunctioned and requires manual intervention.

[0084] In one example, the state decision model may include at least one of a convolutional neural network model, a recurrent neural network model, a feedforward neural network model, a generative adversarial network model, a Transformer model, or other network models.

[0085] In this embodiment, using a state judgment model to replace technicians in judging the test results corresponding to the test data can improve the training efficiency of the entire training process and reduce costs.

[0086] According to an embodiment of this application, the model training method further includes: storing the test data as positive sample data in a positive sample database when the test results meet preset requirements; storing the test data as negative sample data in a negative sample database when the test results do not meet preset requirements, and storing the correction data as positive sample data in a positive sample database; training the initial state judgment model using the positive sample data in the positive sample database and the negative sample data in the negative sample database to obtain a state judgment model, wherein the state judgment model is used to obtain a state judgment result based on the test data, and the state judgment result is used as the test result.

[0087] Specifically, for a target time period, if the test results meet the preset requirements, the test data at this time can be used as positive sample data; if the test results do not meet the preset requirements, the test data at this time can be used as negative sample data, and the corresponding correction data can be used as positive sample data. The initial state judgment model is trained using positive and negative sample data to obtain the state judgment model. During the execution of a task (such as a test task) using the task execution model, the state judgment model can judge the execution state of the task execution model based on the test data (such as image data) to obtain the state judgment result. The relevant content of the state judgment result can be found in the description in the above embodiments; to avoid repetition, it will not be repeated here.

[0088] In one example, the preset requirement may include that the robot performs the test task without errors during the target time period. Further, if the test results meet the preset requirement, it indicates that the task execution model did not make any errors during the target time period, and the test data in this case can be used as positive sample data; if the test results do not meet the preset requirement, it indicates that the task execution model made errors during the target time period, and the test data in this case can be used as negative sample data.

[0089] In this embodiment, during the execution of the test task, test data can be collected at a certain frequency to obtain a large amount of test data. During the collection of test data for any target time period, test data whose results do not meet preset requirements can be corrected to obtain corrected data. Thus, positive and negative sample data can be collected in real time during the testing of the task execution model, improving data collection efficiency. Furthermore, the collected positive and negative sample data can be used to train the initial state judgment model. The training process of this initial state judgment model can be performed synchronously with the process of executing the test task through the task execution model, thereby improving the training efficiency of the initial state judgment model. Of course, the training process of the initial state judgment model can also be performed at other times, such as during the test task execution process through the task execution model.

[0090] According to one embodiment of this application, the model training method further includes: inputting test data into an initial state judgment model to obtain an initial state judgment result, wherein the initial state judgment result is used to characterize whether the execution process of the test task during the target time period is incorrect, as determined by the initial state judgment model, and the test result is used to characterize whether the execution process of the test task during the target time period is actually incorrect; if the test result meets the preset requirements and the initial state judgment result is inconsistent with the test result, the test data is stored as positive sample data in a positive sample database; if the test result does not meet the preset requirements and the initial state judgment result is inconsistent with the test result, the test data is stored as negative sample data in a negative sample database; the initial state judgment model is trained using the positive sample data in the positive sample database and the negative sample data in the negative sample database to obtain a state judgment model, the state judgment model is used to obtain a state judgment result based on the test data, and the state judgment result is used as the test result to characterize whether an error occurs during the execution process of the test task.

[0091] Specifically, when a test task is executed by a robot using a task execution model, the initial state judgment result can be used to characterize whether the robot made an error during the execution of the test task, i.e., whether an error occurred within the target time period. The initial state judgment result is obtained based on the initial state judgment model. The test result can be used to characterize whether the robot actually made an error during the execution of the test task, i.e., whether an error actually occurred within the target time period. The test result can be determined by technicians or by a specific algorithm.

[0092] In one example, the accuracy of the test result is greater than the accuracy of the initial state judgment result. Therefore, when the test result is inconsistent with the initial state judgment result, the test result can be taken as the standard.

[0093] For example, if the test results meet the preset requirements, but the initial state judgment result is inconsistent with the test results, it indicates that the task execution model did not err during the target time period, but the initial state judgment model's judgment regarding the target time period was inaccurate. In this case, the test data can be stored as positive sample data in the positive sample database. Conversely, if the test results do not meet the preset requirements, and the initial state judgment result is inconsistent with the test results, it indicates that the task execution model erred during the target time period, but the initial state judgment model's judgment regarding the target time period was inaccurate. In this case, the test data can be stored as negative sample data in the negative sample database.

[0094] Furthermore, if the test results meet the preset requirements and the initial state judgment result is consistent with the test results, it indicates that the task execution model did not err during the target time period, and the initial state judgment model's judgment for the target time period is accurate. In this case, it is not necessary to store the test data in the database. If the test results do not meet the preset requirements, but the initial state judgment result is consistent with the test results, it indicates that the task execution model erred during the target time period, and the initial state judgment model's judgment for the target time period is accurate. In this case, it is not necessary to store the test data in the database.

[0095] After the initial state judgment model is trained, the state judgment model can be obtained. Subsequently, the state judgment model can be directly used to judge the execution state of the task execution model based on the test data. That is, the state judgment result output by the state judgment model can be used as the test result. In this way, the model can be used directly to judge whether the task execution model has made a mistake in the relevant period.

[0096] In one example, the initial state determination model could be a Cold model, a Mild model, or a Warm model.

[0097] In this embodiment, the test data when the initial state judgment result is inconsistent with the test result is used as sample data, and the initial state judgment model is trained using this sample data. The inaccurate parts of the initial state judgment model can be trained in a focused manner, which can improve the model convergence effect and training efficiency.

[0098] According to one embodiment of this application, the task execution model and the state judgment model belong to the same model. For example, the task execution model and the state judgment model can be different parts of the same model, or the task execution model and the state judgment model have at least partial overlap in the same model.

[0099] In one example, the same model can be a large visual-language-action model, and further, the large visual-language-action model can have autonomous state judgment capabilities. Of course, the same model can be other models that can implement the functions of the method embodiments of this application.

[0100] In this embodiment, integrating the task execution model and the state judgment model into one model can improve the model's convergence effect and training efficiency.

[0101] Figure 4a The diagram shown is a flowchart of a model training method provided in another exemplary embodiment of this application. Figure 4a The example is Figure 2 Examples of the embodiments are provided below; to avoid repetition, the similarities can be referred to the descriptions in the above embodiments, and will not be repeated here. For example... Figure 4a As shown, the model training method may include the following.

[0102] 410: Train the initial model using the initial sample data to obtain the task execution model.

[0103] Specifically, the initial model can be a Cold model, which can be trained using a small amount of initial sample data to obtain the task execution model. In one example, the task execution model could be a Warm model.

[0104] In one example, initial sample data can be acquired through teleoperation or other means. For instance, a technician can remotely control a robot to perform a target task, and the sensory data obtained during the execution of the target task can serve as initial sample data.

[0105] 420: Execute test tasks using the task execution model and obtain test data corresponding to the current time period during the execution of the test tasks.

[0106] 430: If the test results corresponding to the test data do not meet the preset requirements, the test data is corrected to obtain corrected data. The test data is stored as the first negative sample data in the negative sample database, and the corrected data is stored as the first positive sample data in the positive sample database.

[0107] 440: If the test results corresponding to the test data meet the preset requirements, the test data can be stored in the positive sample database as the second positive sample data.

[0108] In one example, a technician can supervise the robot as it executes a test task using the task execution model. If the robot's performance falls short of expectations during the task execution process—that is, if the test results for the current time period do not meet the preset requirements—the technician can take over directly.

[0109] Furthermore, the takeover process for technical personnel can be found in the appendix. Figure 3a Or attached Figure 3b Test data obtained during the takeover process by technical personnel can be used as corrective data.

[0110] In one example, such as Figure 4b As shown, the task execution model executes the test task. The process of executing the test task can be called the testing process or the inference process. When the task execution model is error-free (normal), the test data for the corresponding time period can be stored in the positive sample database. When the task execution model malfunctions, the test data for the corresponding time period can be stored in the negative sample database. The test data is then corrected by manual intervention to obtain corrected data, which is then stored in the positive sample database.

[0111] In one example, steps 430 and 440 may be executed repeatedly as the test task progresses. In one example, the execution order of steps 430 and 440 is not important. In one example, step 440 may be omitted.

[0112] 450: The task execution model is trained using the corrected data in the positive sample database to obtain the trained task execution model.

[0113] In one example, the task execution model can be trained using the corrected data in the positive sample database and the first negative sample data in the negative sample database to obtain the trained task execution model.

[0114] 460: Input the test data into the initial state judgment model to obtain the initial state judgment result.

[0115] 470: If the test results meet the preset requirements and the initial state judgment results are inconsistent with the test results, the test data will be stored in the positive sample database as the second positive sample data. If the test results do not meet the preset requirements and the initial state judgment results are inconsistent with the test results, the test data will be stored in the negative sample database as the second negative sample data.

[0116] In one example, the initial state judgment result is used to characterize whether the test results obtained by the initial state judgment model during the execution of the test task meet the preset requirements, and the test results are used to characterize whether the robot actually meets the preset requirements during the execution of the test task.

[0117] In one example, such as Figure 4b As shown, test data collected in real time during the execution of the test task by the task execution model can be synchronously input into the initial state judgment model to obtain the initial state judgment result. When the actual test result meets the preset requirements, i.e., the task execution model is normal (no error), if the initial state judgment result indicates that the task execution model is wrong, it means that the initial state judgment model has made an error. In this case, the test data can be stored as the second positive sample data in the positive sample database. When the actual test result does not meet the preset requirements, i.e., the task execution model is wrong, if the initial state judgment result indicates that the task execution model is not wrong (normal), it means that the initial state judgment model has made an error. In this case, the test data can be stored as the second negative sample data in the negative sample database.

[0118] 480: The initial state judgment model is trained using the second positive sample data in the positive sample database and the second negative sample data in the negative sample database to obtain the state judgment model.

[0119] The state judgment model is used to obtain state judgment results based on test data. In one example, the state judgment result can serve as the test result, characterizing whether the robot made an error during the execution of the test task.

[0120] In one example, the initial state judgment model can be trained based on a small amount of sample data. For instance, the initial state judgment model can be trained on the base model using the first positive sample data and the first negative sample data. Alternatively, the initial state judgment model can be trained on the base model using other sample data, as long as the sample data enables the initial state judgment model to output the initial state judgment result.

[0121] In one example, the execution order of steps 430 and 460 is not important.

[0122] In one example, steps 430-480 can be repeated until the accuracy of the task execution model and the state judgment model meets the preset accuracy requirements.

[0123] It should be understood that the execution order of the above steps can be adjusted according to actual needs.

[0124] like Figure 5 As shown in the embodiments of this application, a model training method is also provided. Figure 5 The method can be derived from Figure 1 The computing device 110 in the middle performs the operation. For example... Figure 5 As shown, the model training method may include the following.

[0125] 510: Execute test tasks using the task execution model to obtain test data corresponding to the target time period during the execution of the test tasks.

[0126] Specifically, the relevant content regarding the task execution model, test task, target time period, and test data can be found in the descriptions in the above embodiments. To avoid repetition, these details will not be repeated here.

[0127] 520: Input the test data into the initial state judgment model to obtain the initial state judgment result. The initial state judgment result is used to characterize whether the execution process of the test task during the target time period is incorrect, as determined by the initial state judgment model.

[0128] 530: If the test results corresponding to the test data meet the preset requirements, and the initial state judgment result is inconsistent with the test result, the test data will be stored as positive sample data in the positive sample database.

[0129] The test results are used to characterize whether errors actually occurred during the execution of the test task at the target time period.

[0130] 540: If the test results do not meet the preset requirements and the initial state judgment results are inconsistent with the test results, the test data will be stored as negative sample data in the negative sample database.

[0131] Specifically, the initial state judgment model, initial state judgment result, test result, and preset requirements can be referred to the relevant descriptions in the above embodiments. To avoid repetition, they will not be repeated here.

[0132] 550: The initial state judgment model is trained using positive sample data from the positive sample database and negative sample data from the negative sample database to obtain the state judgment model.

[0133] The state judgment model is used to obtain state judgment results based on test data. The state judgment results serve as test results and are used to characterize whether errors occurred during the execution of the test task.

[0134] In one example, Figure 5 The method can be performed by a robot.

[0135] In one example, the task execution model can be a pre-trained model, in which case the test task can be an actual job task performed using the task execution model. Alternatively, the task execution model can be a model under training, in which case the test task can be an actual job task performed using the task execution model, a sample task used to train the task execution model, or a detection task used to test the performance of the task execution model.

[0136] In this embodiment, the test data when the initial state judgment result is inconsistent with the test result is used as sample data, and the initial state judgment model is trained using this sample data. The inaccurate parts of the initial state judgment model can be trained in a focused manner, which can improve the model convergence effect and training efficiency.

[0137] According to one embodiment of this application, the model training method further includes: correcting the test data to obtain corrected data when the test results corresponding to the test data do not meet the preset requirements; and using the corrected data to train the task execution model to obtain the trained task execution model.

[0138] Specifically, the details of correcting the data, correcting the test data to obtain corrected data, and using the corrected data to train the task execution model, as well as the corresponding effects, can be found in the relevant descriptions in the above embodiments. To avoid repetition, they will not be repeated here.

[0139] According to one embodiment of this application, a task execution model is deployed on a robot, and the test data includes real-time perception data collected by the robot during the execution of a test task based on the task execution model. Step 510 may include: controlling the robot to collect perception data in real time, using the task execution model to obtain motion control commands based on the perception data at the current moment, and controlling the robot to move based on the motion control commands to execute the test task.

[0140] Specifically, the details of the perception data, motion control commands, and the robot's execution of the test task using the task execution model, as well as the corresponding effects, can be found in the relevant descriptions in the above embodiments. To avoid repetition, they will not be repeated here.

[0141] According to one embodiment of this application, the target time period includes the current time period, and the preset requirement includes that the robot does not make any errors in the execution process of the test task during the current time period. Correcting the test data to obtain corrected data includes: controlling the robot to execute the corresponding target step in the current time period via teleoperation to obtain test data corresponding to the teleoperation, thereby correcting the test data. The test data corresponding to the teleoperation is the corrected data, which includes the sensory data collected by the robot during the teleoperation process. Further, the model training method also includes: controlling the robot to continue executing the steps in the test task following the target step based on the task execution model.

[0142] Specifically, the details of remote operation, target stage, controlling the robot to execute the target stage through remote operation, subsequent stages, and the specific content and effects of the test data corresponding to the remote operation can be found in the relevant descriptions in the above embodiments. To avoid repetition, they will not be repeated here.

[0143] According to one embodiment of this application, the perception data collected in real time during the process of the robot performing a test task based on the task execution model includes: image data and / or data representing the pose of the upper joints of the robot.

[0144] According to one embodiment of this application, the task execution model and the state judgment model belong to the same model.

[0145] like Figure 6 As shown in the embodiments of this application, a task execution method is also provided. Figure 6 The method can be derived from Figure 1 The computing device 110 in the middle performs the operation. For example... Figure 6 As shown, the task execution method may include the following:

[0146] 610: Obtain input data.

[0147] 620: Utilize the task execution model to execute the target task based on the input data.

[0148] Specifically, the task execution model is obtained according to the model training method provided in any of the above embodiments. For example, it can be based on... Figure 2 or Figure 4a The model training method provided in the embodiment is used to obtain the task execution model.

[0149] In one example, when the target task is static, it can be an image recognition task, a speech recognition task, or other static tasks. Taking an image recognition task as an example, the input data can include image data, and the output data can include a numerical value indicating whether there is a target object in the image data. For example, an output value of 1 indicates that there is a target object in the image data, while an output value of 0 indicates that there is no target object in the image data.

[0150] In one example, when the target task is dynamic, it could be grasping a target object, cleaning a target area, or other dynamic tasks. Taking the task of grasping a target object as an example, the executor of this task could be a mobile device, such as a robot, on which a task execution model could be deployed. The robot can use the task execution model to perform the task of grasping the target object. Specifically, the input data can include sensory data collected by the robot. The specific content of the sensory data can be found in the description in the above embodiments, and will not be repeated here to avoid repetition. The sensory data is input into the task execution model, and the direct output data of the task execution model can include motion control commands. The robot controls its movement based on the motion control commands to perform the task of grasping the target object. Here, while the robot is controlling its movement based on the motion control commands, it will still collect sensory data in real time. This sensory data can serve as a direct reflection of the motion control commands on the robot's actions. Therefore, this sensory data can serve as indirect output data of the task execution model.

[0151] It should be understood that the task execution model is obtained based on the model training method provided in any of the above embodiments. Therefore, the relevant content and corresponding effects of using the task execution model to execute the target task can be found in the description in the above embodiments. To avoid repetition, it will not be repeated here.

[0152] like Figure 7 As shown in the embodiments of this application, a task execution method is also provided. Figure 7 The method can be derived from Figure 1 The computing device 110 in the middle performs the operation. For example... Figure 7 As shown, the task execution method may include the following:

[0153] 710: Utilize the task execution model to execute the target task and obtain the execution data corresponding to the target time period during the execution of the target task.

[0154] 720: Input the execution data into the state judgment model to obtain the state judgment result, whereby the state judgment result is used to characterize whether the execution result corresponding to the execution data meets the preset requirements.

[0155] 730: If the execution result corresponding to the execution data does not meet the preset requirements, issue a reminder message.

[0156] Specifically, the relevant content of the task execution model, target task, target time period, status judgment model, status judgment result, reminder information, and preset requirements can be referred to the relevant descriptions in the above embodiments. To avoid repetition, they will not be repeated here.

[0157] In one example, the execution data corresponds to the test data described above and may include the input and output data of the task execution model during the target time period. For example, the input data may include image data, and the output data may include recognition results. For instance, execution data during the target task execution process can be acquired at a certain frequency, with each acquisition covering a specific time period. In one example, the target time period may be the entire time period for executing the target task, a specific time period within the target task, or the current time period within the target task.

[0158] In one example, the execution results corresponding to the execution data correspond to the test results mentioned above. The execution results corresponding to the execution data characterize the performance of the task execution model for the target task within the target time period, or in other words, whether errors occurred during the execution process for the target task within the target time period. In one example, the preset requirements may include at least one of the following: the target action in the target task is successfully completed within the target time period; the execution time range of at least one action in the target task within the target time period; no unexpected events occur during the execution process of the target task within the target time period, etc. The execution results can characterize at least one of the following: whether the target action in the target task is successfully completed within the target time period; the execution time range of at least one action in the target task within the target time period; whether unexpected events occur during the execution process of the target task within the target time period, etc. When the execution results meet the preset requirements, it indicates that no errors occurred during the execution process for the target task within the target time period.

[0159] In one example, when the target task can be a test task, the corresponding execution data can be test data, and the execution result can be the test result. The test task can be an actual job task executed using the task execution model, a sample task used to train the task execution model, or a detection task used to test the effectiveness of the task execution model.

[0160] In one example, Figure 7 The method can be performed by a robot, and the execution data may include the perception data collected by the robot.

[0161] In other examples, Figure 7 The method can be executed by other electronic devices, and the task execution model can be an image recognition model.

[0162] In this embodiment, a state judgment model can be used to replace technicians in judging the execution results, which can improve the efficiency of the entire task execution process and reduce costs.

[0163] According to one embodiment of this application, the state determination model can be based on... Figure 5 The model training method described in this embodiment allows for focused training on the parts of the state judgment model where judgments are inaccurate, thereby improving model convergence and training efficiency. Furthermore, collecting sample data for training the state judgment model during the task execution model's test task improves sample data acquisition efficiency and further shortens the overall training cycle of the state judgment model.

[0164] In one example, the process by which the state judgment model obtains the state judgment result based on the execution data can be carried out simultaneously with the process by which the task execution model executes the target task (such as obtaining execution data), or the execution data can be collected first, and then the process by which the state judgment model obtains the state judgment result based on the execution data can be executed later.

[0165] According to one embodiment of this application, a task execution model is deployed on a robot, and the execution data includes real-time perception data collected by the robot during the execution of a target task based on the task execution model. The target time period includes the current time period, and the preset requirement includes that the robot does not make any errors in the execution process of the target task during the current time period. The task execution method further includes: based on the reminder information, remotely controlling the robot to execute the corresponding target stage in the current time period; and controlling the robot to continue executing the stage after the target stage in the target task based on the task execution model.

[0166] Specifically, the details and uses of the sensing data and teleoperation can be found in the relevant descriptions in the above embodiments. The details of the target stage and the stages after the target stage can also be found in the relevant descriptions in the above embodiments. To avoid repetition, they will not be repeated here.

[0167] In one example, if the state judgment result indicates that the execution result corresponding to the execution data does not meet the preset requirements, it means that the robot has encountered an error in the execution process of the target task in the current time period. The computing device, such as the robot, can issue a reminder message to the technician to inform them of the robot's error. After receiving the reminder message, the technician can take over the execution process of the target task, such as by remotely controlling the robot to execute the corresponding target stage in the current time period, i.e., the stage where the error occurred.

[0168] Furthermore, after successfully executing the target stage via remote control, the robot can continue to execute subsequent stages of the target task using the task execution model. During the execution of these subsequent stages, execution data can be acquired, and a state judgment model can be used to determine whether the execution results of the subsequent stage's execution data meet preset requirements. If the execution results of the subsequent stage's execution data do not meet the preset requirements, a reminder message can be issued again to allow technical personnel to take over the task execution process.

[0169] In this embodiment, if the execution result does not meet the preset requirements, the robot can continue to execute the target task through remote operation control. This can correct the steps that do not meet the preset requirements, thus ensuring the completion of the target task while improving the execution efficiency and reducing costs.

[0170] According to one embodiment of this application, the task execution method further includes: using the execution data corresponding to the teleoperation as correction data to train the task execution model to obtain the trained task execution model, wherein the correction data includes the perception data collected by the robot during the teleoperation process.

[0171] Specifically, the execution data obtained by the robot based on the task execution model (the execution data corresponding to the target time period in step 710) can be corrected through teleoperation to obtain corrected data. The corrected data can be used to train the task execution model, which is equivalent to strengthening the weak links of the task execution model, thereby rapidly improving the convergence effect of the model, improving the accuracy of the task execution model, reducing the overall training cycle of the model, and improving training efficiency.

[0172] For details on correcting data and using corrected data to train the task execution model, please refer to the relevant descriptions in the above embodiments. To avoid repetition, they will not be repeated here.

[0173] The method provided in this application can improve the training efficiency of models (task execution model and state judgment model). For example, for the training process of a task execution model deployed on a robot, traditional training methods require a week or more to complete the training process. The entire training process may involve factory workers, technical managers, data collectors, auditors, testers, algorithm trainers, etc., resulting in a long training cycle. The method provided in this application can shorten the training time. In some cases, factory workers can teach robots new skills in half a day. For example, robots can be trained on the task execution model by collecting data from error-prone steps, improving the accuracy of the task execution model. Furthermore, the state judgment model can be used to detect the testing or inference process of the task execution model. It can issue reminder information when the task execution model malfunctions, thus freeing up manpower. That is, technicians do not need to stay next to the robot all the time and can take over the control of the robot to perform tasks when they receive reminder information. In addition, during the testing or inference process of the task execution model, data on errors made by the state judgment model can be collected. Using this data to further train the state judgment model can improve the convergence of the state judgment model and simplify the sample data collection process.

[0174] In this embodiment, the final model can be obtained by using a foundation model (such as a Cold or Warm model) and a warmer mechanism, thus enabling rapid convergence by collecting only key data. This significantly reduces the amount of data and speeds up training.

[0175] In this embodiment, the data can be stored locally to ensure data privacy and also save costs (without using a cloud platform).

[0176] This application provides a model training method that integrates sampling, inference (testing), and training, enabling functions such as triggering continued inference, taking over, pausing the robot, supporting automatic rollback of the robot to historical states, and supporting cross-entity operations.

[0177] In this embodiment, if the robot malfunctions while autonomously performing a test task, a technician can take over. In one example, during the technician's takeover, the robot can be controlled to return to its previous position by pressing and holding a specific operation key, allowing the robot to perturb its posture at that position to increase the distribution of robot data. Furthermore, during the takeover process, another operation key can be pressed again to start the teleoperation process, simultaneously triggering data logging to distinguish whether the data source is the robot or a human. Furthermore, after the technician's takeover process is completed, the operation key can be used to switch the robot to continue performing the task based on the task execution model, i.e., continue reasoning.

[0178] In this embodiment, a data collector or relevant technician can control or supervise multiple robots, thereby further improving data collection efficiency. Specifically, the state judgment model can provide autonomous state judgment (outputting state judgment results) functionality. When the autonomous state judgment model has a high success rate, the data collector does not need to constantly monitor each robot. When an autonomous state judgment fails, the system will request the data collector's intervention. Furthermore, the control screen can simultaneously access real-time images from multiple robots, and control signals can be switched between multiple robots. The process of determining whether technical personnel intervention is required can be determined autonomously by the technical personnel, or the robots can automatically determine whether technical personnel intervention is needed based on the robo-gated algorithm. Thus, the method provided in this embodiment can further improve human efficiency, further accelerate the collection of large amounts of data, and improve results.

[0179] Exemplary device

[0180] This application also provides a mobile device, which includes a control module for executing the model training method or task execution method provided in the above embodiments.

[0181] In one example, the mobile device may include a robot, a car, or other device capable of performing a task. The robot may be a humanoid robot, a wheeled robot, a tracked robot, or other types of robot.

[0182] The specific functions and effects of the robot provided in this application embodiment can be referred to the description in the above method embodiment. To avoid repetition, they will not be repeated here.

[0183] Figure 8 The diagram shown is a schematic flowchart of a model training apparatus provided in an exemplary embodiment of this application. Figure 8 As shown, the model training device 800 may include an execution module 810, a correction module 820, and a training module 830.

[0184] The execution module 810 is used to execute test tasks using the task execution model and obtain test data corresponding to the target time period during the execution of the test tasks. The correction module 820 is used to correct the test data to obtain corrected data when the test results corresponding to the test data do not meet the preset requirements. The training module 830 is used to train the task execution model using the corrected data to obtain the trained task execution model.

[0185] This application provides a model training device that tests a task execution model by executing a test task using the model, obtaining test data corresponding to the target time period during the task execution. If the test results do not meet preset requirements, such as indicating an error during task execution, the test data is corrected to obtain corrected data. This corrected data is then used to train the task execution model, resulting in a trained model. This allows for focus on aspects of the task execution model that perform poorly during testing; if test results do not meet preset requirements, the test data is corrected to obtain corrected data. Further training the model using the corrected data can rapidly improve model convergence, reduce the overall training cycle, and increase training efficiency.

[0186] According to one embodiment of this application, a task execution model is deployed on a robot, and the test data includes real-time perception data collected by the robot during the execution of a test task based on the task execution model. The execution module 810 is used to: control the robot to collect perception data in real time; utilize the task execution model to obtain motion control commands based on the perception data at the current moment; and control the robot's movement based on the motion control commands to execute the test task.

[0187] According to one embodiment of this application, the target time period includes the current time period, and the preset requirement includes that the robot's execution process for the test task during the current time period is error-free. The correction module 820 is used to: control the robot to execute the corresponding target stage in the current time period via teleoperation, obtain the test data corresponding to the teleoperation, and correct the test data. The test data corresponding to the teleoperation is the corrected data, which includes the perception data collected by the robot during the teleoperation process. The execution module 810 is further used to: control the robot to continue executing the stages of the test task following the target stage based on the task execution model.

[0188] According to one embodiment of this application, the model training device 800 further includes an input / output module 840, used for: inputting test data into the state judgment model to obtain a state judgment result, wherein the state judgment result serves as a test result to characterize whether the robot has made an error in the execution process of the test task in the current time period; and issuing a reminder message if the test result does not meet the preset requirements.

[0189] According to one embodiment of this application, the model training device 800 further includes a first storage module 850, configured to: store the test data as positive sample data in a positive sample database when the test results meet preset requirements; and store the test data as negative sample data in a negative sample database and the correction data as positive sample data in the positive sample database when the test results do not meet preset requirements. The training module 830 is further configured to: train an initial state judgment model using the positive sample data in the positive sample database and the negative sample data in the negative sample database to obtain a state judgment model, wherein the state judgment model is used to obtain a state judgment result based on the test data, and the state judgment result serves as the test result.

[0190] According to one embodiment of this application, the model training device 800 further includes an input module 860, configured to: input test data into an initial state judgment model to obtain an initial state judgment result, wherein the initial state judgment result is used to characterize whether the execution process of the test task during the target time period, as determined by the initial state judgment model, has an error, and the test result is used to characterize whether the execution process of the test task during the target time period has actually an error. The model training device 800 further includes a second storage module 870, configured to: store the test data as positive sample data in a positive sample database when the test result meets preset requirements and the initial state judgment result is inconsistent with the test result; and store the test data as negative sample data in a negative sample database when the test result does not meet preset requirements and the initial state judgment result is inconsistent with the test result. The training module 830 is further configured to: train the initial state judgment model using the positive sample data in the positive sample database and the negative sample data in the negative sample database to obtain a state judgment model, which is used to obtain a state judgment result based on the test data, and the state judgment result serves as the test result, characterizing whether an error occurred during the execution of the test task.

[0191] According to one embodiment of this application, the perception data collected in real time during the process of the robot performing a test task based on the task execution model includes: image data and / or data representing the pose of the upper joints of the robot.

[0192] According to one embodiment of this application, the task execution model and the state judgment model belong to the same model.

[0193] It should be understood that the operation and function of the execution module 810, correction module 820, training module 830, input / output module 840, first storage module 850, input module 860, and second storage module 870 in the above embodiments can be referred to the above. Figure 2 , Figure 4a or Figure 5 The description of the model training method provided in the embodiments will not be repeated here to avoid repetition.

[0194] Figure 9 The diagram shown is a schematic flowchart of a model training apparatus provided in another exemplary embodiment of this application. Figure 9 As shown, the model training device 900 may include: an execution module 910, an input module 920, a storage module 930, and a training module 940.

[0195] The execution module 910 is used to execute the test task using the task execution model and obtain the test data corresponding to the target time period during the execution of the test task. The input module 920 is used to input the test data into the initial state judgment model to obtain the initial state judgment result, which is used to characterize whether the execution process of the test task in the target time period, as determined by the initial state judgment model, has encountered errors. The storage module 930 is used to: store the test data as positive sample data in the positive sample database when the test result corresponding to the test data meets the preset requirements and the initial state judgment result is inconsistent with the test result; and store the test data as negative sample data in the negative sample database when the test result does not meet the preset requirements and the initial state judgment result is inconsistent with the test result. The training module 940 is used to train the initial state judgment model using the positive sample data in the positive sample database and the negative sample data in the negative sample database to obtain the state judgment model. The state judgment model is used to obtain the state judgment result based on the test data, and the state judgment result serves as the test result, characterizing whether errors have occurred during the execution of the test task.

[0196] This application provides a model training device that uses test data when the initial state judgment result is inconsistent with the test result as sample data, and uses the sample data to train the initial state judgment model. The device can focus on training the parts of the initial state judgment model that are inaccurate, thereby improving the model convergence effect and training efficiency.

[0197] According to one embodiment of this application, the model training device 900 further includes a correction module 950, used to correct the test data to obtain corrected data when the test result corresponding to the test data does not meet preset requirements. The training module 940 is further used to train the task execution model using the corrected data to obtain the trained task execution model.

[0198] According to one embodiment of this application, a task execution model is deployed on a robot, and the test data includes real-time perception data collected by the robot during the execution of a test task based on the task execution model. The execution module 910 is used to: control the robot to collect perception data in real time, obtain motion control commands based on the perception data at the current moment using the task execution model, and control the robot to move based on the motion control commands to execute the test task.

[0199] According to one embodiment of this application, the target time period includes the current time period, and the preset requirement includes that the robot's execution process for the test task during the current time period is error-free. The correction module 950 is used to remotely control the robot to execute the corresponding target stage within the current time period, obtaining test data corresponding to the remote operation, thereby correcting the test data. The test data corresponding to the remote operation is the corrected data, which includes the sensory data collected by the robot during the remote operation. The execution module 910 is further used to control the robot to continue executing the stages following the target stage in the test task based on the task execution model.

[0200] According to one embodiment of this application, the perception data collected in real time during the process of the robot performing a test task based on the task execution model includes: image data and / or data representing the pose of the upper joints of the robot.

[0201] According to one embodiment of this application, the task execution model and the state judgment model belong to the same model.

[0202] It should be understood that the operation and function of the execution module 910, input module 920, storage module 930, training module 940, and correction module 950 in the above embodiments can be referred to the above. Figure 2 , Figure 4a or Figure 5 The description of the model training method provided in the embodiments will not be repeated here to avoid repetition.

[0203] Figure 10 The diagram shown is a flowchart of a task execution apparatus provided in an exemplary embodiment of this application. Figure 10 As shown, the task execution device 1000 may include an acquisition module 1010 and an execution module 1020.

[0204] The acquisition module 1010 is used to acquire input data; the execution module 1020 is used to execute the target task based on the input data using the task execution model.

[0205] It should be understood that the operation and function of the acquisition module 1010 and the execution module 1020 in the above embodiments can be referred to the above. Figure 6 The description of the task execution method provided in the embodiments will not be repeated here to avoid repetition.

[0206] Figure 11 The diagram shown is a flowchart of a task execution apparatus provided in an exemplary embodiment of this application. Figure 11 As shown, the task execution device 1100 may include: an execution module 1110, an input module 1120, and an output module 1130.

[0207] The execution module 1110 is used to execute the target task using the task execution model and obtain execution data corresponding to the target time period during the execution of the target task. The input module 1120 is used to input the execution data into the status judgment model to obtain the status judgment result, wherein the status judgment result is used to indicate whether the execution result corresponding to the execution data meets the preset requirements. The output module 1130 is used to issue a reminder message when the status judgment result indicates that the execution result corresponding to the execution data does not meet the preset requirements.

[0208] This application provides a task execution device that uses a state judgment model to replace technicians in judging the execution results, thereby improving the efficiency of the entire task execution process and reducing costs.

[0209] According to an embodiment of this application, the state judgment model is based on the above. Figure 5 The model training method of the embodiment is obtained.

[0210] According to one embodiment of this application, a task execution model is deployed on a robot. The execution data includes real-time perception data collected by the robot during the execution of a target task based on the task execution model. The target time period includes the current time period, and the preset requirements include that the robot's execution process for the target task during the current time period is error-free. The execution module 1110 is further configured to: based on reminder information, remotely control the robot to execute the corresponding target stage in the current time period; and control the robot to continue executing the stages following the target stage in the target task based on the task execution model.

[0211] According to one embodiment of this application, the task execution device 1100 further includes a training module 1140, which is used to train the task execution model with the execution data corresponding to the teleoperation as correction data to obtain the trained task execution model, wherein the correction data includes the perception data collected by the robot during the teleoperation process.

[0212] It should be understood that the operation and function of the execution module 1110, input module 1120, output module 1130, and training module 1140 in the above embodiments can be referred to the above. Figure 7 The description of the task execution method provided in the embodiments will not be repeated here to avoid repetition.

[0213] Figure 12The diagram shown is a block diagram of an electronic device 1200 for performing a model training method or a task execution method according to an exemplary embodiment of this application. Specifically, the electronic device 1200 may be a server, a robot, a server interacting with a robot, or other devices, or other mobile devices.

[0214] Reference Figure 12 The electronic device 1200 includes a processing component 1210, which further includes one or more processors, and memory resources represented by memory 1220 for storing instructions executable by the processing component 1210, such as application programs. The application programs stored in memory 1220 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1210 is configured to execute instructions to perform the aforementioned model training method or task execution method.

[0215] Electronic device 1200 may also include a power supply component configured to perform power management of electronic device 1200, a wired or wireless network interface configured to connect electronic device 1200 to a network, and an input / output (I / O) interface. Electronic device 1200 can be operated based on an operating system stored in memory 1220, such as Windows Server. TM macOS X TM Unix TM Linux TM FreeBSD TM Or similar.

[0216] A non-transitory computer-readable storage medium, wherein when the instructions in the storage medium are executed by the processor of the aforementioned electronic device 1200, the aforementioned electronic device 1200 is able to execute a model training method or a task execution method.

[0217] A computer program product includes a computer program that, when executed by a processor of a computer device, enables the computer device to perform the model training method or task execution method provided in any of the above embodiments.

[0218] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this application, and will not be described in detail here.

[0219] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0220] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0221] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0222] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0223] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0224] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program verification codes, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0225] It should be noted that in the description of this application, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0226] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0227] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications or equivalent substitutions made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A model training method, characterized in that, include: The test task is executed using a task execution model to obtain test data corresponding to the target time period during the execution of the test task. The test data is input into the initial state judgment model to obtain the initial state judgment result, wherein the initial state judgment result is used to characterize whether the execution process of the test task in the target time period is incorrect, as determined by the initial state judgment model. If the test results corresponding to the test data meet the preset requirements and the initial state judgment result is inconsistent with the test results, the test data will be stored as positive sample data in the positive sample database. The test results are used to characterize whether there are actually errors in the execution process of the test task during the target time period. If the test result does not meet the preset requirements and the initial state judgment result is inconsistent with the test result, the test data will be stored as negative sample data in the negative sample database. The initial state judgment model is trained using positive sample data from the positive sample database and negative sample data from the negative sample database to obtain a state judgment model. This state judgment model is used to obtain state judgment results based on test data. These state judgment results serve as test results, characterizing whether errors occurred during the execution of the test task. The task execution model is deployed on the robot, and the test data includes real-time perception data collected by the robot during the execution of the test task based on the task execution model. The real-time perception data collected by the robot during the execution of the test task based on the task execution model includes: image data and / or data representing the pose of some joints of the robot.

2. The model training method according to claim 1, characterized in that, Also includes: If the test result corresponding to the test data does not meet the preset requirements, the test data is corrected to obtain corrected data; The task execution model is trained using the corrected data to obtain the trained task execution model.

3. The model training method according to claim 2, characterized in that, The step of using a task execution model to execute test tasks and obtaining test data corresponding to the target time period during the execution of the test tasks includes: The robot is controlled to collect perception data in real time, and the task execution model is used to obtain motion control commands based on the perception data at the current moment. The robot is controlled to move based on the motion control commands in order to perform the test task.

4. The model training method according to claim 3, characterized in that, The target time period includes the current time period, and the preset requirement includes that the robot performs the test task without errors during the current time period. The step of correcting the test data to obtain corrected data includes: By remotely controlling the robot to execute the target step corresponding to the current time period, test data corresponding to the remote operation is obtained, thereby correcting the test data. The test data corresponding to the remote operation is the corrected data, which includes the sensor data collected by the robot during the remote operation. The model training method further includes: The robot is controlled to continue executing the steps in the test task that follow the target step, based on the task execution model.

5. The model training method according to any one of claims 1 to 4, characterized in that, The task execution model and the state judgment model belong to the same model.

6. A task execution method, characterized in that, include: The target task is executed using a task execution model, and the execution data corresponding to the target time period during the execution of the target task is obtained. The execution data is input into the state judgment model to obtain the state judgment result, wherein the state judgment result is used to characterize whether the execution result corresponding to the execution data meets the preset requirements; If the state judgment result indicates that the execution result corresponding to the execution data does not meet the preset requirements, a reminder message is issued, wherein the state judgment model is obtained by the model training method according to any one of claims 1 to 5.

7. The task execution method according to claim 6, characterized in that, The execution data includes real-time perception data collected by the robot during the execution of the target task based on the task execution model; the target time period includes the current time period; the preset requirement includes that the robot's execution of the target task during the current time period is error-free; and the task execution method further includes: Based on the reminder information, the robot is remotely controlled to perform the target step corresponding to the current time period; The robot is controlled to continue executing the steps in the target task that follow the target step, based on the task execution model.

8. The task execution method according to claim 7, characterized in that, Also includes: The execution data corresponding to the teleoperation is used as correction data to train the task execution model, thereby obtaining the trained task execution model. The correction data includes the perception data collected by the robot during the teleoperation.

9. A mobile device, characterized in that, It includes a control module, which is used to execute the model training method of any one of claims 1 to 5 or the task execution method of any one of claims 6 to 8.

10. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions. The processor is used to execute the model training method of any one of claims 1 to 5 or the task execution method of any one of claims 6 to 8.

11. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for executing the model training method of any one of claims 1 to 5 or the task execution method of any one of claims 6 to 8.

12. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by the processor of a computer device, enables the computer device to perform the model training method of any one of claims 1 to 5 or the task execution method of any one of claims 6 to 8.

Images