Model training method, mobile device, electronic device and medium

Abstract

The invention provides a model training method, mobile equipment, electronic equipment and a medium, and the model training method comprises the steps: executing a test task through a task execution model, and obtaining test data corresponding to a target time period in the process of executing the test task; under the condition that the test result corresponding to the test data does not meet the preset requirement, correcting the test data to obtain corrected data; and training the task execution model by using the correction data to obtain a trained task execution model. According to the technical scheme, the whole training period of the model can be shortened, and the training efficiency is improved.

Description

Technical Field

[0001] This application relates to the technical field of model training, and in particular, to a model training method, a mobile device, an electronic device, and a medium. Background Art

[0002] In the field of artificial intelligence, a model is a core component of artificial intelligence and a mathematical and computational framework for implementing intelligent behaviors. Specifically, a model can be an abstract framework for describing a system, structure, or process, used to learn patterns from data and establish a mapping relationship from input to output. For example, an image recognition model is trained with sample data so that the trained image recognition model can recognize target objects in an image. Currently, there is a problem of too long a cycle in the process of training a model to obtain a finally satisfactory model. Summary of the Invention

[0003] In view of this, embodiments of this application provide a model training method, a mobile device, an electronic device, and a medium, which can reduce the entire training cycle of the model and improve the training efficiency.

[0004] In a first aspect, an embodiment of this application provides a model training method, including: using a task execution model to execute a test task, and obtaining test data corresponding to a target period during the execution of the test task; in a case where the test result corresponding to the test data does not meet a preset requirement, correcting the test data to obtain corrected data; using the corrected data to train the task execution model to obtain a trained task execution model.

[0005] In a second aspect, an embodiment of this application provides a model training method, including: using a task execution model to execute a test task, and obtaining test data corresponding to a target period during the execution of the test task; inputting the test data into an initial state judgment model to obtain an initial state judgment result, where the initial state judgment result is used to represent whether an error occurs in the execution process of the test task by the initial state judgment model during the target period; in a case where the test result corresponding to the test data meets the preset requirement and the initial state judgment result is inconsistent with the test result, storing the test data as positive sample data in a positive sample database, where the test result is used to represent whether an error actually occurs in the execution process of the test task during the target period; in a case where the test result does not meet the preset requirement and the initial state judgment result is inconsistent with the test result, storing the test data as negative sample data in a negative sample database; using the positive sample data in the positive sample database and the negative sample data in the negative sample database to train the initial state judgment model to obtain a state judgment model, where the state judgment model is used to obtain a state judgment result based on the test data, and the state judgment result is used as the test result to represent whether an error occurs in the execution process of the test task.

[0006] In a third aspect, an embodiment of the present application provides a task execution method, including: obtaining input data; and using a task execution model to execute a target task according to the input data. The task execution model is obtained according to the model training method in the first aspect.

[0007] In a fourth aspect, an embodiment of the present application provides a model training device, including: an execution module, configured to use a task execution model to execute a test task and obtain test data corresponding to a target time period during the execution of the test task; a correction module, configured to correct the test data to obtain corrected data when the test result corresponding to the test data does not meet a preset requirement; and a training module, configured to train the task execution model by using the corrected data to obtain a trained task execution model.

[0008] In a fifth aspect, an embodiment of the present application provides a model training device, including: an execution module, configured to use a task execution model to execute a test task and obtain test data corresponding to a target time period during the execution of the test task; an input module, configured to input the test data into an initial state judgment model to obtain an initial state judgment result, where the initial state judgment result is used to represent whether an error occurs in the execution process of the test task by the initial state judgment model during the target time period; a storage module, configured to: when the test result corresponding to the test data meets the preset requirement and the initial state judgment result is inconsistent with the test result, store the test data as positive sample data in a positive sample database, where the test result is used to represent whether an actual error occurs in the execution process of the test task during the target time period, and when the test result does not meet the preset requirement and the initial state judgment result is inconsistent with the test result, store the test data as negative sample data in a negative sample database; and a training module, configured to train the initial state judgment model by using the positive sample data in the positive sample database and the negative sample data in the negative sample database to obtain a state judgment model, where the state judgment model is used to obtain a state judgment result based on the test data, and the state judgment result is used as the test result to represent whether an error occurs in the execution process of the test task.

[0009] In a sixth aspect, an embodiment of the present application provides a task execution device, including: an obtaining module, configured to obtain input data; and an execution module, configured to use a task execution model to execute a target task according to the input data.

[0010] In a seventh aspect, an embodiment of the present application provides a mobile device, including a control module, where the control module is configured to execute the model training method described in the first aspect, the model training method described in the second aspect, or the task execution method described in the third aspect.

[0011] In an eighth aspect, an embodiment of the present application provides an electronic device, including: a processor; and a memory for storing instructions executable by the processor, wherein the processor is configured to execute the model training method described in the first aspect, the model training method described in the second aspect, or the task execution method described in the third aspect above.

[0012] In a ninth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program for executing the model training method described in the first aspect, the model training method described in the second aspect, or the task execution method described in the third aspect above.

[0013] In a tenth aspect, an embodiment of the present application provides a computer program product including a computer program, which, when executed by a processor of a computer device, enables the computer device to execute the model training method described in the first aspect, the model training method described in the second aspect, or the task execution method described in the third aspect above.

[0014] In an eleventh aspect, an embodiment of the present application provides a chip, including: a processor; and a memory for storing instructions executable by the processor, wherein the processor is configured to execute the model training method described in the first aspect, the model training method described in the second aspect, or the task execution method described in the third aspect above.

[0015] The embodiments of the present application provide a model training method, a mobile device, an electronic device, and a medium. By testing a task execution model, that is, using the task execution model to execute a test task, test data corresponding to a target time period during the execution of the test task is obtained. When the test result corresponding to the test data does not meet the preset requirements, for example, when the test result indicates an error during the execution of the test task, the test data is corrected to obtain corrected data, and the task execution model is trained using the corrected data, and a trained task execution model can be obtained. In this way, attention can be paid to the links where the execution effect of the task execution model is not good during the test process, that is, when the test result does not meet the preset requirements, the test data is corrected to obtain corrected data. Using the corrected data to further train the model can quickly improve the convergence effect of the model, reduce the entire training cycle of the model, and improve the training efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 The figure shows a schematic diagram of the system architecture of a model training system provided by an exemplary embodiment of the present application.

[0017] Figure 2 The figure shows a schematic flowchart of a model training method provided by an exemplary embodiment of the present application.

[0018] Figure 3a The figure shows a schematic diagram of the manual takeover process provided by an exemplary embodiment of the present application.

[0019] Figure 3b The figure shows a schematic diagram of the manual takeover process provided by another exemplary embodiment of the present application.

[0020] Figure 4a The figure shows a schematic flowchart of the model training method provided by another exemplary embodiment of the present application.

[0021] Figure 4b The figure shows a schematic diagram of the data acquisition process provided by an exemplary embodiment of the present application.

[0022] Figure 5 The figure shows a schematic flowchart of the model training method provided by another exemplary embodiment of the present application.

[0023] Figure 6 The figure shows a schematic flowchart of the task execution method provided by an exemplary embodiment of the present application.

[0024] Figure 7 The figure shows a schematic flowchart of the task execution method provided by another exemplary embodiment of the present application.

[0025] Figure 8 The figure shows a schematic flowchart of the model training device provided by an exemplary embodiment of the present application.

[0026] Figure 9 The figure shows a schematic flowchart of the model training device provided by another exemplary embodiment of the present application.

[0027] Figure 10 The figure shows a schematic flowchart of the task execution device provided by an exemplary embodiment of the present application.

[0028] Figure 11 The figure shows a schematic flowchart of the task execution device provided by another exemplary embodiment of the present application.

[0029] Figure 12 The figure shows a block diagram of an electronic device for executing the model training method or the task execution method provided by an exemplary embodiment of the present application. Detailed implementation manners

[0030] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0031] Application Overview

[0032] Before the model is actually used, generally, sample data is used to train the model, and then the trained model is tested. If the test result shows that the accuracy of the model does not meet the requirements, the model is continuously trained and then the trained model is tested again, and so on, until the accuracy of the trained model meets the requirements. The model with the accuracy meeting the requirements can be put into practical application.

[0033] In particular, for the scenario where a robot uses a model deployed on the robot to perform a target task, before training the model, it is necessary to collect sample data. Generally, a factory that needs to use a robot to perform a target task (such as grasping a target object) can submit a task of collecting sample data; the data collection factory can decompose the collection task and collect data according to certain standards. For example, a data collector can control the robot to perform the target task through a remote control handle or the like, and collect sample data such as image data during the execution of the target task; upload the sample data to the cloud and use the sample data to train the model; download the trained model, and the factory can test the trained model. For example, deploy the trained model to the robot, use the robot to perform the target task, and check the execution effect of the target task, that is, obtain the test effect; if the execution effect is not good, it is necessary to re-execute the data collection process, the training process, and the testing process, or re-plan the collection task and re-execute the data collection process, the training process, and the testing process until a model with a good test effect is obtained.

[0034] It can be seen that the current model training process, especially the training process of the model used for a robot to perform tasks, has the problem of too long a cycle.

[0035] To solve the above technical problems, an embodiment of the present application provides a model training method. By testing the task execution model, that is, using the task execution model to perform a test task, test data corresponding to a target time period during the execution of the test task is obtained. When the test result corresponding to the test data does not meet the preset requirements, for example, when the test result shows an error during the execution of the test task, the test data is corrected to obtain corrected data, and the task execution model is trained using the corrected data, and a trained task execution model can be obtained. In this way, attention can be paid to the link where the execution effect of the task execution model is not good during the test process, that is, when the test result does not meet the preset requirements, the test data is corrected to obtain corrected data. Using the corrected data to further train the model can quickly improve the convergence effect of the model, reduce the entire training cycle of the model, and improve the training efficiency.

[0036] Exemplary System

[0037] Figure 1 The following shows a schematic diagram of the system architecture of a model training system provided by an exemplary embodiment of the present application. As Figure 1 shown, the model training system 100 may include a computing device 110 and a task execution model 120 deployed on the computing device 110.

[0038] In one example, the computing device 110 may be a server or an electronic device. The server may be a physical server, a cloud server, a virtual server, or other types of servers. The electronic device may include a computer, a mobile phone, a tablet, or a mobile device, etc. The mobile device may include a robot or a vehicle, etc. The robot may be a humanoid robot, a wheeled robot, a tracked robot, or other types of robots.

[0039] In one example, the computing device 110 may use the task execution model 120 to execute a test task and obtain test data corresponding to a target time period during the execution of the test task. In the case where the test result corresponding to the test data does not meet the preset requirements, the test data may be corrected to obtain corrected data through the computing device 110, or through a technician, or through other devices. The computing device 110 may use the corrected data to train the task execution model 120 to obtain a trained task execution model.

[0040] Further, the computing device 110 may repeatedly execute the process of testing the task execution model 120, that is, the computing device 110 may use the task execution model 120 to execute the test task again and obtain test data corresponding to a target time period during the execution of the test task. In the case where the test result corresponding to the test data does not meet the preset requirements, the test data may be corrected to obtain corrected data. The computing device 110 may use the corrected data to train the task execution model again to obtain a trained task execution model. This cycle continues until the trained task execution model meets the training cut-off requirements, such as meeting the accuracy requirements.

[0041] In one example, the test tasks in each test process may be the same, such as grasping a target object, but the target objects corresponding to the test tasks in each test process may not be the same, and the target time periods corresponding to the test results that do not meet the preset requirements in each test process may be the same or different. Similarly, the test data corresponding to the target time period in each test process may also be the same or different.

[0042] Optionally, when the test task has not been fully executed, after correcting the test data corresponding to the target time period, the subsequent links of the test task can continue to be executed using the task execution model. When executing the subsequent links, the test data corresponding to the subsequent links can also be obtained, and it can be determined whether the test results of the test data corresponding to the subsequent links meet the preset requirements. In the case where the test results of the test data corresponding to the subsequent links do not meet the preset requirements, the test data can be corrected to obtain corrected data, and the task execution model can be trained using the corrected data.

[0043] In one example, after the trained task execution model meets the training cut-off requirements, the trained task execution model can be applied to the actual operation process.

[0044] It should be understood that the above application scenario examples are only shown for the convenience of understanding the spirit and principle of the present application, and the embodiments of the present application are not limited thereto. On the contrary, the embodiments of the present application can be applied to any scenario where they may be applicable.

[0045] Exemplary method

[0046] Figure 2 The following shows a schematic flowchart of a model training method provided by an exemplary embodiment of the present application. Figure 2 The method can be executed by Figure 1 the computing device 110 in Figure 2 As shown below, the model training method may include the following content.

[0047] 210: Use the task execution model to execute the test task, and obtain the test data corresponding to the target time period during the execution of the test task.

[0048] Specifically, the task execution model may include at least one of a convolutional neural network model, a recurrent neural network model, a feedforward neural network model, a generative adversarial network model, a Transformer model, or other network models.

[0049] In one example, at the initial moment, the task execution model may be a model trained with a large amount of sample data, and this model may be called a mature model, or a Warm model or a Hot model. Optionally, at the initial moment, the task execution model may be an initial model that has not been fully trained or has only been pre-trained on a basic level, and this initial model may be called a Cold model, or a cold start model. Optionally, at the initial moment, the task execution model may be an intermediate state model that has been preliminarily trained or fine-tuned with limited data, and this intermediate state model may be called a Mild model, or a gentle transition model.

[0050] It should be understood that the type of the test task belongs to the type of tasks that the task execution model is expected to execute. The test task is a task executed by using the task execution model during the process of testing the task execution model. In one example, the test task, or the task that the task execution model is expected to execute, can be a static task such as an image recognition task or a speech recognition task, or can be a dynamic task such as grasping a target object or cleaning a target area. When the test task is a dynamic task, the computing device can be a mobile device.

[0051] In one example, test data can be obtained during the execution of the test task. The test data can include the input data and output data of the task execution model in the target period. For example, the input data can include image data, and the output data can include recognition results. For example, the test data during the execution of the test task can be obtained at a certain frequency, and the test data within a certain period can be obtained each time. In one example, the target period can be the entire period of executing the test task, or a certain period during the execution of the test task, or the current period during the execution of the test task.

[0052] 220: When the test result corresponding to the test data does not meet the preset requirements, the test data is corrected to obtain corrected data.

[0053] In one example, the preset requirements can be set according to actual needs. The preset requirements can represent the performance of the task execution model expected by the user, or the execution effect of the task, or that there is no error in the execution process of the test task in the target period. Correspondingly, the test result can represent the execution effect of the task execution model for the test task in the target period, or whether there is an error in the execution process of the test task in the target period. In one example, the preset requirements can include at least one of the following: in the target period, the target action in the test task is successfully completed; in the target period, the time length range of at least one action execution in the test task; in the target period, no unexpected event occurs during the execution process of the test task, etc. The test result can represent at least one of the following: in the target period, whether the target action in the test task is successfully completed; in the target period, the time length of at least one action execution in the test task; in the target period, whether an unexpected event occurs during the execution process of the test task, etc. When the test result meets the preset requirements, it indicates that there is no error in the execution process of the test task in the target period.

[0054] In one example, the test result can represent whether there is an error in the execution of the test task by the task execution model in the target period. For example, when the above recognition result is error-free, that is, correct or successful, the test result meets the preset requirements; otherwise, it does not meet the preset requirements. In one example, whether the above recognition result is in error can be judged by a technician.

[0055] When the test results do not meet the preset requirements, the test data is corrected to obtain corrected data. In one example, the output data in the test data can be corrected to obtain the corrected output data, and the corrected output data can be used as the annotation of the input data. The input data plus the corrected output data can be used as the corrected data.

[0056] In one example, when the test task is an image recognition task, the image recognition task may include multiple links, and each link corresponds to specific input data and output data. During the execution of the image recognition task, if the test results corresponding to the test data in the target time period (such as the input data and output data corresponding to the target time period) do not meet the preset requirements, the test data can be corrected.

[0057] 230: Use the corrected data to train the task execution model to obtain the trained task execution model.

[0058] Specifically, the corrected data can be used as training data to train the task execution model to obtain the trained task execution model.

[0059] In one example, steps 210 and 220 can be repeatedly executed to obtain a certain amount of corrected data, and the trained task execution model can be obtained by using these corrected data to train the task execution model. The trained task execution model can have a higher accuracy rate during the subsequent task execution process.

[0060] The embodiment of the present application provides a model training method. By testing the task execution model, that is, using the task execution model to execute the test task, the test data corresponding to the target time period during the execution of the test task is obtained. When the test results corresponding to the test data do not meet the preset requirements, such as when the test results indicate an error during the execution of the test task, the test data is corrected to obtain the corrected data, and the trained task execution model can be obtained by using the corrected data to train the task execution model. In this way, the links with poor execution effects of the task execution model during the test process can be concerned. That is, when the test results do not meet the preset requirements, the test data is corrected to obtain the corrected data. Using the corrected data to further train the model can quickly improve the convergence effect of the model, reduce the entire training cycle of the model, and improve the training efficiency.

[0061] According to an embodiment of the present application, the task execution model is deployed on a robot, and the test data includes the perception data collected in real time during the process of the robot executing the test task based on the task execution model. Among them, step 210 may include: controlling the robot to collect perception data in real time, using the task execution model to obtain a motion control instruction based on the perception data at the current moment; controlling the robot to move based on the motion control instruction to execute the test task.

[0062] Specifically, the computing device can be a robot, and the robot executes tasks through a task execution model deployed on the robot. In one example, the robot can collect perception data in real time, and the perception data at the current moment can be used as the input data of the task execution model. In one example, the perception data can include image data and / or data representing the poses of some joints on the robot. For example, the robot can collect image data through an image acquisition sensor provided on the robot, and the image data can reflect the surrounding environment conditions of the robot at the current moment, such as the pose of the robot at this time, the poses of the objects around the robot, etc. Further, the robot can obtain data representing the poses of some joints on the robot, such as joint positions and joint angles, through an inertial measurement unit provided on the robot.

[0063] In one example, the task execution model can output a motion control instruction based on the perception data, and the motion control instruction can include data representing the poses of some joints on the robot at a future moment, such as joint positions and joint angles. In some scenarios, the motion control instruction can also include parameters such as the opening and closing degree of the robot arm and the pitching and lifting parameters of the robot waist.

[0064] In one example, motion control instructions can be smoothed using motion control calculation algorithms, such as filtering, collision detection, admittance compliance, etc., to obtain processed motion control instructions. Controlling the robot's motion based on the processed motion control instructions can improve the smoothness of the robot's actions.

[0065] Optionally, the task execution model can obtain a motion control instruction based on the perception data, smooth the motion control instruction, and then output the smoothed motion control instruction.

[0066] In one example, during the process of controlling the robot's motion based on the motion control instruction or the processed motion control instruction, the robot still collects perception data in real time. At this time, the perception data (such as image data, or image data and data representing the poses of some joints on the robot) can be used as an intuitive reflection of the motion control instruction or the processed motion control instruction on the robot's actions. At this time, the perception data can be used as the indirect output data of the task execution model.

[0067] In this embodiment, the test data can include the input data of the task execution model during the target period and the corresponding indirect output data.

[0068] In one example, when the test results do not meet the preset requirements, the indirect output data in the test data can be corrected to obtain the corrected output data, and the corrected output data can be used as the annotation of the input data. The input data plus the corrected output data can be used as the correction data. For example, the input data may include image data, and the indirect output data also includes image data. The whole of the input data and the indirect output data may include multiple frames of images continuously collected by the robot. In one example, the indirect output data can be corrected through an algorithm to obtain the corrected output data, and the corrected output data can reflect that the robot has successfully executed the corresponding target link in the target period, that is, the actions that have not been successfully executed in the target period, such as picking up the target object.

[0069] In this embodiment, when the test results do not meet the preset requirements, the test data is corrected to obtain the correction data, and the task execution model is trained using the correction data. In this way, the links with poor execution effects of the task execution model during the test process can be concerned, and then the convergence effect of the model can be quickly improved, the entire training cycle of the model can be reduced, and the training efficiency can be improved, that is, the robot task learning efficiency can be improved.

[0070] According to an embodiment of the present application, the target period includes the current period, and the preset requirements include that the robot does not make mistakes during the execution process of the test task in the current period. Among them, step 220 may include: controlling the robot to execute the corresponding target link in the current period through teleoperation to obtain the test data corresponding to the teleoperation, so as to correct the test data. The test data corresponding to the teleoperation is the correction data, and the correction data includes the perception data collected by the robot during the teleoperation process. Further, the model training method further includes: controlling the robot to continue to execute the link after the target link in the test task based on the task execution model.

[0071] Specifically, the teleoperation may include operating the robot to move through a handle, a remote control or other means. During the process of the robot executing the test task based on the task execution model, if the test data with test results not meeting the preset requirements is obtained, that is, when the robot makes a mistake, the robot can be controlled to continue to execute the test task through teleoperation. In one example, the robot can be controlled to execute the corresponding target link in the current period through teleoperation, that is, the link where the error occurs. In other words, the way of using teleoperation is used to replace the way of using the task execution model to analyze the input data to control the robot to execute the target joint in the test task. During the process of the technician using teleoperation to control the robot to execute the target joint, the robot can continue to collect perception data in real time, and this perception data is the test data corresponding to the teleoperation. The teleoperation can make up for the mistakes of the robot. Therefore, obtaining the corresponding test data through teleoperation is equivalent to correcting the mistakes of the robot, that is, the test data corresponding to the teleoperation can be used as the correction data.

[0072] Further, after the robot successfully executes the target link through teleoperation control, the task execution model can be continued to be used to control the robot to execute the subsequent links after the target link in the test task. When executing the subsequent links, the test data corresponding to the subsequent links can also be obtained, and it can be determined whether the test results of the test data corresponding to the subsequent links meet the preset requirements. In the case where the test results of the test data corresponding to the subsequent links do not meet the preset requirements, the test data can be corrected to obtain corrected data, and the task execution model can be trained using the corrected data.

[0073] In one example, the test task may include one link or multiple links, and each link may include one step. Taking the test task including multiple steps as an example, the test task can be making a sandwich, and the multiple steps may include: Step 1, pick up a slice of bread on the table and place it on the operating table; Step 2, pick up the meat slices on the table and place them on the slice of bread; Step 3, pick up another slice of bread on the table and place it on the meat slices, thus obtaining a sandwich.

[0074] The robot first executes the test task of making a sandwich based on the task execution model. In one case, during the execution of Step 1, after the robot picks up the slice of bread, the slice of bread falls to the ground and is not placed on the operating table, that is, Step 1 in the test task fails to execute, or in other words, the robot makes an error when executing Step 1. At this time, the robot can be taken over manually, that is, the technician can control the robot to pick up another slice of bread from the table and place it on the operating table through teleoperation, that is, complete Step 1. In one example, the manual takeover process can be implemented through the first operation key (such as a button) on the robot. For example, by clicking the first operation key, the control process of the task execution model can be stopped and the teleoperation control process can be started. After completing Step 1, the teleoperation can be cancelled, and the robot can continue to autonomously execute the subsequent steps, that is, the robot can continue to execute the subsequent Step 2 and Step 3 based on the task execution model. In one example, cancelling the teleoperation can be implemented through the second operation key (such as a button) on the robot. For example, by clicking the second operation key, the teleoperation control process can be stopped and the control process of the task execution model can be started. In one example, the first operation key and the second operation key can be the same or different operation keys.

[0075] Such as Figure 3aAs shown, the robot autonomously executes based on the task execution model from time t0 to time t1. An error occurs at time t2, and at this time, step 1 is not completed. At time t3, a technician starts to take over. The robot can retreat to a certain position, such as from the position corresponding to time t3, retreat to the position corresponding to time t2 at time t2', and then return to the position corresponding to time t1 at time t1'. For example, the position at time t1' can be the same as the position at time t1, and time t1 can be a certain moment before the robot makes an error. After returning to the position at time t1, the technician controls the robot to continue executing the test task through teleoperation, that is, through times t1', t2", t3', and t4, the technician controls the robot to correctly complete step 1. At time t5, the test task is returned to the task execution model for execution until an error occurs and is taken over by the technician again. Here, the perception data from time t3' to t5 (excluding time t5) can be used as correction data, or the perception data from time t1' to t5 (excluding time t5) can be used as correction data, as long as the correction data covers the error-prone link (step 1).

[0076] In another case, during the execution of step 1, the robot fails to pick up the slice of bread successfully. The slice of bread is still on the table and has not been placed on the operating table, that is, step 1 fails to execute, or in other words, the robot makes an error when executing step 1. At this time, the robot can be taken over manually, that is, the technician can control the robot to directly pick up a slice of bread from the table and place it on the operating table through teleoperation, that is, complete step 1. After completing step 1, the teleoperation can be cancelled, and the robot continues to autonomously execute the subsequent steps.

[0077] As Figure 3b As shown, the robot autonomously executes based on the task execution model from time t0 to time t1. An error occurs at time t2, and at this time, step 1 is not completed. At time t3, a technician starts to take over. The technician directly controls the robot to continue executing the test task through teleoperation, that is, through times t4 - t5, the technician controls the robot to correctly complete step 1. At time t6, the test task is returned to the task execution model for execution until an error occurs and is taken over by the technician again. Here, the perception data from time t3 to t6 (excluding time t6) can be used as correction data so that the correction data covers the error-prone link (step 1).

[0078] In this embodiment, when the test result does not meet the preset requirements, the robot is remotely operated to continue executing the test task, so that the link that does not meet the preset requirements can be corrected, that is, correctly executed, and then correction data can be obtained. The task execution model can be trained using the correction data to improve the correct execution of the task execution model for the link corresponding to the correction data, that is, to intensively train the weak link of the task execution model, thereby quickly improving the convergence effect of the model, reducing the entire training cycle of the model, and improving the training efficiency.

[0079] According to an embodiment of the present application, the model training method further includes: inputting test data into a state judgment model to obtain a state judgment result, where the state judgment result is used as a test result to indicate whether an error occurs during the execution of the test task by the robot in the current period; when the test result does not meet the preset requirements, a reminder message is sent.

[0080] Specifically, the state judgment model can be used to replace the technician to judge the test result corresponding to the test data.

[0081] In one example, when the test data is input into the state judgment model, the state judgment model can output a state judgment result. The state judgment result can be used to indicate whether a situation that does not meet the preset requirements occurs during the execution of the test task by the robot.

[0082] In one example, the state judgment result can be used to indicate whether an error occurs during the execution of the test task by the robot, that is, to indicate whether an error occurs during the execution process of the test task in the target period. This state judgment result can be used as a test result.

[0083] Furthermore, when the test result does not meet the preset requirements, for example, when the test result indicates that an error occurs during the execution of the test task by the robot, a reminder message is sent. The reminder message can include at least one of information such as text information, voice information, and light information. The reminder message is used to remind the technician that the task execution model has an error and manual takeover is required.

[0084] In one example, the state judgment model can include at least one of a convolutional neural network model, a recurrent neural network model, a feedforward neural network model, a generative adversarial network model, a Transformer model, or other network models.

[0085] In this embodiment, using the state judgment model to replace the technician to judge the test result corresponding to the test data can improve the training efficiency of the entire training process and reduce costs.

[0086] According to an embodiment of the present application, the model training method further includes: when the test result meets the preset requirements, storing the test data as positive sample data in the positive sample database; when the test result does not meet the preset requirements, storing the test data as negative sample data in the negative sample database, and storing the corrected data as positive sample data in the positive sample database; training the initial state judgment model using the positive sample data in the positive sample database and the negative sample data in the negative sample database to obtain a state judgment model, where the state judgment model is used to obtain a state judgment result based on the test data, and the state judgment result serves as the test result.

[0087] Specifically, for a target time period, if the test result meets the preset requirements, the test data at this time can be used as positive sample data; if the test result does not meet the preset requirements, the test data at this time can be used as negative sample data, and the corresponding corrected data can be used as positive sample data. Training the initial state judgment model using the positive sample data and the negative sample data can obtain a state judgment model. During the process of using the task execution model to execute a task (such as a test task), the state judgment model can judge the execution state of the task execution model based on the test data (such as image data) to obtain a state judgment result. The relevant content of the state judgment result can be referred to the description in the above embodiment. To avoid repetition, it will not be elaborated here.

[0088] In an example, the preset requirements may include that the robot does not make mistakes when executing the test task during the target time period. Further, if the test result meets the preset requirements, it indicates that the task execution model does not make mistakes during this target time period, and the test data at this time can be used as positive sample data; if the test result does not meet the preset requirements, it indicates that the task execution model makes mistakes during this target time period, and the test data at this time can be used as negative sample data.

[0089] In this embodiment, during the process of executing the test task, the test data can be collected at a certain frequency to obtain a large amount of test data. During the process of collecting the test data for any target time period, the test data whose test result does not meet the preset requirements can be corrected to obtain the corrected data. In this way, the positive sample data and the negative sample data can be collected in real time during the test process of the task execution model, improving the efficiency of data collection. Further, the collected positive sample data and negative sample data can be used to train the initial state judgment model, and the training process of the initial state judgment model can be synchronized with the process of executing the test task through the task execution model, so as to improve the training efficiency of the initial state judgment model. Of course, the training process of the initial state judgment model can also not be synchronized with the process of executing the test task through the task execution model, such as in other time periods.

[0090] According to an embodiment of the present application, the model training method further includes: inputting test data into the initial state judgment model to obtain an initial state judgment result, where the initial state judgment result is used to represent whether an error occurs in the execution process of the test task by the initial state judgment model during the target period, and the test result is used to represent whether an actual error occurs in the execution process of the test task during the target period; in the case where the test result meets the preset requirements and the initial state judgment result is inconsistent with the test result, storing the test data as positive sample data in the positive sample database; in the case where the test result does not meet the preset requirements and the initial state judgment result is inconsistent with the test result, storing the test data as negative sample data in the negative sample database; training the initial state judgment model using the positive sample data in the positive sample database and the negative sample data in the negative sample database to obtain a state judgment model, where the state judgment model is used to obtain a state judgment result based on the test data, and the state judgment result is used as the test result to represent whether an error occurs in the execution process of the test task.

[0091] Specifically, when the test task is executed by a robot using a task execution model, the initial state judgment result can be used to represent whether an error occurs during the process of the robot executing the test task, that is, whether an error occurs during the target period, and the initial state judgment result is obtained based on the initial state judgment model. The test result can be used to represent whether an actual error occurs during the process of the robot executing the test task, that is, whether an actual error occurs during the target period, and the test result can be determined by a technician or by a specific algorithm.

[0092] In an example, the accuracy of the test result is greater than the accuracy of the initial state judgment result. Therefore, when the test result is inconsistent with the initial state judgment result, the test result can be used as the standard.

[0093] For example, in the case where the test result meets the preset requirements and the initial state judgment result is inconsistent with the test result, it indicates that the task execution model does not make an error during the target period, but the initial state judgment model's judgment for the target period is inaccurate. At this time, the test data can be stored as positive sample data in the positive sample database. In the case where the test result does not meet the preset requirements and the initial state judgment result is inconsistent with the test result, it indicates that the task execution model makes an error during the target period, but the initial state judgment model's judgment for the target period is inaccurate. At this time, the test data can be stored as negative sample data in the negative sample database.

[0094] Further, when the test result meets the preset requirements and the initial state judgment result is consistent with the test result, it indicates that the task execution model has not made an error during the target period, and the initial state judgment model's judgment for the target period is accurate. In this case, the test data does not need to be stored in the database. When the test result does not meet the preset requirements and the initial state judgment result is consistent with the test result, it indicates that the task execution model has made an error during the target period, and the initial state judgment model's judgment for the target period is accurate. In this case, the test data does not need to be stored in the database either.

[0095] After the initial state judgment model is trained, a state judgment model can be obtained. Subsequently, the state judgment model can be directly used to judge the execution state of the task execution model based on the test data, that is, the state judgment result output by the state judgment model can be used as the test result. In this way, it can be directly determined whether the task execution model has made an error during the relevant period through the model.

[0096] In one example, the initial state judgment model can be a Cold model or a Mild model or a Warm model.

[0097] In this embodiment, the test data when the initial state judgment result is inconsistent with the test result is used as sample data, and the initial state judgment model is trained using this sample data. The parts where the initial state judgment model is inaccurate can be trained intensively. In this way, the model convergence effect can be improved and the training efficiency can be enhanced.

[0098] According to an embodiment of the present application, the task execution model and the state judgment model belong to the same model. For example, the task execution model and the state judgment model can be different parts of the same model, or there is at least a partially overlapping part between the task execution model and the state judgment model in the same model.

[0099] In one example, the same model can be a vision-language-action large model. Further, the vision-language-action large model can have an autonomous state judgment function. Of course, the same model can be other models that can implement the functions of the method embodiment of the present application.

[0100] In this embodiment, integrating the task execution model and the state judgment model on one model can improve the model convergence effect and the model training efficiency.

[0101] Figure 4a The following shows a schematic flowchart of a model training method provided by another exemplary embodiment of the present application. Figure 4a The embodiment is Figure 2 an example of the embodiment. To avoid repetition, the same parts can refer to the description in the above embodiment and will not be elaborated here. As Figure 4a shown, the model training method may include the following contents.

[0102] 410: Train the initial model with the initial sample data to obtain a task execution model.

[0103] Specifically, the initial model can be a Cold model. A small amount of initial sample data can be used to train the Cold model to obtain a task execution model. In one example, the task execution model can be a Warm model.

[0104] In one example, the initial sample data can be collected by means of teleoperation or other means. For example, a technician can control a robot to perform a target task through teleoperation, and the perception data obtained during the execution of the target task can be used as the initial sample data.

[0105] 420: Use the task execution model to perform a test task, and obtain the test data corresponding to the current time period during the execution of the test task.

[0106] 430: In the case where the test result corresponding to the test data does not meet the preset requirements, correct the test data to obtain corrected data, store the test data as the first negative sample data in the negative sample database, and store the corrected data as the first positive sample data in the positive sample database.

[0107] 440: In the case where the test result corresponding to the test data meets the preset requirements, the test data can be stored as the second positive sample data in the positive sample database.

[0108] In one example, during the process of using the task execution model to perform a test task, a technician can supervise beside. During the process of the robot performing the test task based on the task execution model, if the execution effect does not meet the expectation, that is, in the case where the test result of the test data corresponding to the current time period does not meet the preset requirements, the technician can directly take over.

[0109] Further, the takeover process of the technician can be referred to in Appendix Figure 3a or Appendix Figure 3b , and the test data obtained during the technician's takeover process can be used as corrected data.

[0110] In one example, as Figure 4b shown, the task execution model performs a test task. The process of performing the test task can be referred to as a test process or an inference process. When the task execution model does not make a mistake (is normal), the test data corresponding to the corresponding time period can be stored in the positive sample database. When the task execution model makes a mistake, the test data corresponding to the corresponding time period can be stored in the negative sample database, and the test data can be corrected through manual takeover to obtain corrected data, and the corrected data can be stored in the positive sample database.

[0111] In one example, as the test task progresses, steps 430 and 440 can be repeatedly executed. In one example, the execution order of steps 430 and 440 can be in any order. In one example, step 440 can be omitted.

[0112] 450: Train the task execution model using the correction data in the positive sample database to obtain the trained task execution model.

[0113] In one example, the task execution model can be trained using the correction data in the positive sample database and the first negative sample data in the negative sample database to obtain the trained task execution model.

[0114] 460: Input the test data into the initial state judgment model to obtain the initial state judgment result.

[0115] 470: When the test result meets the preset requirements and the initial state judgment result is inconsistent with the test result, store the test data as the second positive sample data in the positive sample database. When the test result does not meet the preset requirements and the initial state judgment result is inconsistent with the test result, store the test data as the second negative sample data in the negative sample database.

[0116] In one example, the initial state judgment result is used to represent whether the test result of the robot during the execution of the test task obtained by the initial state judgment model meets the preset requirements, and the test result is used to represent whether the robot actually meets the preset requirements during the execution of the test task.

[0117] In one example, as Figure 4b shown, the test data collected in real time during the execution of the test task by the task execution model can be synchronously input into the initial state judgment model to obtain the initial state judgment result. When the actual test result meets the preset requirements, that is, when the task execution model is normal (without errors), if the initial state judgment result indicates that the task execution model has an error, it means that the initial state judgment model has made a wrong judgment. At this time, the test data can be stored as the second positive sample data in the positive sample database. When the actual test result does not meet the preset requirements, that is, when the task execution model has an error, if the initial state judgment result indicates that the task execution model is not in error (normal), it means that the initial state judgment model has made a wrong judgment. At this time, the test data can be stored as the second negative sample data in the negative sample database.

[0118] 480: Train the initial state judgment model using the second positive sample data in the positive sample database and the second negative sample data in the negative sample database to obtain the state judgment model.

[0119] The state judgment model is used to obtain a state judgment result based on the test data. In one example, the state judgment result can be used as a test result to indicate whether the robot has made any errors in performing the test task.

[0120] In one example, the initial state judgment model can be obtained by training based on a small amount of sample data. For example, a basic model can be trained based on first positive sample data and first negative sample data to obtain the initial state judgment model. Alternatively, the basic model can be trained based on other sample data to obtain the initial state judgment model, as long as the sample data can enable the initial state judgment model to output the initial state judgment result.

[0121] In one example, step 430 and step 460 may be performed in any order.

[0122] In one example, steps 430 - 480 may be repeatedly performed until the accuracy of the task execution model and the state judgment model meets a preset accuracy requirement.

[0123] It should be understood that the execution order of the above steps can be adjusted according to actual needs.

[0124] like Figure 5 As shown, the embodiment of the present application also provides a model training method, Figure 5 The method can be Figure 1 The computing device 110 in FIG. Figure 5 As shown, the model training method may include the following contents.

[0125] 510: Execute the test task using the task execution model, and obtain test data corresponding to the target period during the execution of the test task.

[0126] Specifically, the relevant contents of the task execution model, test tasks, target time period and test data can refer to the relevant descriptions in the above embodiments, and will not be repeated here to avoid repetition.

[0127] 520: Input the test data into the initial state judgment model to obtain an initial state judgment result, wherein the initial state judgment result is used to indicate whether an execution process of the test task in the target time period is wrong as determined by the initial state judgment model.

[0128] 530: When the test result corresponding to the test data meets the preset requirement and the initial state judgment result is inconsistent with the test result, the test data is stored as positive sample data in the positive sample database.

[0129] The test results are used to indicate whether there are any errors in the execution process of the test task during the target period.

[0130] 540: When the test result does not meet the preset requirements and the initial state judgment result is inconsistent with the test result, the test data is stored as negative sample data in the negative sample database.

[0131] Specifically, for the relevant content of the initial state judgment model, the initial state judgment result, the test result, and the preset requirements, reference can be made to the relevant descriptions in the above embodiments. To avoid repetition, it will not be elaborated here.

[0132] 550: Use the positive sample data in the positive sample database and the negative sample data in the negative sample database to train the initial state judgment model to obtain a state judgment model.

[0133] The state judgment model is used to obtain a state judgment result based on the test data, and the state judgment result serves as the test result, which is used to characterize whether an error occurs during the execution of the test task.

[0134] In one example, Figure 5 the method can be executed by a robot.

[0135] In one example, the task execution model can be a trained model. In this case, the test task can be an actual operation task executed using the task execution model. Optionally, the task execution model can be a model in training. In this case, the test task can be an actual operation task executed using the task execution model, or a sample task used to train the task execution model, or a detection task used to detect the effect of the task execution model.

[0136] In this embodiment, when the initial state judgment result is inconsistent with the test result, the test data is used as sample data, and the initial state judgment model is trained using this sample data, which can focus on training the parts of the initial state judgment model that are inaccurately judged. In this way, the model convergence effect can be improved and the training efficiency can be increased.

[0137] According to an embodiment of the present application, the model training method further includes: when the test result corresponding to the test data does not meet the preset requirements, correcting the test data to obtain corrected data; using the corrected data to train the task execution model to obtain a trained task execution model.

[0138] Specifically, for the specific content of the corrected data, correcting the test data to obtain the corrected data, and using the corrected data to train the task execution model, as well as the corresponding effects, reference can be made to the relevant descriptions in the above embodiments. To avoid repetition, it will not be elaborated here.

[0139] According to an embodiment of the present application, the task execution model is deployed on a robot, and the test data includes perception data collected in real time during the process of the robot executing a test task based on the task execution model. Among them, step 510 may include: controlling the robot to collect perception data in real time, using the task execution model to obtain a motion control instruction based on the perception data at the current moment; controlling the robot to move based on the motion control instruction to execute the test task.

[0140] Specifically, the specific content and corresponding effects of the perception data, the motion control instruction, and the robot executing the test task using the task execution model may refer to the relevant descriptions in the above embodiments. To avoid repetition, they will not be elaborated here.

[0141] According to an embodiment of the present application, the target time period includes the current time period, and the preset requirement is that the robot does not make mistakes during the execution process of the test task in the current time period. Among them, correcting the test data to obtain corrected data includes: controlling the robot to execute the corresponding target link in the current time period through teleoperation to obtain the test data corresponding to the teleoperation, so as to correct the test data. The test data corresponding to the teleoperation is the corrected data, and the corrected data includes the perception data collected by the robot during the teleoperation process. Further, the model training method further includes: controlling the robot to continue to execute the links after the target link in the test task based on the task execution model.

[0142] Specifically, the specific content and corresponding effects of the teleoperation, the target link, controlling the robot to execute the target link through teleoperation, the links after the target link, and the test data corresponding to the teleoperation may refer to the relevant descriptions in the above embodiments. To avoid repetition, they will not be elaborated here.

[0143] According to an embodiment of the present application, the perception data collected in real time during the process of the robot executing a test task based on the task execution model includes: image data and / or data representing the poses of some joints on the robot.

[0144] According to an embodiment of the present application, the task execution model and the state judgment model belong to the same model.

[0145] As Figure 6 shown, an embodiment of the present application further provides a task execution method. Figure 6 The method can be executed by Figure 1 the computing device 110 in Figure 6 As

[0146] 610: Obtain input data.

[0147] 620: Use the task execution model to execute the target task according to the input data.

[0148] Specifically, the task execution model is obtained according to the model training method provided in any of the above embodiments. For example, it can be obtained according to the model training method provided in the embodiment of Figure 2 or Figure 4a to obtain the task execution model.

[0149] In one example, when the target task is a static task, the target task can be an image recognition task, a speech recognition task, or other static tasks. Taking the image recognition task as an example, the input data can include image data, and the output data can include a numerical value, which is used to indicate whether there is a target object in the image data. For example, when the output data is 1, it indicates that there is a target object in the image data, and when the output data is 0, it indicates that there is no target object in the image data.

[0150] In one example, when the target task is a dynamic task, the target task can be grasping a target object, cleaning a target area, or other dynamic tasks. Taking the task of grasping a target object as an example, the execution entity of the task of grasping a target object can be a mobile device, such as a robot, and a task execution model can be deployed on the robot. The robot can use the task execution model to execute the task of grasping a target object. Specifically, the input data can include the perception data collected by the robot. For the specific content of the perception data, refer to the description in the above embodiments. To avoid repetition, it will not be elaborated here. The perception data is input into the task execution model, and the direct output data of the task execution model can include motion control instructions. The robot controls its movement based on the motion control instructions to execute the task of grasping a target object. Here, during the process of the robot controlling its movement based on the motion control instructions, it will still collect perception data in real time. At this time, the perception data can be used as an intuitive reflection of the motion control instructions on the actions of the robot. Therefore, the perception data at this time can be used as the indirect output data of the task execution model.

[0151] It should be understood that the task execution model is obtained based on the model training method provided in any of the above embodiments. Therefore, for the relevant content and corresponding effects of using the task execution model to execute the target task, refer to the description in the above embodiments. To avoid repetition, it will not be elaborated here.

[0152] As Figure 7 shown, an embodiment of the present application also provides a task execution method, Figure 7 The method of Figure 1 can be executed by the computing device 110 in Figure 7 As shown, the task execution method may include the following content.

[0153] 710: Use the task execution model to execute the target task, and obtain the execution data corresponding to the target time period during the execution of the target task.

[0154] 720: Input the execution data into the status judgment model to obtain a status judgment result, where the status judgment result is used to indicate whether the execution result corresponding to the execution data meets the preset requirements.

[0155] 730: When the status judgment result indicates that the execution result corresponding to the execution data does not meet the preset requirements, send a reminder message.

[0156] Specifically, for the relevant content of the task execution model, target task, target time period, status judgment model, status judgment result, reminder message, and preset requirements, reference can be made to the relevant descriptions in the above embodiments. To avoid repetition, it will not be elaborated here.

[0157] In one example, the execution data corresponds to the above test data and may include the input data and output data of the task execution model in the target time period. For example, the input data may include image data, and the output data may include recognition results. For example, the execution data during the execution of the target task can be obtained at a certain frequency, and the execution data within a certain time period can be obtained each time. In one example, the target time period can be the entire time period for executing the target task, or a certain time period during the execution of the target task, or the current time period during the execution of the target task.

[0158] In one example, the execution result corresponding to the execution data corresponds to the above test result. The execution result corresponding to the execution data can indicate the execution effect of the task execution model for the target task in the target time period, or whether there is an error in the execution process of the target task in the target time period. In one example, the preset requirements may include at least one of the following: in the target time period, the target action in the target task is successfully completed; in the target time period, the time length range of at least one action execution in the target task; no unexpected events occur during the execution process of the target task in the target time period, etc. The execution result can indicate at least one of the following: in the target time period, whether the target action in the target task is successfully completed; in the target time period, the time length of at least one action execution in the target task; whether unexpected events occur during the execution process of the target task in the target time period, etc. When the execution result meets the preset requirements, it indicates that there is no error in the execution process of the target task in the target time period.

[0159] In one example, when the target task is a test task, correspondingly, the execution data can be test data, and the execution result can be a test result. The test task can be an actual operation task executed using the task execution model, or a sample task for training the task execution model, or a detection task for detecting the effect of the task execution model.

[0160] In one example, Figure 7 The method can be executed by a robot, and the execution data can include the perception data collected by the robot.

[0161] In other examples, Figure 7 the method may be executed by other electronic devices, and the task execution model may be an image recognition model.

[0162] In this embodiment, a state judgment model can be used to replace technicians to judge the execution result, which can improve the efficiency of the entire task execution process and reduce costs.

[0163] According to an embodiment of the present application, the state judgment model may be obtained according to the Figure 5 model training method of the embodiment. In this way, the parts where the state judgment model is inaccurate can be trained intensively, so as to improve the model convergence effect and training efficiency. Further, during the process of the task execution model executing the test task, sample data for training the state judgment model is collected, which can improve the acquisition efficiency of the sample data and further shorten the entire training cycle of the state judgment model.

[0164] In an example, the process of the state judgment model obtaining the state judgment result based on the execution data may be synchronized with the process of the task execution model executing the target task (such as obtaining the execution data), or the execution data may be collected first, and then the process of the state judgment model obtaining the state judgment result based on the execution data may be executed later.

[0165] According to an embodiment of the present application, the task execution model is deployed on a robot, the execution data includes perception data collected in real time during the process of the robot executing the target task based on the task execution model, the target time period includes the current time period, and the preset requirement includes that the execution process of the robot for the target task in the current time period is error-free. Among them, the task execution method further includes: based on the reminder information, controlling the robot to execute the corresponding target link in the current time period through teleoperation; controlling the robot to continue to execute the link after the target link in the target task based on the task execution model.

[0166] Specifically, for the specific content and uses of the perception data and teleoperation, reference may be made to the relevant descriptions in the above embodiments. For the specific content of the target link and the link after the target link, reference may be made to the relevant descriptions in the above embodiments. To avoid repetition, they will not be elaborated here.

[0167] In an example, when the state judgment result indicates that the execution result corresponding to the execution data does not meet the preset requirement, it indicates that the robot makes an error in the execution process of the target task in the current time period. A computing device such as the robot can send a reminder information to remind the technician of the error situation of the robot. After receiving the reminder information, the technician can take over the execution process of the target task, such as controlling the robot to execute the corresponding target link in the current time period, that is, the link where the error occurs.

[0168] Further, after the robot successfully executes the target link through teleoperation control, the task execution model can be continuously used to control the robot to execute the subsequent links after the target link in the target task. When executing the subsequent links, the execution data corresponding to the subsequent links can also be obtained, and the state judgment model can be used to judge whether the execution result of the execution data corresponding to the subsequent links meets the preset requirements. In the case where the execution result of the execution data corresponding to the subsequent links does not meet the preset requirements, a reminder message can be sent again to facilitate the technician to take over the execution process of the task.

[0169] In this embodiment, in the case where the execution result does not meet the preset requirements, the robot is controlled to continue executing the target task through teleoperation, so that the link that does not meet the preset requirements can be corrected, that is, correctly executed. Thus, on the premise of improving the execution efficiency of the target task and reducing costs, the completion effect of the target task can be ensured.

[0170] According to an embodiment of the present application, the task execution method further includes: using the execution data corresponding to the teleoperation as correction data to train the task execution model, and obtaining a trained task execution model, where the correction data includes the perception data collected by the robot during the teleoperation process.

[0171] Specifically, through teleoperation, the execution data obtained by the robot based on the task execution model (the execution data corresponding to the target time period in step 710) can be corrected to obtain correction data. Using the correction data to train the task execution model is equivalent to strengthening the training of the weak links of the task execution model, so that the convergence effect of the model can be quickly improved, the accuracy of the task execution model can be improved, the entire training cycle of the model can be reduced, and the training efficiency can be improved.

[0172] The relevant content of the correction data and using the correction data to train the task execution model can refer to the relevant descriptions in the above embodiments. To avoid repetition, it will not be elaborated here.

[0173] The method provided by the embodiments of the present application can improve the training efficiency of models (task execution models and state judgment models). For example, in the training process of a task execution model deployed on a robot, traditional training methods require one week or more to complete the training process of the model. The entire training process may involve factories, technical managers, data collectors, auditors, testing, algorithm training personnel, etc., and the training cycle is long. The method provided by the embodiments of the present application can shorten the training time. In some cases, factory blue-collar workers can teach a robot new skills in half a day. For example, the robot can collect data on error-prone links to train the task execution model and improve the accuracy of the task execution model. Further, a state judgment model can be used to detect the testing process or inference process of the task execution model, and a reminder message can be sent when the task execution model makes an error, so as to liberate human resources, that is, technicians do not need to stay beside the robot all the time and can take over the process of controlling the robot to execute tasks when receiving the reminder message. In addition, during the testing process or inference process of the task execution model, data judged wrongly by the state judgment model can be collected, and these data can be used to further train the state judgment model, which can improve the convergence degree of the state judgment model and simplify the process of collecting sample data.

[0174] In the embodiments of the present application, a foundation model (such as a Cold model or a Warm model) can be used for preheating and a warmer mechanism can be used to obtain the final model, so that only key data needs to be collected to quickly converge. The amount of data is greatly reduced and the training speed is fast.

[0175] In the embodiments of the present application, data can be stored locally to ensure data privacy and save costs at the same time (without using a cloud platform).

[0176] The embodiments of the present application provide a model training method integrating sampling, inference (testing), and training, which can implement functions such as continued inference triggering, takeover, robot pause, supporting the robot to automatically roll back to the historical state, and supporting cross-ontology operations.

[0177] In the embodiments of the present application, in the case where a robot makes an error during autonomous execution of a test task, a technician can take over. In one example, during the process of the technician taking over, the robot can be controlled to roll back to a previous specific position by long pressing a specific operation key, and the robot can be perturbed in a certain posture at this position to facilitate increasing the distribution of robot data. Further, during the takeover process, another operation key can be pressed again to start the teleoperation process, and at the same time, data marking can be triggered to distinguish whether the data source is the robot or the human. Further, after the process of the technician taking over is completed, the robot can be switched to continue executing the task based on the task execution model through an operation key, that is, continue inference.

[0178] In the embodiments of the present application, a collector or a relevant technician can control or supervise multiple robots, thereby further improving the collection efficiency. Specifically, the status judgment model can provide an autonomous status judgment (outputting a status judgment result) function. When the success rate of the autonomous status judgment of the status judgment model is relatively high, the collector does not need to always pay attention to each robot. When an autonomous status judgment fails, the system will request the collector to intervene. Further, the control screen can simultaneously access the real-time screens of multiple robots, and the control signal can be switched among multiple robots. In addition to the technician's independent judgment, the robot can also automatically judge whether a technician needs to take over based on the robo gated algorithm. In this way, the method provided by the embodiments of the present application can further improve the human efficiency, quickly collect a large amount of data, and improve the effect.

[0179] Exemplary device

[0180] The embodiments of the present application further provide a mobile device, which includes a control module for executing the model training method or the task execution method provided in the above embodiments.

[0181] In one example, the mobile device may include a robot, a vehicle, or other devices that can execute tasks. The robot can be a humanoid robot, a wheeled robot, a tracked robot, or other types of robots.

[0182] For the specific functions and effects of the robot provided by the embodiments of the present application, reference may be made to the descriptions in the above method embodiments. To avoid repetition, they will not be elaborated here.

[0183] Figure 8 The following shows a schematic flowchart of a model training device provided by an exemplary embodiment of the present application. As Figure 8 shown, the model training device 800 may include: an execution module 810, a correction module 820, and a training module 830.

[0184] The execution module 810 is used to execute a test task using the task execution model and obtain the test data corresponding to the target time period during the execution of the test task. The correction module 820 is used to correct the test data to obtain corrected data when the test result corresponding to the test data does not meet the preset requirements. The training module 830 is used to train the task execution model using the corrected data to obtain a trained task execution model.

[0185] An embodiment of the present application provides a model training device. By testing a task execution model, that is, using the task execution model to execute a test task, test data corresponding to a target time period during the execution of the test task is obtained. When the test result corresponding to the test data does not meet the preset requirements, for example, when the test result indicates an error during the execution of the test task, the test data is corrected to obtain corrected data, and the task execution model is trained using the corrected data, and a trained task execution model can be obtained. In this way, attention can be paid to the links where the execution effect of the task execution model is not good during the test process, that is, when the test result does not meet the preset requirements, the test data is corrected to obtain corrected data. Further training the model using the corrected data can quickly improve the convergence effect of the model, reduce the entire training cycle of the model, and improve the training efficiency.

[0186] According to an embodiment of the present application, the task execution model is deployed on a robot, and the test data includes perception data collected in real time during the process of the robot executing the test task based on the task execution model. The execution module 810 is configured to: control the robot to collect perception data in real time, use the task execution model to obtain a motion control instruction based on the perception data at the current moment; and control the robot to move based on the motion control instruction to execute the test task.

[0187] According to an embodiment of the present application, the target time period includes the current time period, and the preset requirements include that the robot does not make an error during the execution process of the test task in the current time period. The correction module 820 is configured to: control the robot to execute the corresponding target link in the current time period through teleoperation to obtain the test data corresponding to the teleoperation, so as to implement the correction of the test data, where the test data corresponding to the teleoperation is the corrected data, and the corrected data includes the perception data collected by the robot during the teleoperation process. The execution module 810 is further configured to: control the robot to continue to execute the link after the target link in the test task based on the task execution model.

[0188] According to an embodiment of the present application, the model training device 800 further includes an input / output module 840, configured to: input the test data into a state judgment model to obtain a state judgment result, where the state judgment result is used as the test result to represent whether the robot makes an error during the execution process of the test task in the current time period; and send a reminder message when the test result does not meet the preset requirements.

[0189] According to an embodiment of the present application, the model training device 800 further includes a first storage module 850, which is configured to: when the test result meets the preset requirements, store the test data as positive sample data in the positive sample database; when the test result does not meet the preset requirements, store the test data as negative sample data in the negative sample database, and store the corrected data as positive sample data in the positive sample database. The training module 830 is further configured to: train the initial state judgment model by using the positive sample data in the positive sample database and the negative sample data in the negative sample database to obtain a state judgment model, where the state judgment model is used to obtain a state judgment result based on the test data, and the state judgment result is used as the test result.

[0190] According to an embodiment of the present application, the model training device 800 further includes an input module 860, which is configured to: input the test data into the initial state judgment model to obtain an initial state judgment result, where the initial state judgment result is used to represent whether there is an error in the execution process of the test task by the initial state judgment model during the target period, and the test result is used to represent whether there is actually an error in the execution process of the test task during the target period. The model training device 800 further includes a second storage module 870, which is configured to: when the test result meets the preset requirements and the initial state judgment result is inconsistent with the test result, store the test data as positive sample data in the positive sample database; when the test result does not meet the preset requirements and the initial state judgment result is inconsistent with the test result, store the test data as negative sample data in the negative sample database. The training module 830 is further configured to: train the initial state judgment model by using the positive sample data in the positive sample database and the negative sample data in the negative sample database to obtain a state judgment model, and the state judgment model is used to obtain a state judgment result based on the test data, and the state judgment result is used as the test result, which is used to represent whether there is an error in the execution process of the test task.

[0191] According to an embodiment of the present application, the perception data collected in real time during the process of the robot executing the test task based on the task execution model includes: image data and / or data representing the poses of some joints on the robot.

[0192] According to an embodiment of the present application, the task execution model and the state judgment model belong to the same model.

[0193] It should be understood that the operations and functions of the execution module 810, correction module 820, training module 830, input / output module 840, first storage module 850, input module 860, and second storage module 870 in the above embodiments can refer to the descriptions in the model training method provided in the above Figure 2 、 Figure 4a or Figure 5 embodiments. To avoid repetition, they will not be elaborated here.

[0194] Figure 9 The following is a schematic flowchart of a model training device provided by another exemplary embodiment of the present application. As Figure 9 shown, the model training device 900 may include: an execution module 910, an input module 920, a storage module 930, and a training module 940.

[0195] The execution module 910 is configured to execute a test task by using a task execution model, and obtain test data corresponding to a target time period during the execution of the test task. The input module 920 is configured to input the test data into an initial state judgment model to obtain an initial state judgment result, where the initial state judgment result is used to represent whether an error occurs in the execution process of the test task by the initial state judgment model during the target time period. The storage module 930 is configured to: when the test result corresponding to the test data meets a preset requirement and the initial state judgment result is inconsistent with the test result, store the test data as positive sample data in a positive sample database, where the test result is used to represent whether an error actually occurs in the execution process of the test task during the target time period; when the test result does not meet the preset requirement and the initial state judgment result is inconsistent with the test result, store the test data as negative sample data in a negative sample database. The training module 940 is configured to train the initial state judgment model by using the positive sample data in the positive sample database and the negative sample data in the negative sample database to obtain a state judgment model, where the state judgment model is configured to obtain a state judgment result based on the test data, and the state judgment result is used as the test result to represent whether an error occurs in the execution process of the test task.

[0196] The embodiment of the present application provides a model training device, which uses the test data when the initial state judgment result is inconsistent with the test result as sample data, and trains the initial state judgment model by using the sample data, so as to focus on training the parts where the initial state judgment model is inaccurate, thereby improving the model convergence effect and training efficiency.

[0197] According to an embodiment of the present application, the model training device 900 further includes: a correction module 950, configured to correct the test data to obtain corrected data when the test result corresponding to the test data does not meet the preset requirement. The training module 940 is further configured to train the task execution model by using the corrected data to obtain a trained task execution model.

[0198] According to an embodiment of the present application, the task execution model is deployed on a robot, and the test data includes the perception data collected in real time during the process in which the robot executes a test task based on the task execution model. Among them, the execution module 910 is used to: control the robot to collect perception data in real time, use the task execution model to obtain a motion control instruction based on the perception data at the current moment; control the robot to move based on the motion control instruction to execute the test task.

[0199] According to an embodiment of the present application, the target time period includes the current time period, and the preset requirement includes that the robot does not make mistakes during the execution process of the test task in the current time period. Among them, the correction module 950 is used to control the robot to execute the corresponding target link in the current time period through teleoperation to obtain the test data corresponding to the teleoperation, so as to realize the correction of the test data. The test data corresponding to the teleoperation is the correction data, and the correction data includes the perception data collected by the robot during the teleoperation process. The execution module 910 is further used to: control the robot to continue to execute the link after the target link in the test task based on the task execution model.

[0200] According to an embodiment of the present application, the perception data collected in real time during the process in which the robot executes a test task based on the task execution model includes: image data and / or data representing the poses of some joints on the robot.

[0201] According to an embodiment of the present application, the task execution model and the state judgment model belong to the same model.

[0202] It should be understood that the operations and functions of the execution module 910, input module 920, storage module 930, training module 940, and correction module 950 in the above embodiments can be referred to the descriptions in the model training method provided in the above Figure 2 、 Figure 4a or Figure 5 embodiments. To avoid repetition, they will not be elaborated here.

[0203] Figure 10 The figure shows a schematic flowchart of a task execution device provided by an exemplary embodiment of the present application. As Figure 10 shown, the task execution device 1000 may include: an acquisition module 1010 and an execution module 1020.

[0204] The acquisition module 1010 is used to acquire input data; the execution module 1020 is used to execute a target task according to the input data by using the task execution model.

[0205] It should be understood that the operations and functions of the acquisition module 1010 and the execution module 1020 in the above embodiments can be referred to the descriptions in the task execution method provided in the above Figure 6 embodiments. To avoid repetition, they will not be elaborated here.

[0206] Figure 11 The figure shows a flow chart of a task execution device provided by an exemplary embodiment of the present application. Figure 11 As shown, the task execution device 1100 may include: an execution module 1110 , an input module 1120 and an output module 1130 .

[0207] The execution module 1110 is configured to execute the target task using the task execution model and obtain execution data corresponding to the target period during the execution of the target task. The input module 1120 is configured to input the execution data into the state judgment model to obtain a state judgment result, wherein the state judgment result indicates whether the execution result corresponding to the execution data meets the preset requirements. The output module 1130 is configured to issue a reminder message if the state judgment result indicates that the execution result corresponding to the execution data does not meet the preset requirements.

[0208] The embodiment of the present application provides a task execution device that uses a state judgment model to replace technical personnel to judge the execution results, which can improve the efficiency of the entire task execution process and reduce costs.

[0209] According to an embodiment of the present application, the state judgment model is based on the above Figure 5 The model training method of the embodiment is obtained.

[0210] According to one embodiment of the present application, a task execution model is deployed on a robot. The execution data includes real-time sensory data collected by the robot during execution of a target task based on the task execution model. The target period includes the current period. The preset requirement includes the robot's error-free execution of the target task during the current period. The execution module 1110 is further configured to: based on the reminder information, remotely control the robot to execute the target step corresponding to the current period; and control the robot to continue executing the step of the target task subsequent to the target step based on the task execution model.

[0211] According to one embodiment of the present application, the task execution device 1100 also includes: a training module 1140, which is used to train the task execution model using the execution data corresponding to the remote operation as correction data to obtain a trained task execution model, wherein the correction data includes perception data collected by the robot during the remote operation process.

[0212] It should be understood that the operations and functions of the execution module 1110, the input module 1120, the output module 1130, and the training module 1140 in the above embodiment can refer to the above Figure 7 To avoid repetition, the description of the task execution method provided in the embodiment will not be repeated here.

[0213] Figure 12FIG2 is a block diagram of an electronic device 1200 for executing a model training method or a task execution method according to an exemplary embodiment of the present application. The electronic device 1200 may be a server, a robot, a server or other device interacting with a robot, or other mobile device.

[0214] Reference Figure 12 The electronic device 1200 includes a processing component 1210, which further includes one or more processors, and a memory resource represented by a memory 1220 for storing instructions executable by the processing component 1210, such as an application. The application stored in the memory 1220 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1210 is configured to execute the instructions to perform the above-mentioned model training method or task execution method.

[0215] The electronic device 1200 may further include a power supply component configured to perform power management of the electronic device 1200, a wired or wireless network interface configured to connect the electronic device 1200 to a network, and an input / output (I / O) interface. The electronic device 1200 may be operated based on an operating system stored in the memory 1220, such as Windows Server 200. TM , MacOS X TM , Unix TM , Linux TM , FreeBSD TM or similar.

[0216] A non-temporary computer-readable storage medium, when the instructions in the storage medium are executed by the processor of the above-mentioned electronic device 1200, enables the above-mentioned electronic device 1200 to execute a model training method or a task execution method.

[0217] A computer program product includes a computer program. When the computer program is executed by a processor of a computer device, the computer device is enabled to execute the model training method or task execution method provided in any of the above embodiments.

[0218] All of the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, and will not be described in detail here.

[0219] Those of ordinary skill in the art will realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application.

[0220] Those skilled in the art can clearly understand that for the sake of convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0221] In several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of devices or units can be electrical, mechanical, or other forms.

[0222] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0223] In addition, the functional units in each embodiment of this application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0224] When the above-mentioned function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program verification codes.

[0225] It should be noted that in the description of this application, the terms "first", "second", "third", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "a plurality of" is two or more.

[0226] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data that have been authorized by the user or fully authorized by all parties. And the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.

[0227] The above are only the preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent replacements, etc. made within the spirit and principle of this application shall be included in the protection scope of this application.

Claims

1. A model training method, characterized in that, Including: Executing a test task using a task execution model, and obtaining test data corresponding to a target time period during the execution of the test task; When the test result corresponding to the test data does not meet the preset requirements, correcting the test data to obtain corrected data; Training the task execution model using the corrected data to obtain a trained task execution model.

2. The model training method according to claim 1, wherein The task execution model is deployed on a robot, and the test data includes perception data collected in real time during the process of the robot executing the test task based on the task execution model. Among them, executing a test task using a task execution model and obtaining test data corresponding to a target time period during the execution of the test task includes: Controlling the robot to collect perception data in real time, and using the task execution model to obtain a motion control instruction based on the perception data at the current moment; Controlling the robot to move based on the motion control instruction to execute the test task.

3. The model training method according to claim 2, wherein The target time period includes the current time period, and the preset requirements include that the robot does not make an error during the execution process of the test task in the current time period. Among them, correcting the test data to obtain corrected data includes: Controlling the robot to execute the corresponding target link in the current time period through teleoperation to obtain the test data corresponding to the teleoperation, so as to realize the correction of the test data. The test data corresponding to the teleoperation is the corrected data, and the corrected data includes the perception data collected by the robot during the teleoperation process. Among them, the model training method further includes: Controlling the robot to continue to execute the link after the target link in the test task based on the task execution model.

4. The model training method according to claim 3, wherein, Also including: Inputting the test data into a state judgment model to obtain a state judgment result, where the state judgment result is used as the test result to indicate whether the robot makes an error during the execution process of the test task in the current time period; When the test result does not meet the preset requirements, sending a reminder message.

5. The model training method according to claim 1, characterized in that, Also including: When the test result meets the preset requirements, storing the test data as positive sample data in a positive sample database; When the test result does not meet the preset requirements, storing the test data as negative sample data in a negative sample database, and storing the corrected data as positive sample data in the positive sample database; Training an initial state judgment model using the positive sample data in the positive sample database and the negative sample data in the negative sample database to obtain a state judgment model, where the state judgment model is used to obtain a state judgment result based on test data, and the state judgment result is used as the test result.

6. The model training method according to claim 1, wherein Also including: Input the test data into the initial state judgment model to obtain an initial state judgment result, where the initial state judgment result is used to represent whether there is an error in the execution process of the test task by the initial state judgment model during the target period, and the test result is used to represent whether there is actually an error in the execution process of the test task during the target period; In the case where the test result meets the preset requirements and the initial state judgment result is inconsistent with the test result, store the test data as positive sample data in the positive sample database; In the case where the test result does not meet the preset requirements and the initial state judgment result is inconsistent with the test result, store the test data as negative sample data in the negative sample database; Use the positive sample data in the positive sample database and the negative sample data in the negative sample database to train the initial state judgment model to obtain a state judgment model. The state judgment model is used to obtain a state judgment result based on the test data, and the state judgment result is used as the test result to represent whether there is an error in the execution process of the test task.

7. The model training method according to claim 6, wherein The task execution model and the state judgment model belong to the same model.

8. The model training method according to any one of claims 1 to 7, characterized in that The task execution model is deployed on a robot. The test data includes perception data collected in real time during the process of the robot executing the test task based on the task execution model. The perception data collected in real time during the process of the robot executing the test task based on the task execution model includes: image data and / or data representing the poses of some joints on the robot.

9. A task execution method, characterized in that, Includes: Obtain input data; Use the task execution model to execute the target task according to the input data, where the task execution model is obtained by the model training method according to any one of claims 1 to 8.

10. A mobile device, characterized in that, Includes a control module, and the control module is used to execute the model training method according to any one of claims 1 to 8 or the task execution method according to claim 9.

11. An electronic device, characterized in that, Includes: A processor; A memory for storing instructions executable by the processor, where the processor is used to execute the model training method according to any one of claims 1 to 8 or the task execution method according to claim 9.

12. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is used to execute the model training method according to any one of claims 1 to 8 or the task execution method according to claim 9.

13. A computer program product, characterized in that, The computer program product includes a computer program. When the computer program is executed by the processor of the computer device, the computer device can execute the model training method according to any one of claims 1 to 8 or the task execution method according to claim 9.

Images