A model training method, device and robotic arm system

Abstract

The embodiment of the present application discloses a model training method, device and robot system; the present application displays a data collection page, displays a slave robot and a target object, and then controls the slave robot to operate on the target object in a virtual environment in response to a control instruction for the slave robot to perform a target task, and then collects the posture data of the slave robot in the virtual environment and the image information of the target object until the slave robot completes the target task to obtain a training data set, which is used to train a neural network model corresponding to the target task. The scheme can map the slave robot in the real environment to the virtual environment through 3D modeling and other methods, which can not only train the neural network model, but also the slave robot and the target object in the virtual environment, which is conducive to reducing the training cost of the neural network model and improving the flexibility and versatility of the training.

Description

Technical Field

[0001] The present invention relates to the technical field of robotic arms, and in particular to a model training method, device and robotic arm system. Background Art

[0002] In recent years, with the technological breakthroughs based on large models and AI, the application of general artificial intelligence (AGI) in the field of robots (or robotic arms) has become more and more extensive. Through a large number of model training for robots (or robotic arms), robots (or robotic arms) in real environments can be made intelligent, so that they can interact with the environment like humans and perform various tasks. Summary of the invention

[0003] The present application embodiment provides a model training method, including:

[0004] Displaying a data acquisition page, which includes a slave robot arm and a target object in a virtual environment;

[0005] In response to a control instruction for the slave robot arm, controlling the slave robot arm to operate on the target object in the virtual environment to perform a target task;

[0006] The posture data of the slave robot in the virtual environment and the image information of the target object are collected until the slave robot completes the target task to obtain a training data set, which is used to train a neural network model corresponding to the target task.

[0007] The embodiment of the present application provides a model training device, the model training device has a display interface, the display interface can display a virtual slave robot arm and a virtual target object, and can display a control module and a collection module;

[0008] The control module has at least one line of code and is used to control the slave robot arm to operate on the target object in the virtual environment in response to a control instruction for the slave robot arm to perform a target task;

[0009] The acquisition module has at least one line of code and is used to collect the posture data of the slave robot in the virtual environment and the image information of the target object until the slave robot completes the target task to obtain a training data set, which is used to train a neural network model corresponding to the target task.

[0010] The present application embodiment provides a robotic arm system using a model training method, including:

[0011] A control device, comprising a display interface, the display interface being capable of displaying a virtual slave robot arm and a virtual target object, wherein an actuator is disposed at the end of the slave robot arm, the actuator having m degrees of freedom, and the actuator being used to operate the target object;

[0012] A main robotic arm, comprising N joint motors and a handle, wherein the N joint motors enable the handle to have m degrees of freedom, and the handle is configured to allow a user to remotely operate an actuator;

[0013] When the actuator is remotely operated by the handle, the control device obtains the posture data of the robot arm, the angular positions of N joint motors and image information including the target object, and inputs them into the neural network model to train the neural network model.

[0014] The beneficial effects of the present application are as follows: the present application displays the slave manipulator and the target object by displaying a data acquisition page; then controls the slave manipulator to operate on the target object in a virtual environment in response to a control instruction for the slave manipulator to perform a target task; then collects the position data of the slave manipulator in the virtual environment and the image information of the target object until the slave manipulator completes the target task to obtain a training data set, which is used to train a neural network model corresponding to the target task. Since the scheme can map the slave manipulator in the real environment to the virtual environment through 3D modeling and other methods, it can not only train the neural network model, but also the slave manipulator and the target object in the virtual environment, which is conducive to reducing the training cost of the neural network model and improving the flexibility and versatility of the training. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 It is a structural schematic diagram of an embodiment of a robotic arm system provided in an embodiment of the present application;

[0017] Figure 2 It is a flowchart of an embodiment of a model training method provided in an embodiment of the present application;

[0018] Figure 3 This is a page diagram of an embodiment of a data collection page provided in an embodiment of the present application;

[0019] Figure 4 It is a structural schematic diagram of an embodiment of a model training device provided in an embodiment of the present application;

[0020] Figure 5 It is a structural schematic diagram of an embodiment of a model training device provided in an embodiment of the present application;

[0021] Figure 6 It is a structural diagram of an embodiment of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0023] Combination Figure 1 , the robot system 100 may include a control device 110 and a main robot 120. The main robot 120 described in the present application may refer to a traditional mechanical structure fixed at a certain station to perform certain functions, such as the main robot 120 being a collaborative robot, or other emerging mechanical structures (or a part thereof), such as the main robot 120 being a humanoid robot (or its arm).

[0024] Among them, the control device 110 can be a server or a terminal or other device. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud storage, and cloud computing services such as big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application does not limit this.

[0025] Further, the control device 110 includes a display interface A1, which can display a virtual slave manipulator 130 and a virtual target object. An actuator is provided at the end of the slave manipulator 130, and the actuator has m degrees of freedom. The actuator is used to operate the target object. Among them, the slave manipulator 130 described in the present application can be understood as a mapping (or 3D model) of the master manipulator 120 in a virtual environment, so the slave manipulator 130 is the same or similar to the master manipulator 120. The virtual target object can be understood as a mapping of the operation object in the virtual environment for the manipulator system 100 to perform the target task. For example, taking the target task of folding clothes as an example, the virtual target object can include at least one image of clothes that need to be folded mapped to the virtual environment, or, taking the target task of washing dishes as an example, the virtual target object can include at least one dish that needs to be washed in the virtual environment. The mapping, and so on. In the virtual environment displayed on the display interface A1, the actuator provided at the end of the slave manipulator 130 can operate the virtual target object to perform various target tasks.

[0026] The main mechanical arm 120 includes N joint motors and a handle, the N joint motors enable the handle to have m degrees of freedom, and the handle is configured for a user to remotely operate an actuator. The handle may include a handle body, a steering gear connected to the handle body, and a wrench connected to the steering gear, and the steering gear allows the wrench to rotate relative to the handle body.

[0027] Teleoperation can be understood as the process of controlling a device or system by remote means. This technology allows operators to perform various operations and controls through wireless communication or network connection without direct contact with the target device. Teleoperation is common in applications such as drones, remote-controlled vehicles, robots, and remote-operated devices. In this solution, teleoperation can be understood as the user operating the master robotic arm to achieve the target task of the teleoperated virtual slave robotic arm.

[0028] Furthermore, when the handle remotely operates the actuator, the control device 110 can obtain the posture data of the robot arm, the angular positions of N joint motors and image information including the target object, and input them into the neural network to train the neural network model, as described in the following text for details.

[0029] Among them, the neural network model can be understood as an action block algorithm model suitable for the field of robot (or robotic arm) technology. The neural network model can use Transformers (a deep learning model architecture) to implement the action block strategy and train it as a conditional variational autoencoder (CVAE) to capture the variability in human data. CVAE is a deep learning model that combines the concepts of variational autoencoder (VAE) and conditional generation. When generating data, CVAE not only relies on sampling points in the latent space, but also introduces an additional conditional input, so that samples that meet specific conditions can be generated. The basic structure of CVAE consists of two parts: an encoder and a decoder. The encoder compresses the input data into low-dimensional latent variables, while the decoder reconstructs or generates data based on these latent variables and conditional inputs.

[0030] In some embodiments, a sensor may be provided on the handle of the main mechanical arm 120, and the sensor has a first level signal when the user holds the handle, and has a second level signal when the user does not hold the handle. The control device 110 may be configured to obtain the level signal of the sensor, and lock the main mechanical arm when the level signal changes from the first level signal to the second level signal. In this way, the level signal output by the sensor can ensure that the main mechanical arm 120 is locked when the user leaves the main mechanical arm 120, thereby preventing the main mechanical arm from falling or dropping.

[0031] In some embodiments, in combination Figure 1 , the method of controlling the master robot 120 and the virtual slave robot 10C to perform model training is further described from the perspective of the control device 110 in the robot system 100. Accordingly, as Figure 2 As shown, the specific process of the model training method for the robotic arm system 100 is as follows:

[0032] 210. Display the data collection page.

[0033] The data collection page includes the slave robot and the target object in the virtual environment. The virtual environment can be understood as an operating environment or space generated and presented by virtual reality technology, which is different from the real environment. Figure 3 Figure 2 shows a schematic diagram of the data collection page. Figure 3 As shown, the data acquisition page can be understood as a display carrier of various virtual images in the virtual environment, such as a page displayed on a computer display screen, such as a display page provided by a virtual reality display device. The data acquisition page can display various types of slave manipulators (including slave manipulator 1, slave manipulator 2, slave manipulator 3, etc.) in the virtual environment, so that the user can select an adapted slave manipulator from them according to the requirements of the target task. The data acquisition page can also display various types of target objects (including target object 1, target object 2, target object 3, etc.) in the virtual environment, so that the user can select an adapted target object from them according to the requirements of the target task.

[0034] There are many ways to display the data acquisition page. For example, virtual reality technology can be used to map the slave robot of the robot system in the real environment to the virtual environment to generate a virtual slave robot. At the same time, the operation object (i.e., the target object) for the robot system 100 to perform the target task can also be mapped to the virtual environment to generate a virtual target object. Based on the virtual slave robot and the virtual target object, a data acquisition page is generated and displayed.

[0035] There are many ways to generate a virtual slave robotic arm and a virtual target object. For example, first basic data of the slave robotic arm in the real environment and second basic data of the target object in the real environment can be obtained; then, based on the first basic data, a virtual image of the slave robotic arm in the real environment is created in the virtual environment to obtain the virtual slave robotic arm, and based on the second basic data, a virtual image of the target object in the real environment is created in the same virtual environment to obtain the virtual target object.

[0036] The first basic data can be understood as the size, position, structure, structural relationship data, position relationship data, internal line data or other inherent data of the slave robot in the real environment, etc. There are many ways to obtain the first basic data of the slave robot in the real environment, for example, the first basic data of the slave robot in the real environment sent by the terminal or the client can be received, or the slave robot in the real environment can be scanned and the scanned data obtained can be used as the first basic data, or the slave robot in the real environment can be three-dimensionally modeled to obtain three-dimensional image data of the slave robot, and the three-dimensional image data of the slave robot can be used as the first basic data, etc.

[0037] The second basic data can be understood as the size, position, structure, category, identification, position relationship data / connection relationship data of the target object in the real environment relative to the robot arm in the real environment, or other inherent information of the target object, etc. There are many ways to obtain the second basic data, for example, the second basic data of the target object sent by the terminal or the client can be received, or the target object in the real environment can be scanned and the scanned data obtained can be used as the second basic data, or the target object in the real environment can be three-dimensionally modeled to obtain the three-dimensional image data of the target object, and the three-dimensional image data of the target object can be used as the second basic data, etc.

[0038] For ease of description, the slave robot in the virtual environment is referred to as the slave robot, and the target object in the virtual environment is referred to as the target object. Thus, the virtual environment at least includes the slave robot and the target object, and the slave robot can operate the target object under the control of the control instruction to perform a given target task.

[0039] It can be understood that the target object can be determined according to the target task actually performed by the robot system 200, and the target object may be different for different target tasks. For example, if the target task is to wash dishes, the target object may be a bowl or a plate; if the target task is to fold clothes, the target object may be clothes; if the target task is to peel an apple, the target object may be a fruit knife or an apple, and so on.

[0040] 220. In response to a control instruction for the slave robotic arm, control the slave robotic arm to operate on the target object in the virtual environment to perform a target task.

[0041] Among them, the control instruction can be understood as the interactive instruction set by the user in the real environment for the slave robot in the virtual environment, and the object of the control instruction is the slave robot in the virtual environment. The process of the slave robot operating the target object in the virtual environment can be understood as the process of the slave robot making a series of actions for the target object under the control of the control instruction. For any target task, the process of the slave robot executing the target task may include actions of multiple time steps. If each control instruction corresponds to controlling the action of the slave robot in one time step, multiple control instructions are required for the slave robot to complete the target task.

[0042] Among them, an actuator may be provided at the end of the slave robot arm, and the control instruction may include a motion instruction and an operation instruction, wherein the motion instruction controls the slave robot arm to move in the virtual environment, and the operation instruction controls the actuator to operate the target object in the virtual environment. As mentioned above, there may be many types of actuators, such as a gripper, a suction cup, or other mechanisms that can perform target operations.

[0043] In some embodiments, in response to the control instruction for the slave robotic arm, before controlling the slave robotic arm to operate the target object in the virtual environment, a control instruction for the slave robotic arm may be generated. There may be multiple ways to generate the control instruction for the slave robotic arm, for example, the control instruction for the slave robotic arm may be generated through the posture data of the master robotic arm.

[0044] The slave robot in the virtual environment is remotely operated by the master robot in the real environment. The master robot includes N joint motors and a handle. The N joint motors make the degree of freedom of the handle the same as the degree of freedom of the actuator. The handle is configured for the user to remotely operate the actuator. In the process of the master robot in the real environment remotely operating the slave robot in the virtual environment, the slave robot moves following the movement of the master robot. In the process of the user operating the handle of the master robot, the angles of the N joint motors change following the user's movements. At the same time, the user's movements of operating the handle and the angles of the N joint motors are automatically mapped to the virtual environment, so that the slave robot in the virtual environment can perform the same movements as the master robot, thereby realizing remote operation of the slave robot in the virtual environment.

[0045] The posture data of the master robot may include the angular positions of the N joint motors and the status information of the handles. There are many ways to generate control instructions for the slave robot using the posture data of the master robot. For example, the angular positions of the N joint motors and the status information of the handles may be obtained; motion instructions may be generated based on the angular positions, and operation instructions may be generated based on the status information, and the motion instructions and operation instructions may be used as control instructions for the slave robot.

[0046] The handle may include a handle body, a steering gear connected to the handle body, and a wrench connected to the steering gear, the steering gear allows the wrench to rotate relative to the handle body, and the state information includes the angular position of the steering gear. The user can operate the wrench to rotate the wrench relative to the handle body.

[0047] In some embodiments, current loop values ​​of N joint motors may be acquired, and based on the current loop values, the N joint motors may be controlled to perform gravity compensation.

[0048] Among them, the current loop value can be understood as the value of the current loop. The so-called current loop, as the innermost control loop in the servo control system, can ensure that the current in the motor winding can quickly and accurately track the given reference value. The performance of the current loop directly affects the dynamic response and control accuracy of the entire servo system.

[0049] In some embodiments, the control instructions may come from a variety of sources, including the posture data of the master robot in the real environment, the augmented posture data after data augmentation of the historical posture data of the master robot, the code language directly written by the developer based on the target task, the voice information corresponding to the user's real-time needs, etc. Different sources correspond to different control instruction generation methods. Specifically, the control instructions for the slave robot may also include generation using at least one of the following three methods:

[0050] (1) Generate control instructions through augmented pose data.

[0051] For example, the historical posture data of the master robot can be obtained, and the historical posture data includes the historical angular positions of the N joint motors of the master robot and the historical state information of the handle, and the historical angular positions are augmented to obtain the augmented angular positions, and the historical state information is augmented to obtain the augmented state information, thereby obtaining the augmented posture data. Based on the augmented angular positions, motion instructions can be generated to control the movement of the slave robot in the virtual environment; and based on the augmented state information, operation instructions can be generated to control the actuator to operate the target object in the virtual environment.

[0052] Among them, the historical posture data can be understood as the posture data of the main robot arm recorded in the process of the user operating the main robot arm to perform the target task in the real environment. The historical posture data may include the historical angular positions of the N joint motors of the main robot arm and the historical state information of the handle. Among them, the historical angular position can reflect the angle information, torque information, etc. corresponding to each time step of each joint that constitutes the main robot arm during the movement process, and the historical angular position can be used as the source of generating motion instructions; the historical state information can reflect the state information of the handle at each time step when the user uses the handle to operate the target object in the real environment or virtual environment, and the historical state information can be used as the source of generating operation instructions.

[0053] Among them, data augmentation is a technique that increases the number of original data sets by applying a series of transformations, aiming to improve the effect of model training. This method helps the model better adapt to sample changes and noise by introducing new data variants, thereby improving the generalization ability and robustness of the model and reducing the possibility of overfitting.

[0054] It should be noted that the number of historical pose data may be a natural number greater than or equal to 1, and the augmented pose data obtained after data augmentation based on the historical pose data may be a natural number greater than or equal to 1. For example, if the number of historical pose data is two, they are historical pose data A and historical pose data B. Historical pose data A includes historical angular position A and historical state information A, and historical pose data B includes historical angular position B and historical state information B. The augmented pose data obtained after data augmentation of historical pose data A may include augmented pose data A-1, augmented pose data A-2, ..., augmented position data An, and the augmented pose data obtained after data augmentation of historical pose data B may include augmented pose data B-1, augmented pose data B-2, ..., augmented pose data Bn, and n may be a natural number greater than 2. The augmented pose data A-1 includes the augmented angular position A-1 and the augmented state information A-1, the augmented pose data A-2 includes the augmented angular position A-2 and the augmented state information A-2, the augmented pose data B-1 includes the augmented angular position B-1 and the augmented state information B-1, and the contents contained in the remaining augmented pose data are similar and will not be repeated one by one.

[0055] In this scheme, historical pose data collected in the real environment is obtained, and data augmentation is performed based on the historical pose data to obtain augmented pose data. Control instructions for the slave robot in the virtual environment are then generated based on the augmented pose data, thereby reducing the data volume requirement for historical pose data in the real environment, making full use of limited historical pose data to generate new pose data, increasing the diversity of samples, and using the new pose data (augmented pose data) as the basis for generating control instructions.

[0056] (2) Generate control instructions through the code language written directly by developers based on the target task.

[0057] For example, a control code of a slave robot arm may be obtained, the control code including a motion control code and an operation control code. Based on the motion control code, a motion instruction may be generated to control the slave robot arm to move in a virtual environment; and based on the operation control code, an operation instruction may be generated to control the actuator to operate a target object in the virtual environment.

[0058] The control code can be understood as the program code written by the developer for the actions that the slave robot needs to perform according to the requirements of the target task. The control code is used to control the slave robot to perform the target task. The control code can be converted into control instructions for the slave robot in the virtual environment. Therefore, running the control code can realize the control of the slave robot in the virtual environment.

[0059] In this solution, the control code of the slave robot is obtained, and control instructions are generated based on the control code, including motion instructions and operation instructions. This process does not need to rely on the historical posture data of the master robot in the real environment, and does not require the user to teach based on the master robot in the real environment, which greatly reduces the dependence on the real environment in the process of determining the control instructions, and provides a more convenient and efficient way for the collection of subsequent training data.

[0060] (3) Generate control instructions through voice information corresponding to the user’s real-time needs.

[0061] For example, the user's voice information can be collected; the voice information is subjected to voice recognition to obtain at least one of motion information and operation information. The motion information can be understood as the motion control information for the slave robot in the virtual environment contained in the user's voice information, and the operation information can be understood as the operation control information for requiring the slave robot to operate the target object contained in the user's voice information. Accordingly, motion instructions are generated based on the motion information to control the movement of the slave robot in the virtual environment; and / or, operation instructions are generated based on the operation information to control the actuator to operate the target object in the virtual environment. The motion instructions and / or operation instructions can be used as control instructions.

[0062] Among them, voice information can be understood as the control instructions that the user requires the robot arm to complete the corresponding task through voice output. For example, the user's voice information can include "put the apple in the refrigerator" or "charge the mobile phone on the table, etc." There are many ways to collect the user's voice information. For example, the user's voice information can be collected through a voice collection device, or the user's voice information sent by the client or terminal can be received, or the control information of the target modality (such as video, text or other modalities other than voice modality) sent by the user can be received, and the control information is converted into voice information, etc.

[0063] In this application, the user's voice information is collected and then control instructions are generated based on the user's voice information. This process does not need to rely on the historical posture data of the main robot arm in the real environment, and the user does not need to teach based on the main robot arm in the real environment. This greatly reduces the dependence on the real environment in the process of determining the control instructions, and provides a more convenient and efficient idea for the collection of subsequent training data.

[0064] It can be understood that the control instruction of the slave robot arm can be determined by any one of the above four methods, or by at least two of the above four methods. If at least two of the above four methods are collected for determination, the reliability of generating the control instruction for controlling the slave robot arm can be increased.

[0065] 230. Collect the posture data of the slave robot in the virtual environment and the image information of the target object until the slave robot completes the target task to obtain a training data set.

[0066] Among them, the training data set is used to train the neural network model corresponding to the target task.

[0067] The posture data may include the angular positions of the M shutdown motors of the slave robot in the virtual environment, and may further include the state information of the actuator installed at the end of the slave robot in the virtual environment. The type of state information may be related to the actuator. When the actuator is a gripper, the state information may be the degree of opening and closing of the gripper. When the actuator is a suction cup, the state information may be the vacuum degree of the suction cup, and so on.

[0068] The image information may include an image of at least one scale of the target object after the robotic arm moves in the virtual environment.

[0069] Among them, the slave robot needs to complete a preset number of step-by-step actions to complete the target task, and each step-by-step action corresponds to a time step. For example, taking the target task of instant noodles as an example, when the slave robot is executing the target task, the steps that need to be executed in chronological order may include: unpacking the instant noodle package, taking out the seasoning packet, tearing open the seasoning packet and pouring in the seasoning, adding hot water, covering the instant noodle bowl, etc. The step-by-step action can be understood as the above-mentioned execution steps. The time step corresponding to the step-by-step action can be understood as the time required to execute the step-by-step action. For example, unpacking the instant noodle package corresponds to a time step, taking out the seasoning packet corresponds to a time step, and the rest of the execution steps are similar, and each execution step corresponds to a time step.

[0070] There are many ways to collect the posture data of the robot arm in the virtual environment and the image information of the target object until the robot arm completes the target task to obtain the training data set, which can be as follows:

[0071] For example, the pose data of the robot arm and the image information of the target object at each time step in the virtual environment can be collected to obtain an initial training data set, and the data labels corresponding to the initial training data set can be obtained, and the data labels can be added to the initial training data set to obtain the training data set, which can be specifically as follows:

[0072] S1. Collect the pose data of the robot arm and the image information of the target object at each time step in the virtual environment to obtain the initial training data set.

[0073] Among them, the initial training data set can be understood as the data obtained after data collection of the process of the slave robot arm performing the target task in the virtual environment. The initial training data set may include the posture data of the slave robot arm in the virtual environment at each time step, and the image information of the target object at each time step. For example, if the slave robot arm needs to complete 5 step-by-step actions (respectively, step-by-step action 1, step-by-step action 2, step-by-step action 3, step-by-step action 4 and step-by-step action 5) to perform the target task, the time to complete the target task can be divided into 5 time steps (respectively, time step 1, time step 2, time step 3, time step 4 and time step 5), wherein the slave robot arm completes step-by-step action 1 in time step 1, and completes step-by-step action 2 in time step 2. Similarly, the corresponding step-by-step actions are completed in other time steps, which are not described one by one here. For any of the time steps, for the sake of convenience, taking time step 1 as an example, the model training device can collect the posture data generated by the slave robot arm completing step-by-step action 1 at time step 1, and the image information of the target object when the slave robot arm completes step-by-step action 1 at time step 1. After collecting the pose data and image information corresponding to each time step, the pose data and image information of all time steps are integrated to obtain the initial training data set.

[0074] Among them, the data acquisition page may include a first camera and a second camera in the virtual environment. The robotic arm system is also configured with a camera, and the camera may include a global camera and a local camera. The global camera can capture images of the slave robotic arm operating on the target object, and the global camera can be installed independently of the slave robotic arm; the local camera can capture state images of the target object under the operation of the slave robotic arm, and the local camera can be installed on the end effector of the slave robotic arm. Therefore, the first camera can be understood as the mapping of the global camera in the virtual environment, and the second camera can be understood as the mapping of the local camera in the virtual environment. It can be understood that the number of the first camera and the second camera can be determined according to the actual configuration of the robotic arm system, and the installation positions of the first camera and the second camera can also be determined according to the actual configuration of the robotic arm system.

[0075] There are many ways to collect the pose data of the robot arm and the image information of the target object at each time step in the virtual environment, which can be as follows:

[0076] For example, the pose data of the slave robot arm in each time step in the virtual environment is collected to obtain a pose data set; the first image information of the target object in the virtual environment at each time step is collected by a first camera to obtain a first image information set, and the first image information includes the target object and the slave robot arm; the second image information of the target object in the virtual environment at each time step is collected by a second camera to obtain a second image information set, and the second image information includes the target object; the pose data set, the first image information set and the second image information set are used as the initial training data set.

[0077] Among them, for the collection of posture data, as mentioned above, the slave robot needs to complete a preset number of step-by-step actions to complete the target task, and each step-by-step action corresponds to a time step, that is, the process of the slave robot completing the target task can be divided into step-by-step actions corresponding to multiple time steps. Accordingly, in the process of collecting the initial training data set, the posture data corresponding to each time step of the slave robot in the virtual environment can be collected in chronological order, and the posture data corresponding to each time step can be combined to obtain a posture data set.

[0078] As for the acquisition of the image information of the target object, since two cameras (the first camera and the second camera) are configured in the virtual environment, the first camera can be used to acquire the global image of the slave manipulator in the virtual environment operating the target object, and the second camera can be used to acquire the state change image of the target object in the virtual environment under the operation of the slave manipulator. In the process of acquiring images, the first camera and the second camera can also acquire the corresponding image information in steps according to the time steps included in the target task. Specifically, the first camera can be used to acquire the first image information of the target object in each time step in the virtual environment, and the first image information corresponding to each time step can be integrated to obtain the first image information set. The first image information includes the target object and the slave manipulator, so the first image information can be understood as the global image of the slave manipulator in the virtual environment operating the target object. While acquiring the first image information, the second camera can also be used to acquire the second image information of the target object in each time step in the virtual environment, and the second image information corresponding to each time step can be integrated to obtain the second image information set. The second image information includes the target object, so the second image information can be understood as the state change image of the target object in the virtual environment under the operation of the slave manipulator.

[0079] It is understood that there are many ways to integrate the above, for example, the first image information can be arranged and combined in chronological order to obtain the first image information set. Similarly, the second image information can be arranged and combined in chronological order to obtain the second image information set.

[0080] After collecting the pose data set, the first image information set and the second image information set generated by the robotic arm completing the target task in the virtual environment, the pose data set, the first image information set and the second image information set can be packaged and the packaged data set can be used as the initial training data set.

[0081] S2. Obtain data labels corresponding to the initial training data set, and add the data labels to the initial training data set to obtain the training data set.

[0082] The data label can be understood as the target that the slave robot arm in the real environment needs to achieve in the process of completing the target task. As a target reference object, the data label can be used to judge the accuracy and completion of the action of the slave robot arm in the real environment in the process of performing the target task. Each target task has a corresponding data label. By adding the data label to the initial training data set corresponding to the target task, a training data set for the target task can be obtained.

[0083] Since the source of the control instruction can include generation based on the posture data of the main robot arm (angular position of N joint motors and status information of the handle), data generation based on historical posture data after data augmentation, generation based on control code or generation based on voice information, etc. Therefore, the method of obtaining the data label corresponding to the initial training data set can specifically include: when the control instruction is generated based on the posture data of the main robot arm, the main arm posture data corresponding to the slave arm posture data in the posture data set is screened out in the main arm posture data to obtain the data label; when the control instruction is generated based on the historical posture data after data augmentation, the data label corresponding to the initial training data set is determined based on the augmented angular position and augmented status information; when the control instruction is generated based on the control code, the data label corresponding to the initial training data set is generated based on the control code; when the control instruction is generated based on voice information, the data label corresponding to the initial training data set is generated based on the voice information. It can be specifically as follows:

[0084] (1) When the control instruction is generated based on the posture data of the main robot arm, the target posture data belonging to the same time step as the posture data in the posture data set can be screened out from the posture data of the main robot arm, and the target posture data can be used as the data label.

[0085] Among them, the posture data of the master robot can be understood as the posture data generated by the user operating the master robot in the real environment in the process of completing the corresponding task. As mentioned above, since the slave robot can be remotely operated by the master robot in the real environment, it means that the posture data of the master robot can be converted into control instructions for the slave robot in the virtual environment in real time, so that the slave robot in the virtual environment can follow the master robot and make the same action as the master robot. Therefore, the posture data of the master robot can be used as the goal that the slave robot in the real environment needs to achieve in the process of completing the corresponding task, that is, the posture data of the master robot can be used as the source of data labels.

[0086] (2) When the control instruction generates augmented data based on the historical posture data, the augmented angular position and augmented state information of each time step can be used as the augmented posture data of each time step. The augmented posture data of the corresponding time step in the posture data set can be filtered out from the augmented posture data to obtain the data label of each time step.

[0087] (3) When the control instruction is generated based on the control code, the target angular position corresponding to the joint motor can be extracted from the control code, and the target state information corresponding to the actuator can be extracted from the control code, and the target angular position and the target state information can be used as data labels corresponding to the initial training data set.

[0088] Among them, the control code can be the program code written by the developer according to the requirements of the target task, for the actions that the slave robot needs to perform. This means that the control code contains the various requirements for the slave robot to perform the target task, that is, the control code can contain the various action indicators that the slave robot needs to complete, such as the angular position that each joint motor needs to reach in the process of the slave robot performing the target task, and the status information of each actuator, so that this information is used as the data label corresponding to the initial training data set.

[0089] (4) When the control instructions are generated based on voice information, the angular positions that each joint motor needs to reach and the status information of each actuator can be pre-set for each target task, and a preset action indicator data set for the target task can be obtained. When voice information is detected, the voice information can be recognized, and a preset action indicator data set corresponding to the voice information can be determined based on the recognition result, and the preset action indicator data set can be used as the data label corresponding to the initial training data set.

[0090] Furthermore, there are many ways to add data labels to the initial training data set. For example, the training data (pose data and image information) of each time step in the initial training data set can be associated with the data label of the corresponding time step to obtain the training data set.

[0091] In some embodiments, after collecting the posture data of the slave robot in the virtual environment and the image information of the target object until the slave robot completes the target task to obtain a training data set, the neural network model corresponding to the target task can be trained based on the training data set to obtain a target neural network model corresponding to the target task. There are many ways to train the neural network model corresponding to the target task based on the training data set. For example, based on the training data set, a neural network model can be used to predict the posture data of the next time step of each time step to obtain a predicted posture data set, and the target data label corresponding to the predicted posture data in the predicted posture data set is screened out in the training data set, and the posture loss corresponding to the predicted posture data is determined based on the target data label, and the neural network model is converged based on the posture loss to obtain the target neural network model.

[0092] The predicted pose data set may include the predicted pose data of the slave robot at each time step. In this solution, the initial training data set for the target task in the training data set may be input into the neural network model, and the neural network model may output the predicted pose data set for the target task. Since the initial training data set includes pose data of multiple time steps and image information of the target object, the pose data of the slave robot at the next time step of each time step may be predicted in sequence according to the order of the time steps, thereby obtaining the predicted pose data set.

[0093] Among them, there can be many ways to filter out the target data labels corresponding to the predicted pose data in the predicted pose data set in the training data set. For example, the data label of the time step corresponding to each step-by-step action can be filtered out in the training data set as the target data label of the time step, thereby obtaining the target data label corresponding to the predicted pose data of each time step.

[0094] After filtering out the target data labels corresponding to the predicted pose data, the pose loss corresponding to the predicted pose data can be determined based on the target data labels. There are many ways to determine the pose loss corresponding to the predicted pose data. For example, the difference between the predicted pose data and the corresponding target loss can be calculated to obtain the pose loss corresponding to the predicted pose data.

[0095] After determining the pose loss corresponding to the predicted pose data, the neural network model can be converged based on the pose loss to obtain the target neural network model. There are many ways to converge the neural network model based on the pose loss. For example, the gradient descent algorithm can be used to update the network parameters of the neural network model based on the pose loss to obtain an updated neural network model, and the updated neural network model is used as the neural network model to return to the step of determining the pose loss corresponding to the predicted pose data based on the target data label until the neural network model converges to obtain the target neural network model. Alternatively, the momentum algorithm can be used to update the network parameters of the neural network model to obtain an updated neural network model, and the updated neural network model is used as the neural network model to return to the step of determining the pose loss corresponding to the predicted pose data based on the target data label until the neural network model converges to obtain the target neural network model, and so on.

[0096] In some embodiments, in order to improve the training accuracy of the neural network model, the neural network model can also be assisted in training using a training data set corresponding to the target task in a real environment. There can be many ways to assist in training. For example, the training data set after the target task is trained in a virtual environment can be used as the first training data set, and the neural network model is trained using the first training data set to obtain a candidate neural network model; the training data set after the target task is trained in a real environment is obtained as the second training data set; the candidate neural network model is trained based on the second training data set to obtain a target neural network model.

[0097] There are many ways to obtain the training data set after the target task is trained in the real environment. For example, the first posture data after the main robot arm moves can be obtained, and the second posture data after the slave robot arm follows the movement of the main robot arm in the real environment and the first image information of the target object can be obtained. Then, the step of obtaining the first posture data after the main robot arm moves is returned to execute until the target task is completed, and the second training data set is obtained.

[0098] Among them, there are many ways to obtain the first posture data of the main robot arm after the movement. For example, the first posture data of the main robot arm after the movement can be directly read, or the first posture data of the main robot arm after the movement can be received from the terminal or client corresponding to the robot arm system, or the first position of the handle of the main robot arm in the Cartesian coordinate system sent by a third-party device can be received, and the first position can be inversely kinematically solved to obtain the first angular position of the N joint motors of the main machine, and the first angular position can be used as the first posture data of the main robot arm after the movement, and so on.

[0099] After obtaining the first position data after the main robot moves, the second position data of the slave robot in the real environment after following the main robot and the first image information of the target object can be obtained. There are many ways to obtain the second position data and the first image information of the target object. For example, based on the first position data, a control instruction to control the slave robot in the real environment can be generated. Based on the control instruction, the slave robot in the real environment is controlled to follow the main robot to move, and the second angular position of the slave robot after following the main robot and the second state information of the actuator are obtained. Of course, only the second angular position can be obtained to obtain the second position data. Then, the image acquisition device can be used to acquire at least one scale of the target object after the slave robot follows the main robot to obtain the first image information.

[0100] After acquiring the second posture data of the slave manipulator in the real environment and the first image information of the target object, the step of acquiring the first posture data after the movement of the master manipulator can be returned to execute until the target task is completed, and the second training data set is obtained. For example, the step of acquiring the first posture data after the movement of the master manipulator can be returned to execute until the slave manipulator completes a target task. When the stop condition is to execute a target task once, at this time, the acquisition of the first posture data after the movement of the master manipulator can be stopped, and the acquisition of the second posture data and the first image information of the slave manipulator following the movement of the master manipulator in the real environment can be stopped, and the acquired at least one first posture data, at least one second posture data and at least one first image information can be used as the second training data set corresponding to the target task; when the stop condition is to execute the target task for a preset number of times, at this time, the step of acquiring the first posture data after the movement of the master manipulator can be returned to execute until the target task is executed for a preset number of times, and the acquired at least one first posture data, at least one second posture data and at least one first image information can be used as the second training data set corresponding to the target task.

[0101] In this solution, the neural network model can be preliminarily trained based on the first training data set to obtain a candidate neural network model. Compared with the initial neural network model, the model parameters of the candidate neural network model have been preliminarily optimized, and the prediction accuracy has been improved. After obtaining the candidate neural network model, the candidate neural network model can be further trained based on the second training data set until the candidate neural network model converges, and the target neural network model can be obtained. The process of training the candidate neural network model can be understood as the process of fine-tuning the model parameters of the candidate neural network model.

[0102] In the process of using the second training data set to assist model training, model training can be divided into two steps. The first step is to train the initial neural network model based on the first training data set to obtain a candidate neural network model; the second step is to train the candidate neural network model based on the second training data set to obtain a target neural network model. In this way, the training data set collected in the virtual environment (the first training data set) and the training data set collected in the real environment (the second training data set) can be fully utilized. In addition, by adopting a two-stage training method, a candidate neural network model with a certain prediction accuracy can be obtained after the first step of training, and the second training data set can be used in the second step to fine-tune the candidate neural network model to reduce the overfitting of the model and improve the prediction accuracy and robustness of the trained target neural network model.

[0103] In some embodiments, the method of assisting model training with a second training data set may include: using the training data set as the first training data set to obtain a second training data set after the target task is trained in a real environment; fusing the first training data set and the second training data set to obtain a target training data set; and using the target training data set to train the neural network model to obtain a target neural network model.

[0104] Furthermore, there are many ways to fuse the first training data set and the second training data set. For example, the first training data set and the second training data set can be directly put into the same data set, which is the target training data set. Alternatively, the data corresponding to each step-by-step action in the first training data set and the second training data set can be reorganized and exchanged in units of time steps. In this process, a new training data set can be generated to further increase the diversity of the training data set. For the newly generated training data set, it can be put into the same data set together with the first training data set and the second training data set, which is the target training data set. Furthermore, after the first training data set and the second training data set are fused, the neural network model can be trained based on the fused target training data set to obtain the target neural network model. The method of training the neural network model based on the fused target training data set can be similar to the method of training the neural network model based on the training data, as described above, and will not be repeated here.

[0105] In some embodiments, after the neural network model is trained, the trained target neural network model can also be corrected. There can be many ways to correct the target neural network model. For example, the third angular position of the M joint motors of the slave robotic arm and the third image information of the target object in the virtual environment can be obtained. Based on the third angular position and the third image information, the target neural network model is used to control the slave robotic arm to operate on the target object to perform the target task. When the correction signal for the slave robotic arm is obtained, the slave robotic arm is corrected to obtain a target training data set after correction. The target neural network model is updated based on the target training data set, and the updated neural network model is used as the target neural network model.

[0106] Among them, the method of obtaining the third angle position of the M joint motors of the slave robot arm and the third image information of the target object in the virtual environment can be similar to the method of obtaining the angular position of the M joint motors of the slave robot arm and the image information of the target object in the virtual environment. See the above for details and will not be repeated here.

[0107] Among them, when a correction signal for the slave robotic arm is obtained, there can be multiple ways to correct the slave robotic arm. For example, when voice control information is detected, a correction signal for the slave robotic arm is generated, and based on the voice control information, the angular position change of the M joint motors is determined, and the slave robotic arm is corrected according to the angular position change to obtain a target training data set after correction; or, when the movement of the main robotic arm is detected, a correction signal for the slave robotic arm is generated, and the fourth angular position of the N joint motors after the movement of the main robotic arm is obtained, and then, the fifth angular position of the N joint motors of the main robotic arm before movement is obtained, and the difference between the fourth angular position and the fifth angular position is calculated to obtain the angular position change of the M joint motors, and the slave robotic arm is corrected according to the angular position change to obtain the target training data set after correction.

[0108] Among them, there are many ways to determine the angular position change of M joint motors based on the voice control information. For example, the voice control information can be recognized based on the correction signal to obtain voice recognition content, and at least one keyword can be recognized in the voice recognition content. Based on the keyword, the displacement change of the end of the robotic arm is determined, and the current absolute coordinates of the end in Cartesian coordinates at the current time step are obtained. Based on the displacement change, the current absolute coordinates are adjusted to obtain the target absolute coordinates, and the angular position change of the M joint motors is calculated based on the target absolute coordinates and the current absolute coordinates.

[0109] Among them, based on keywords, there can be multiple ways to determine the displacement change of the end of the robotic arm. For example, a first keyword can be identified in the keywords, and a second keyword can be queried in the keywords. The first keyword indicates the movement direction of the end, and the second keyword indicates the movement distance of the end in the movement direction. When the keyword includes the second keyword, the displacement change of the end is determined based on the movement direction and the movement distance. When the keyword does not include the second keyword, the movement distance corresponding to the movement direction is screened out in the preset displacement distance set, and the displacement change of the end is determined based on the movement direction and the movement distance.

[0110] After determining the displacement change of the end, the current absolute coordinates of the end in Cartesian coordinates at the current time step can be obtained, and based on the displacement change, the current absolute coordinates are adjusted to obtain the target absolute coordinates. Based on the target absolute coordinates and the current absolute coordinates, the angular position changes of the M joint motors are calculated.

[0111] Furthermore, the slave robot is corrected according to the change in angular position to obtain a target training data set after correction. For example, the predicted angular position of the M joint motors predicted by the target neural network at the current time step can be obtained, the predicted angular position and the change in angular position can be added to obtain the target angular position of the M joint motors, and the slave robot is controlled to operate the target object based on the target angular position, and the step of obtaining the third angular position of the M joint motors and the third image information of the target object is returned to execute until the target task is completed to obtain the target training data set after correction.

[0112] In some embodiments, after the neural network model is trained, the trained target neural network model can also be used to control the slave robot in the real environment to operate on the target object. For example, at least one modal information of the operating area where the robot is located can be obtained, and the at least one modal information can be identified using a task recognition model to determine the task scenario. According to the task scenario, the target neural network model is selected from the preset neural network model set, and based on the target neural network model, the robot is controlled to operate on at least one target object.

[0113] Among them, the working area allows the placement of at least one target object. The target object can be understood as an object placed in the working area to be operated by the robotic arm. For example, if an apple is placed in the working area, and the robotic arm can cut or peel the apple, then the apple is the target object. Furthermore, the modal information may include various types of images, videos, audios or texts, etc. Accordingly, the image information of the area where the robotic arm is located can be collected by an image acquisition device to obtain modal information under at least one image modality, the audio information of the area where the robotic arm is located can be collected by an audio acquisition device to obtain modal information under an audio modality, and various types of sensors can be used to collect information of the area where the robotic arm is located to obtain at least one modal information, and so on.

[0114] There are many ways to identify at least one modal information using a task recognition model to determine the task scene. For example, the task recognition model can be used to identify the object information of the target object in at least one modal information. When there is only one target object, the object information can be used as the first object information, and the task scene can be screened out from a preset task scene set based on the first object information. When there are multiple target objects, the object information can be used as the second object information, and the target objects can be compared based on the second object information, and the task scene can be screened out from a preset task scene set based on the comparison result and the operation instruction.

[0115] After determining the task scenario, the target neural network model can be selected from the preset neural network model set. For example, the corresponding target task can be determined according to the task scenario, and the neural network model corresponding to the target task can be selected from the preset neural network model set to obtain the target neural network model.

[0116] After screening out the target neural network model, the posture data of the robotic arm and the image information of at least one target object can be obtained. Based on the posture data and the image information, the target neural network model is used to predict the posture data of the robotic arm at the next time step at the current moment to obtain the predicted posture data. Based on the predicted posture data, a control instruction for the robotic arm is generated. According to the control instruction, the robotic arm is controlled to perform operations on at least one target object, and the step of obtaining the posture data of the robotic arm and the image information of at least one target object is returned to execute until the preset stop condition is reached to execute the target task in the task scenario.

[0117] As can be seen from the above, the present application displays the slave robot and the target object by displaying a data collection page; then, in response to the control instructions for the slave robot, the slave robot is controlled to operate on the target object in a virtual environment to perform the target task; and then the posture data of the slave robot in the virtual environment and the image information of the target object are collected until the slave robot completes the target task to obtain a training data set, which is used to train the neural network model corresponding to the target task. Since the scheme can map the slave robot in the real environment to the virtual environment through 3D modeling and other methods, not only can the neural network model be trained, but also the slave robot and the target object in the virtual environment are conducive to reducing the training cost of the neural network model and improving the flexibility and versatility of the training.

[0118] An embodiment of the present application also provides a model training device, which can be integrated into an electronic device, such as a server or a terminal, and the terminal may include a tablet computer, a laptop computer and / or a personal computer, etc.

[0119] For example, Figure 4 As shown, the second model training device has a display interface, which can display the virtual slave robot arm and the virtual target object, and can display the control module 310 and the acquisition module 320, which can be specifically as follows:

[0120] (1) A control module 310 , having at least one line of code, and configured to respond to a control instruction for the slave robot arm and control the slave robot arm to operate on a target object in a virtual environment to perform a target task.

[0121] (2) An acquisition module 320 having at least one line of code and used to acquire posture data of the slave robot in the virtual environment and image information of the target object until the slave robot completes the target task, so as to obtain a training data set, which is used to train a neural network model corresponding to the target task.

[0122] In some embodiments, the second model training device may further include a training module 330, such as Figure 5 The training module 330 has at least one line of code and is used to train the neural network model corresponding to the target task through the training data to obtain the target neural network model.

[0123] For example, the training module 330 can be specifically used to predict the posture data of the next time step of each time step based on the training data set, using a neural network model to obtain a predicted posture data set, screen out target data labels corresponding to the predicted posture data in the predicted posture data set in the training data set, determine the posture loss corresponding to the predicted posture data based on the target data label, and converge the neural network model based on the posture loss to obtain a target neural network model.

[0124] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units can refer to the previous method embodiments, which will not be repeated here.

[0125] As can be seen from the above, the present application displays a virtual slave robot and a virtual target object through a display interface, and displays a control module 310 and an acquisition module 320, wherein the control module 310 controls the slave robot to operate on the target object in a virtual environment in response to a control instruction for the slave robot to perform a target task; the acquisition module 320 acquires the posture data of the slave robot in the virtual environment and the image information of the target object until the slave robot completes the target task, so as to obtain a training data set, which is used to train a neural network model corresponding to the target task. Since the scheme can map the slave robot in the real environment to the virtual environment through 3D modeling and other methods, not only can the neural network model be trained, but also the slave robot and the target object in the virtual environment are conducive to reducing the training cost of the neural network model and improving the flexibility and versatility of the training.

[0126] The present application also provides an electronic device, such as Figure 6 As shown, it shows a schematic diagram of the structure of the electronic device involved in the embodiment of the present application, specifically:

[0127] The electronic device may include components such as a processor 410 with one or more processing cores, a memory 420 with one or more computer-readable storage media, a power supply 430, and an input unit 440. Those skilled in the art will appreciate that Figure 6 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0128] The processor 410 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device, and performs various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 420, and calling data stored in the memory 420.

[0129] The memory 420 can be used to store software programs and modules, and the processor 410 executes various functional applications and data processing by running the software programs and modules stored in the memory 420. The memory 420 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the electronic device, etc.

[0130] The power supply 430 may be logically connected to the processor 410 through a power management system, so that functions such as charging, discharging, and power consumption management can be implemented through the power management system.

[0131] The input unit 440 may be used to receive input digital or character information, and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0132] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 410 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 420 according to the following instructions, and the processor 410 will run the application programs stored in the memory 420, thereby realizing various functions, as follows:

[0133] A data acquisition page is displayed, which includes a slave robot and a target object in a virtual environment. In response to a control instruction for the slave robot, the slave robot is controlled to operate on the target object in the virtual environment to perform a target task. The posture data of the slave robot in the virtual environment and the image information of the target object are collected until the slave robot completes the target task to obtain a training data set, which is used to train a neural network model corresponding to the target task.

[0134] The specific implementation of the above operations can be found in the previous embodiments and will not be described in detail here.

[0135] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0136] To this end, an embodiment of the present application provides a computer-readable storage medium, which stores multiple instructions, which can be loaded by a processor to execute any step in the model training provided in the embodiment of the present application. For example, the instruction can execute the following steps:

[0137] A data acquisition page is displayed, which includes a slave robot and a target object in a virtual environment. In response to a control instruction for the slave robot, the slave robot is controlled to operate on the target object in the virtual environment to perform a target task. The posture data of the slave robot in the virtual environment and the image information of the target object are collected until the slave robot completes the target task to obtain a training data set, which is used to train a neural network model corresponding to the target task.

[0138] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0139] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0140] Since the instructions stored in the computer-readable storage medium can execute the steps in any model training method provided in the embodiments of the present application, the beneficial effects that can be achieved by any model training method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0141] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the method provided in various optional implementations of the above-mentioned application and model training of a robotic arm system.

[0142] The above is a detailed introduction to a model training method, device and robotic arm system provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A model training method, characterized in that: include: Displaying a data acquisition page, the data acquisition page including a slave robot arm and a target object in a virtual environment, wherein a first camera and a second camera are configured in the virtual environment; In response to a control instruction for the slave robot arm, the slave robot arm is controlled to operate on the target object in the virtual environment to perform a target task, wherein the slave robot arm needs to complete a preset number of step-by-step actions to complete the target task, and each step-by-step action corresponds to a time step; Collecting the position and posture data of the slave robot arm at each time step in the virtual environment to obtain a position and posture data set; Collecting first image information of the target object in each time step in the virtual environment by the first camera to obtain a first image information set, wherein the first image information includes the target object and the slave robot arm; Collecting, by the second camera, second image information of the target object in each time step in the virtual environment to obtain a second image information set, wherein the second image information includes the target object; Using the pose data set, the first image information set and the second image information set as initial training data sets; A data label corresponding to the initial training data set is obtained, and the data label is added to the initial training data set to obtain the training data set, where the training data set is used to train a neural network model corresponding to the target task.

2. The model training method according to claim 1, characterized in that: An actuator is provided at the end of the slave robot arm, and the control instruction includes a motion instruction and an operation instruction. The motion instruction controls the slave robot arm to move in the virtual environment, and the operation instruction controls the actuator to operate the target object in the virtual environment.

3. The model training method according to claim 2, characterized in that: The slave robot arm is remotely operated by a master robot arm in a real environment, the master robot arm comprises N joint motors and a handle, the N joint motors make the degree of freedom of the handle the same as the degree of freedom of the actuator, and the handle is configured for a user to remotely operate the actuator; Before the step of controlling the slave robotic arm to operate the target object in the virtual environment in response to the control instruction for the slave robotic arm, the method further comprises: Obtaining the angular positions of the N joint motors and the status information of the handle; The motion command is generated based on the angular position, and the operation command is generated based on the state information.

4. The model training method according to claim 3, characterized in that: The handle includes a handle body, a servo connected to the handle body, and a wrench connected to the servo, the servo allows the wrench to rotate relative to the handle body, and the status information includes an angular position of the servo.

5. The model training method according to claim 3, characterized in that: Also includes: Obtaining current loop values ​​of the N joint motors; Based on the current loop value, the N joint motors are controlled to perform gravity compensation.

6. The model training method according to claim 3, characterized in that: Before the step of responding to the control instruction for the slave robot arm and controlling the slave robot arm to operate the target object in the virtual environment, the step further includes: Acquire historical posture data of the main robot arm, wherein the historical posture data includes historical angular positions of N joint motors of the main robot arm and historical state information of a handle; Performing data augmentation on the historical angular position to obtain augmented angular position, and performing data augmentation on the historical state information to obtain augmented state information; The motion instruction is generated based on the augmented angular position, and the operation instruction is generated based on the augmented state information.

7. The model training method according to claim 2, characterized in that: Before the step of responding to the control instruction for the slave robot arm and controlling the slave robot arm to operate the target object in the virtual environment, the step further includes: Acquire a control code of the slave robotic arm, wherein the control code is used to control the slave robotic arm to perform the target task, and the control code includes a motion control code and an operation control code; The motion instruction is generated based on the motion control code, and the operation instruction is generated based on the operation control code.

8. The model training method according to claim 2, characterized in that: Before the step of responding to the control instruction for the slave robot arm and controlling the slave robot arm to operate the target object in the virtual environment, the step further includes: Collect user's voice information; Performing voice recognition on the voice information to obtain at least one of movement information and operation information; The motion instruction is generated based on the motion information; and / or the operation instruction is generated based on the operation information.

9. The model training method according to claim 1, characterized in that: The obtaining of data labels corresponding to the initial training data set includes: When the control instruction is generated based on the posture data of the main robot arm, target posture data corresponding to the posture data in the posture data set are screened out from the posture data of the main robot arm to obtain the data label; When the control instruction generates data after data augmentation based on the historical posture data, determining a data label corresponding to the initial training data set based on the augmented angular position and the augmented state information; When the control instruction is generated based on the control code, generating a data label corresponding to the initial training data set based on the control code; When the control instruction is generated based on voice information, a data label corresponding to the initial training data set is generated based on the voice information.

10. The model training method according to claim 1, characterized in that: After acquiring the data labels corresponding to the initial training data set and adding the data labels to the initial training data set to obtain the training data set, the method further includes: Based on the training data set, a neural network model is used to predict the posture data of the next time step of each time step to obtain a predicted posture data set; Filtering out target data labels corresponding to the predicted pose data in the predicted pose data set from the training data set; Based on the target data label, the pose loss corresponding to the predicted pose data is determined, and the neural network model is converged based on the pose loss to obtain a target neural network model.

11. The model training method according to claim 1, characterized in that: After acquiring the data labels corresponding to the initial training data set and adding the data labels to the initial training data set to obtain the training data set, the method further includes: Using the training data set as a first training data set, and using the first training data set to train the neural network model to obtain a candidate neural network model; Acquire a second training data set after completing training of the target task in a real environment; The candidate neural network model is trained based on the second training data set to obtain a target neural network model.

12. The model training method according to claim 1, characterized in that: After acquiring the data labels corresponding to the initial training data set and adding the data labels to the initial training data set to obtain the training data set, the method further includes: Using the training data set as a first training data set, obtaining a second training data set after the target task is trained in a real environment; Merging the first training data set and the second training data set to obtain a target training data set; The target training data set is used to train the neural network model to obtain a target neural network model.

13. A robotic arm system, characterized in that: include: A control device, comprising a display interface, wherein the display interface can display a virtual slave robot arm and a virtual target object, wherein an actuator is disposed at the end of the slave robot arm, wherein the actuator has m degrees of freedom, and wherein the actuator is used to operate the target object; A main robot arm, comprising N joint motors and a handle, wherein the N joint motors enable the handle to have m degrees of freedom, and the handle is configured to allow a user to remotely operate the actuator; Wherein, when the handle remotely operates the actuator, the control device executes the model training method as described in any one of claims 1-12.

14. The robotic arm system according to claim 13, characterized in that: The handle is provided with a sensor, the sensor having a first level signal when the user holds the handle, and the sensor having a second level signal different from the first level signal when the user does not hold the handle; The control device is configured to obtain a level signal from the sensor, and lock the main robot arm when the level signal changes from the first level signal to the second level signal.

Images