Training method and device for robot arm control model and robot arm control system

Abstract

The present application provides a training method, device and robot control system for a robot control model, and relates to the field of robot control technology. The method adopts a preset initial robot control model to control the virtual model of the preset robot in the virtual working environment to perform multiple interactive tasks on the virtual model of the preset workpiece in the virtual working environment, and generate multiple interactive trajectory data; sample multiple transfer data from each interactive trajectory data; determine the candidate playback target set at multiple time points from the multiple transfer data; update the completed target position in the transfer data corresponding to each time point according to the candidate playback target set at each time point, and obtain an updated transfer data; generate the target robot control model according to the updated multiple transfer data and the initial robot control model. The target robot control model can be trained by this method, so that engineers can get rid of the complicated teaching work and improve the production efficiency of the robot.

Description

Technical Field

[0001] The present application relates to the field of robotic arm control technology, and in particular to a training method and device for a robotic arm control model and a robotic arm control system. Background Art

[0002] In traditional manufacturing, workpieces are usually sorted manually, which has problems such as low efficiency and low sorting accuracy. With the in-depth implementation of the "Made in China 2025" strategy, China's modern manufacturing industry has developed rapidly, and robotic arms have shown high stability and high efficiency in workpiece sorting. The workpiece sorting function realized by robotic arms has become the mainstream solution to replace manual sorting, and the demand for the use of robotic arms by manufacturing companies has continued to increase.

[0003] However, the robotic arms with workpiece sorting functions currently used in production all achieve their functions through manual programming and teaching training. Engineers need to spend a lot of energy to program and teach the robotic arms, which takes up a lot of engineers' time. Summary of the invention

[0004] The purpose of this application is to provide a training method and device for a robot control model and a robot control system to address the deficiencies in the prior art, so as to solve the problems existing in the prior art.

[0005] To achieve the above purpose, the technical solution adopted in the embodiment of the present application is as follows:

[0006] In a first aspect, an embodiment of the present application provides a training method for a robotic arm control model, comprising:

[0007] Using a preset initial robot arm control model, controlling a virtual model of a preset robot arm in a virtual working environment to perform multiple interactive tasks on a virtual model of a preset workpiece in the virtual working environment, and generating multiple interactive trajectory data; wherein each interactive trajectory data is trajectory data of the virtual model of the preset robot arm performing one interactive task;

[0008] Sampling a plurality of transfer data from each of the interaction trajectory data, the plurality of transfer data comprising: process data of a plurality of time points at which the virtual model of the preset robot arm executes the same interaction task;

[0009] Determine, from the plurality of transfer data, candidate playback target sets for the plurality of time points, wherein the candidate playback target set for each time point includes: a completed target position of the virtual model of the preset workpiece at at least one time point after each time point;

[0010] According to the candidate playback target set at each time point, updating the completed target position in the transfer data corresponding to each time point to obtain an updated transfer data;

[0011] The initial robot arm control model is trained according to the updated multiple transfer data to generate a target robot arm control model.

[0012] In one embodiment, sampling a plurality of transfer data from each interaction trajectory data comprises:

[0013] Sampling the interaction observation data of the virtual model of the preset robot arm at the multiple time points from each interaction trajectory data;

[0014] Sampling the motion data of the virtual model of the preset robot arm at the multiple time points from each of the interaction trajectory data; wherein the motion data includes: the position and posture of the virtual model of the preset workpiece and the position and posture of the end of the virtual model of the preset robot arm;

[0015] Sampling reward data for completing the same interactive task at the multiple time points from each of the interactive trajectory data;

[0016] Sampling the target position of the same interactive task at the multiple time points from each of the interactive trajectory data;

[0017] Sampling completed target positions at the plurality of time points from each interaction trajectory data;

[0018] The multiple transfer data are generated respectively according to the interaction observation data, action data, reward data, target position and completed target position at the multiple time points.

[0019] In one embodiment, updating the completed target position in the transfer data corresponding to each time point according to the candidate playback target set at each time point to obtain an updated transfer data includes:

[0020] Determine the playback target position at each time point from each candidate position in the candidate playback target set at each time point;

[0021] According to the playback target position at each time point, the completed target position in the transfer data corresponding to each time point is updated to obtain the updated transfer data.

[0022] In one embodiment, determining the playback target position at each time point from each candidate position in the candidate playback target set at each time point includes:

[0023] According to each candidate position at each time point and the target position of the same interactive task, calculating the distance parameter corresponding to each candidate position;

[0024] Calculate, according to each candidate position at each time point and the completed target position in the transfer data at each time point, a similarity parameter corresponding to each candidate position;

[0025] Performing weighted processing according to the distance parameters corresponding to the candidate positions and the similarity parameters corresponding to the candidate positions to obtain the distance evaluation parameters of the candidate positions;

[0026] According to the distance evaluation parameters of the candidate positions, the candidate position with the highest distance evaluation parameter is selected as the playback target position at each time point.

[0027] In one embodiment, the training of the initial robot arm control model according to the updated plurality of transfer data to generate the target robot arm control model includes:

[0028] Calculating the target reward data at the multiple time points respectively according to the updated multiple transfer data;

[0029] Updating the reward data at each time point in the updated multiple transfer data to the target reward data at each time point to obtain multiple target transfer data;

[0030] The initial robot arm control model is trained according to the multiple target transfer data to generate the target robot arm control model.

[0031] In one embodiment, respectively calculating the target reward data at the multiple time points according to the updated multiple transfer data includes:

[0032] Calculate a first distance parameter at the first time point according to the completed target position at a second time point before the first time point in the updated plurality of transfer data and the target position of the same interactive task;

[0033] Calculating a second distance parameter at the first time point according to the completed target positions at the first time point in the updated plurality of transfer data and the target position of the same interactive task;

[0034] Calculate target reward data at the first time point according to the first distance parameter and the second distance parameter.

[0035] In one embodiment, calculating the target reward data at the first time point according to the first distance parameter and the second distance parameter includes:

[0036] Calculating the progress reward data at the first time point according to the difference between the first distance parameter and the second distance parameter;

[0037] The target reward data at the first time point is calculated based on the progress reward data at the first time point and the reward data at the first time point after the updated plurality of transfer data.

[0038] In one embodiment, the training of the initial robot arm control model according to the plurality of target transfer data to generate the target robot arm control model includes:

[0039] updating the parameters of the initial robot arm control model according to the plurality of target transfer data;

[0040] Using the initial robot arm control model after parameter update, controlling the virtual model of the preset robot arm to perform an interactive task on the virtual model of the preset workpiece, and obtaining a control success rate of the initial robot arm control model;

[0041] If the success rate does not meet the preset threshold, the virtual model of the initial robot arm is re-controlled to perform multiple interactive tasks on the virtual model of the preset workpiece until the control success rate of the target robot arm control model meets the preset threshold.

[0042] In a second aspect, an embodiment of the present application provides a training device for a robotic arm control model, comprising:

[0043] A control module, used to use a preset initial robot arm control model to control a virtual model of a preset robot arm in a virtual working environment to perform multiple interactive tasks on a virtual model of a preset workpiece in the virtual working environment, and generate multiple interactive trajectory data; wherein each interactive trajectory data is trajectory data of the virtual model of the preset robot arm performing one interactive task;

[0044] A collection module, used for sampling a plurality of transfer data from each interaction trajectory data, wherein the plurality of transfer data include: process data of a plurality of time points at which the virtual model of the preset robot arm executes the same interaction task;

[0045] A determination module, configured to determine, from the plurality of transfer data, candidate playback target sets at the plurality of time points, wherein the candidate playback target set at each time point comprises: a completed target position of the virtual model of the preset workpiece at at least one time point after each time point;

[0046] An updating module, configured to update the completed target position in the transfer data corresponding to each time point according to the candidate playback target set at each time point, to obtain an updated transfer data;

[0047] The training module is used to train the initial robot arm control model according to the updated multiple transfer data to generate a target robot arm control model.

[0048] In a third aspect, an embodiment of the present application provides a robot arm control system, the robot arm control system comprising: a computer device, a preset robot arm, a controller, and an environment acquisition module; the computer device is pre-integrated with the target robot arm control model and a virtual working environment, wherein the virtual working environment has a virtual model of the preset robot arm and a virtual model of the preset workpiece; the target robot arm control model is a model trained by the training method of the robot arm control model described in the above embodiment;

[0049] The computer device is in communication connection with the environment acquisition module to obtain environmental parameters in the actual working environment where the preset robot arm is located, and based on the environmental parameters and the preset target position, the target robot arm control model is used to generate robot arm control parameters;

[0050] The computer device is communicatively connected to the controller, and the controller is connected to the robotic arm, so that when the computer device controls the virtual model of the preset robotic arm in the virtual working environment to perform interactive operations on the virtual model of the preset workpiece based on the robotic arm control parameters, the preset robotic arm in the actual working environment is controlled by the controller to perform interactive operations on the preset workpiece.

[0051] In a fourth aspect, an embodiment of the present application provides a computer device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores program instructions executable by the processor, and when the computer device is running, the processor communicates with the storage medium through the bus, and the processor executes the program instructions to perform the steps of the above-mentioned training method of the robotic arm control model.

[0052] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the training method of the above-mentioned robotic arm control model are executed.

[0053] The beneficial effects of the present application are as follows: the present application provides a training method, device and a robotic arm control system for a robot arm control model, the training method for the robot arm control model includes: using a preset initial robot arm control model, controlling the virtual model of the preset robot arm in the virtual working environment to perform multiple interactive tasks on the virtual model of the preset workpiece in the virtual working environment, and generating multiple interactive trajectory data; sampling multiple transfer data from each interactive trajectory data; determining a candidate playback target set at multiple time points from the multiple transfer data; updating the completed target position in the transfer data corresponding to each time point according to the candidate playback target set at each time point, and obtaining an updated transfer data; training the initial robot arm control model according to the updated multiple transfer data, and generating a target robot arm control model. Using this method, the initial robot arm control model can be trained to obtain a trained target robot arm control model, which can be used to control the virtual model of the preset robot arm to place the virtual model of the preset workpiece at a preset position. The present application realizes the automatic training of the initial robot arm control model, freeing engineers from the complicated teaching work and improving the production efficiency of the robot arm. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0055] Figure 1 A flowchart of a method for training a robotic arm control model provided in one embodiment of the present application;

[0056] Figure 2 A schematic diagram of a method flow for obtaining transfer data provided in an embodiment of the present application;

[0057] Figure 3 A schematic diagram of a method flow for updating transfer data provided in an embodiment of the present application;

[0058] Figure 4 A schematic diagram of a method flow for determining a playback target position provided in an embodiment of the present application;

[0059] Figure 5 A flowchart of another method for training a robot arm control model provided in one embodiment of the present application;

[0060] Figure 6 A flowchart of a method for calculating target reward data provided in one embodiment of the present application;

[0061] Figure 7 A flowchart of a specific method for calculating target reward data provided in an embodiment of the present application;

[0062] Figure 8 A schematic flow chart of a method for testing a target robotic arm control model provided in one embodiment of the present application;

[0063] Fig. 9 A schematic diagram of the structure of a training device for a robotic arm control model provided in one embodiment of the present application;

[0064] FIG. 10( a ) and FIG. 10( b ) are schematic diagrams of the structure of a robotic arm control system provided in one embodiment of the present application;

[0065] Fig.11 A schematic diagram of the structure of a computer device provided in one embodiment of the present application.

[0066] Explanation of the accompanying drawings: 1. Computer equipment; 2. Preset robotic arm; 3. Controller; 4. Environment acquisition module; 5. Preset workpiece; 6. Storage box; 7. Power supply; 8. Transformer. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0068] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0069] In the description of the present application, it should be noted that if the terms "upper", "lower", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the drawings, or is the orientation or position relationship in which the product of the application is usually placed when used. It is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application.

[0070] In addition, the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0071] It should be noted that, in the absence of conflict, the features in the embodiments of the present application may be combined with each other.

[0072] An embodiment of the present application provides a training method, device and robotic arm control system for a robotic arm control model. The training method for the robotic arm control model can be generated by any computer device that integrates an initial robotic arm control model generation algorithm. The computer device can be, for example, a terminal-oriented computer device or a back-end server.

[0073] The following is a specific example description of the training method, device and robot control system of the robot control model provided in the present application through multiple examples in combination with multiple drawings.

[0074] Figure 1 A flowchart of a method for training a robot arm control model provided in one embodiment of the present application is shown below. Figure 1 As shown, the method includes:

[0075] S101, using a preset initial robot arm control model to control a virtual model of a preset robot arm in a virtual working environment to perform multiple interactive tasks on a virtual model of a preset workpiece in the virtual working environment, and generate multiple interactive trajectory data.

[0076] First of all, it should be noted that the initial robot arm control model is a deep neural network with parameters initialized, which can be used to enable the preset robot arm in the virtual working environment to perform interactive tasks on the virtual model of the preset workpiece in the virtual working environment, but the probability of the virtual model controlling the preset robot arm to successfully place the virtual model of the preset workpiece at the preset position is random. The preset position refers to the pre-specified position where the virtual model of the preset workpiece needs to be placed.

[0077] The virtual working environment, the virtual model of the preset robotic arm, the virtual model of the preset workpiece, and the preset position of this embodiment are respectively generated by a computer device based on the actual working environment, the preset robotic arm in the actual working environment, the preset workpiece in the actual working environment, and the preset position in the actual working environment. The virtual working environment is consistent with the actual working environment, the virtual model of the preset robotic arm is consistent with the robotic arm of the preset model in the actual working environment, the virtual model of the preset workpiece is consistent with the workpiece of the preset shape and size in the actual working environment, and the preset position is consistent with the preset position in the actual working environment. It can be understood that the virtual working environment of this embodiment is exactly the same as the actual working environment. This embodiment trains the preset initial robotic arm control model in the virtual working environment to obtain a target robotic arm control model. The trained target robotic arm control model can be used to control the virtual model of the preset robotic arm in the virtual working environment to place the virtual model of the preset workpiece at the preset position, and can also be used to control the preset robotic arm in the actual working environment through a controller to place the preset workpiece at the preset position.

[0078] It can be seen that this embodiment trains the preset initial robot arm control model. Therefore, before training the initial robot arm control model, it is necessary to use the preset initial robot arm control model to control the virtual model of the preset robot arm in the virtual working environment to perform multiple interactive tasks on the virtual model of the preset workpiece in the virtual working environment, generate multiple interactive trajectory data, and then train the initial robot arm control model according to the multiple interactive trajectory data.

[0079] Among them, an interactive task refers to the action of the preset robotic arm virtual model in the virtual working environment grabbing the virtual model of the preset workpiece and placing the virtual model of the preset workpiece, and each interactive trajectory data is the trajectory data of the virtual model of the preset robotic arm executing an interactive task, that is, in this embodiment, the initial robotic arm control model controls the virtual model of the preset robotic arm to perform multiple interactive tasks on the virtual model of the preset workpiece, and generates multiple interactive trajectory data, each interactive trajectory data corresponds to an interactive task, but each interactive trajectory data is not necessarily the data of successfully executing the interactive task.

[0080] In summary, it can be understood that using an untrained initial robot arm control model to control the virtual model of the preset robot arm cannot guarantee the success rate of the virtual model of the preset robot arm placing the virtual model of the preset workpiece to the preset position. According to steps S101-S105 of this embodiment, the target robot arm control model is trained, and the target robot arm control model can control the virtual model of the preset robot arm to successfully place the virtual model of the preset workpiece to the preset position.

[0081] S102: Sample multiple transfer data from each interaction trajectory data.

[0082] After generating multiple interaction trajectory data according to step S101, multiple transfer data can be sampled from each interaction trajectory data. It can be understood that, among the multiple interaction trajectory data, each interaction trajectory data contains multiple transfer data, and the multiple transfer data corresponding to the same interaction trajectory data correspond to the same interaction task, that is, the multiple transfer data corresponding to the same interaction trajectory data include process data at multiple time points when the virtual model of the preset robotic arm performs the same interaction task.

[0083] Figure 2 A flow chart of a method for obtaining transfer data provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, multiple transfer data are sampled from each interaction trajectory data, which may include the following examples:

[0084] S201, sampling interaction observation data of a virtual model of a preset robotic arm at multiple time points from each interaction trajectory data.

[0085] Specifically, the interactive observation data is: observation data of a virtual model of a preset robot arm at multiple time points sampled from an interactive trajectory data. In this embodiment, the observation data presets multiple postures of the virtual model of the workpiece, and multiple postures of the end of the virtual model of the preset robot arm corresponding to the multiple postures of the workpiece.

[0086] S202: sampling the motion data of the virtual model of the preset robot arm at multiple time points from each interaction trajectory data.

[0087] Specifically, the motion data includes: robot arm motion data output by a robot arm control model sampled from an interactive trajectory data. In this embodiment, the robot arm motion data includes the rotation angles of each joint of a virtual model of a preset robot arm.

[0088] S203. Sample reward data for completing the same interactive task at multiple time points from each interactive trajectory data.

[0089] Specifically, the reward data includes: reward data for completing the same interactive task at multiple time points sampled from an interactive trajectory data, and the reward data is used to evaluate the interactive task. In this embodiment, when the virtual model of the preset robotic arm completes the task of grasping and placing the virtual model of the preset workpiece, the reward data is 0, and in other cases the reward data is -1.

[0090] S204: Sample target positions of the same interaction task at multiple time points from each interaction trajectory data.

[0091] Specifically, the target position includes: coordinates of a target position for placing a virtual model of a preset workpiece in three-dimensional coordinates in the virtual working environment.

[0092] S205: Sample completed target positions at multiple time points from each interaction trajectory data.

[0093] Specifically, the completed target position includes: the current coordinates of the virtual model of the preset workpiece in the three-dimensional coordinates in the virtual working environment.

[0094] S206. Generate a plurality of transfer data respectively according to the interaction observation data, action data, reward data, target position and completed target position at a plurality of time points.

[0095] Based on S201-S205, the i-th interactive trajectory data T i Take the jth time point in as an example, T i The transfer data at the jth time point can be defined as in, is the interaction trajectory data T i The interactive observation data at the jth time point in , is the interaction trajectory data T i The action data at the jth time point in , is the interaction trajectory data T i The reward data at the jth time step in g i is the interaction trajectory data T i The target location, is the interaction trajectory data T i The completed target position at the jth time step in . The transfer data at other time points of the interaction trajectory data, and the multiple transfer data at multiple time points of other interaction trajectory data, and so on, are not repeated here.

[0096] It should be noted that in terms of time series, S201-S205 can occur sequentially or simultaneously. Here, only S201-S205 is taken as an example for specific description. It cannot be assumed that the method described in S201-S205 must be performed in the above order. S201-S205 can occur simultaneously or in any order.

[0097] S103: Determine candidate playback target sets at multiple time points from the multiple transfer data.

[0098] After obtaining multiple transfer data, since the interaction trajectory data corresponding to each transfer data is not necessarily the data when the interaction task is successfully executed, the transfer data needs to be updated and corrected. Specifically, the candidate playback target sets G at multiple time points are determined from the multiple transfer data. ji, wherein the candidate playback target set at each time point includes: the completed target position of the virtual model of the preset workpiece at all time points after each time point (at least one of all time points after each time point), for example, the i-th interaction trajectory data T i The candidate playback target set at time point j can be Where L is the interactive trajectory data T i Length.

[0099] S104: Update the completed target position in the transfer data corresponding to each time point according to the candidate playback target set at each time point to obtain an updated transfer data.

[0100] After obtaining the candidate playback target set at each time point, the completed target position in the transfer data corresponding to each time point can be updated according to the candidate playback target set at each time point to obtain an updated transfer data at each time point.

[0101] S105. Train the initial robot arm control model according to the updated multiple transfer data to generate a target robot arm control model.

[0102] After obtaining the updated transfer data at each time point, the initial robot arm control model can be trained according to the updated multiple transfer data to generate a target robot arm control model, which can control the interactive tasks between the virtual model of the preset robot arm and the virtual model of the preset workpiece in the virtual working environment, and can also control the interactive tasks between the preset robot arm and the preset workpiece in the actual working environment. The initial robot arm control model is a neural network model of a preset type, for example, a multi-layer perceptron model including three hidden layers.

[0103] In summary, this embodiment provides a training method for a robot control model, including using a preset initial robot control model to control the virtual model of the preset robot in the virtual working environment to perform multiple interactive tasks on the virtual model of the preset workpiece in the virtual working environment, and generate multiple interactive trajectory data; sampling multiple transfer data from each interactive trajectory data; determining a candidate playback target set at multiple time points from the multiple transfer data; updating the completed target position in the transfer data corresponding to each time point according to the candidate playback target set at each time point, and obtaining an updated transfer data; training the initial robot control model according to the updated multiple transfer data to generate a target robot control model. Using the method of this embodiment, the initial robot control model can be trained to obtain a trained target robot control model, and the target robot control model can be used to control the virtual model of the preset robot to place the virtual model of the preset workpiece at a preset position. This embodiment realizes the automatic training of the initial robot control model, freeing engineers from the complicated teaching work and improving the production efficiency of the robot.

[0104] Figure 3 A flow chart of a method for updating transfer data provided in an embodiment of the present application is as follows: Figure 3 As shown, in step S104, according to the candidate playback target set at each time point, the completed target position in the transfer data corresponding to each time point is updated to obtain an updated transfer data, including:

[0105] S301 : Determine a playback target position at each time point from candidate positions in a candidate playback target set at each time point.

[0106] As described in the above embodiment, the candidate playback target set for each time point includes multiple candidate positions. Therefore, before updating the completed target position in the transfer data corresponding to each time point according to the candidate playback target set for each time point, it is necessary to first determine the playback target position of each time point.

[0107] Take the i-th interaction trajectory data T i The set of candidate playback targets at time point j For example, it is necessary to determine the playback target position, that is, the candidate position of the optimal candidate playback target, among multiple candidate playback targets in the set.

[0108] S302: According to the playback target position at each time point, the completed target position in the transfer data corresponding to each time point is updated to obtain updated transfer data.

[0109] After obtaining the candidate position of the optimal candidate replay target at a time point, the completed target position in the transfer data corresponding to the time point can be updated to the candidate replay target of the candidate position. The above operation can be repeated for each time point, and the completed target position in the transfer data corresponding to each time point can be updated to the candidate replay target of the candidate position corresponding to the time point.

[0110] Through the method of this embodiment, the transfer data can be updated and corrected to avoid the inconsistency between the generated target robotic arm control model and the initial robotic arm control model due to the deviation of the transfer data during the training of the initial robotic arm control model using the initial robotic arm control model, thereby ensuring the reliability of the target robotic arm control model.

[0111] On this basis, Figure 4 A flowchart of a method for determining a playback target position provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, in step S301, determining the playback target position at each time point from each candidate position in the candidate playback target set at each time point includes:

[0112] S401. Calculate the distance parameter corresponding to each candidate position according to each candidate position at each time point and the target position of the same interactive task.

[0113] In this embodiment, the transfer data at each candidate position is represented by Indicates that the target position of the same interactive task is represented by g i Indicates that the distance parameter can be, for example,

[0114] S402: Calculate a similarity parameter corresponding to each candidate position according to each candidate position at each time point and a completed target position in the transfer data at each time point.

[0115] In this embodiment, the transfer data at each candidate position is represented by Indicates that the completed target position in the transfer data at each time point is represented by Indicates that the distance parameter can be, for example,

[0116] S403 , performing weighted processing according to the distance parameters corresponding to each candidate position and the similarity parameters corresponding to each candidate position to obtain the distance evaluation parameters of each candidate position.

[0117] After obtaining the distance parameter corresponding to each candidate position in step S401 and the similarity parameter corresponding to each candidate position in step S402, weighted processing can be performed according to the distance parameter corresponding to each candidate position and the similarity parameter corresponding to each candidate position to calculate the distance evaluation parameter of each candidate position. And j′∈[j+1,L], the distance evaluation parameter of each candidate position can be calculated by the following formula:

[0118]

[0119] Among them, ω1 is the task target distance weight, and ω2 is the similarity weight between the playback target and the completed target.

[0120] S404 . According to the distance evaluation parameters of the candidate positions, select the candidate position with the highest distance evaluation parameter as the playback target position at each time point.

[0121] After obtaining the distance evaluation parameters of each candidate position, the candidate position with the highest distance evaluation parameter is calculated as the playback target position at each time point according to the following formula:

[0122]

[0123] Finally, the data will be transferred Achieved target data in Replace with the transfer data at the candidate location Right now Get the updated transfer data.

[0124] An embodiment of the present application further provides a method for updating the reward data in the updated transfer data. Based on the above embodiment, Figure 5 A flowchart of another method for training a robot arm control model provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, according to the updated multiple transfer data, the initial robot arm control model is trained to generate a target robot arm control model, including:

[0125] S501. Calculate target reward data at multiple time points respectively according to the updated multiple transfer data.

[0126] When multiple transfer data are updated, target reward data at multiple time points can also be calculated.

[0127] S502. Update the reward data at each time point in the updated multiple transfer data to the target reward data at each time point to obtain multiple target transfer data.

[0128] After the target reward data at multiple time points are calculated, the reward data at each time point in the updated multiple transfer data can be updated to the target reward data at each time point to obtain multiple target transfer data.

[0129] S503: Train the initial robot arm control model according to the multiple target transfer data to generate a target robot arm control model.

[0130] The initial robot arm control model is trained according to a plurality of target transfer data to generate a target robot arm control model.

[0131] In this embodiment, the reward data of the transfer data is updated and corrected to avoid the inconsistency between the generated target robotic arm control model and the initial robotic arm control model due to the deviation of the reward data during the training of the initial robotic arm control model using the initial robotic arm control model. On the basis of the above embodiment, the reliability of the target robotic arm control model is further ensured.

[0132] Figure 6 A flow chart of a method for calculating target reward data provided in an embodiment of the present application is as follows: Figure 6 As shown, in step S501, target reward data at multiple time points are calculated respectively according to the updated multiple transfer data, including:

[0133] S601: Calculate a first distance parameter at a first time point according to a completed target position at a second time point before the first time point in a plurality of updated transfer data and a target position of the same interactive task.

[0134] In this embodiment, the first time point is any time point in the transfer data, and the second time point is a time point before each first time point.

[0135] Take the i-th interaction trajectory data T i At time point j as the first time point, the completed target position at the second time point is given by Indicates that the target position of the same interactive task is represented by g i Indicates that the first distance parameter at the first time point can be, for example, in, is the trajectory data T i The achieved target data at time point j-1.

[0136] S602: Calculate a second distance parameter at the first time point according to the completed target positions at the first time point and the target position of the same interactive task in the updated plurality of transfer data.

[0137] The completed target position at the first time point can be represented by Indicates that the second distance parameter at the first time point can be, for example,

[0138] S603: Calculate target reward data at a first time point according to the first distance parameter and the second distance parameter.

[0139] After obtaining the first distance parameter at the first time point in step S601 and obtaining the second distance parameter at the first time point in step S602, the target reward data at the first time point can be calculated according to the first distance parameter and the second distance parameter.

[0140] On this basis, Figure 7 A flowchart of a specific method for calculating target reward data provided in an embodiment of the present application is shown as follows: Figure 7 As shown, in step S603, the target reward data at the first time point is calculated according to the first distance parameter and the second distance parameter, including:

[0141] S701. Calculate progress reward data at a first time point according to a difference between a first distance parameter and a second distance parameter.

[0142] After obtaining the first distance parameter at the first time point according to step S601 and obtaining the second distance parameter at the first time point according to step S602, the progress reward data at the first time point can be calculated according to the following formula:

[0143]

[0144] S702. Calculate the target reward data at the first time point based on the progress reward data at the first time point and the reward data at the first time point after the updated plurality of transfer data.

[0145] When the progress reward data at each first time point is obtained, that is, after the progress reward data at each time point is obtained, the target reward data at the first time point can be calculated according to the following formula based on the progress reward data at the first time point and the reward data at the first time point after the updated multiple transfer data:

[0146]

[0147] Finally, for the updated multiple transfer data Data will be transferred Reward data in Replace with Total Rewards Right now The transfer data after the reward data is updated is obtained.

[0148] An embodiment of the present application also provides a method for testing a target robotic arm control model generated by training, Figure 8 A flow chart of a method for testing a target robotic arm control model provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, in step S503, training the initial manipulator control model according to the multiple target transfer data to generate the target manipulator control model may include:

[0149] S801. Update the parameters of the initial robot arm control model according to multiple target transfer data.

[0150] After acquiring multiple target transfer data, the parameters of the initial robotic arm control model can be updated according to the multiple target transfer data. The updated initial robotic arm control model has a higher success rate in controlling the virtual model of the preset robotic arm to perform interactive tasks.

[0151] S802: using the initial robot arm control model after parameter update, controlling the virtual model of the preset robot arm to perform an interactive task on the virtual model of the preset workpiece, and obtaining the control success rate of the initial robot arm control model.

[0152] After the parameters of the initial robot arm control model are updated, the initial robot arm control model with updated parameters is used to control the virtual model of the preset robot arm to perform multiple interactive tasks on the virtual model of the preset workpiece, and the control success rate of the updated initial robot arm control model is obtained, and the control success rate is used to determine whether the initial robot arm control model is successfully trained. The success rate can be: success rate = (number of successful controls / total number of controls) × 100%.

[0153] S803: If the success rate does not meet the preset threshold, the virtual model of the initial robot arm is re-controlled to perform multiple interactive tasks on the virtual model of the preset workpiece until the control success rate of the obtained target robot arm control model meets the preset threshold.

[0154] If the success rate does not meet the preset threshold, the virtual model of the initial robotic arm is re-controlled to perform multiple interactive tasks on the virtual model of the preset workpiece, that is, the training process of the above embodiments S101-S105 is repeated until the control success rate of the initial robotic arm control model obtained meets the preset threshold. The initial robotic arm control model whose control success rate meets the preset threshold is regarded as the target robotic arm control model, and the training of the initial robotic arm control model is completed.

[0155] It should be noted that the preset threshold can be set according to actual needs. In the present embodiment, the preset threshold can be, for example, 95%. That is, the success rate of the initial robotic arm control model controlling the virtual model of the preset robotic arm to perform interactive tasks is greater than 95%. Only then can the initial robotic arm control model be considered to be successfully trained.

[0156] In this embodiment, by verifying the control success rate of the initial robotic arm control model and retraining the initial robotic arm control model whose success rate does not meet the preset threshold, the control precision and accuracy of the trained target robotic arm control model for the preset robotic arm are guaranteed.

[0157] The following is a corresponding explanation of the training device, robotic arm control system, computer equipment and storage medium provided for executing any of the above embodiments of the present application. The specific implementation process and the technical effects produced are the same as those of the corresponding method embodiments mentioned above. For the sake of brief description, for the parts not mentioned in this embodiment, please refer to the corresponding content in the method embodiment.

[0158] Fig. 9 A schematic diagram of the structure of a training device for a robotic arm control model provided in an embodiment of the present application is shown in FIG. Fig. 9 As shown, the training device of the robot arm control model includes:

[0159] The control module 901 is used to adopt a preset initial robot arm control model to control the virtual model of the preset robot arm in the virtual working environment to perform multiple interactive tasks on the virtual model of the preset workpiece in the virtual working environment, and generate multiple interactive trajectory data; wherein each interactive trajectory data is the trajectory data of the virtual model of the preset robot arm in executing an interactive task.

[0160] The acquisition module 902 is used to sample a plurality of transfer data from each interactive trajectory data, wherein the plurality of transfer data includes: process data of a plurality of time points at which a virtual model of a preset robot arm executes a corresponding same interactive task.

[0161] The determination module 903 is used to determine candidate playback target sets at multiple time points from multiple transfer data, wherein the candidate playback target set at each time point includes: a completed target position of the virtual model of the preset workpiece at at least one time point after each time point.

[0162] The updating module 904 is used to update the completed target position in the transfer data corresponding to each time point according to the candidate playback target set at each time point, and obtain an updated transfer data.

[0163] The training module 905 is used to train the initial robot arm control model according to the updated multiple transfer data to generate a target robot arm control model.

[0164] In a possible implementation example, the acquisition module 902 is also used to sample the interaction observation data of the virtual model of the preset robotic arm at multiple time points from each interaction trajectory data; sample the action data of the virtual model of the preset robotic arm at multiple time points from each interaction trajectory data; wherein the action data includes: the posture of the virtual model of the preset workpiece and the posture of the end of the virtual model of the preset robotic arm; sample the reward data for completing the same interaction task at multiple time points from each interaction trajectory data; sample the target position of the same interaction task at multiple time points from each interaction trajectory data; sample the completed target position at multiple time points from each interaction trajectory data; and generate multiple transfer data respectively according to the interaction observation data, action data, reward data, target position and completed target position at multiple time points.

[0165] In a possible implementation example, the update module 904 is also used to determine the playback target position at each time point from the candidate positions in the candidate playback target set at each time point; according to the playback target position at each time point, the completed target position in the transfer data corresponding to each time point is updated to obtain updated transfer data.

[0166] In a possible implementation example, the update module 904 is further used to calculate the distance parameter corresponding to each candidate position according to each candidate position at each time point and the target position of the same interactive task; calculate the similarity parameter corresponding to each candidate position according to each candidate position at each time point and the completed target position in the transfer data at each time point; perform weighted processing according to the distance parameter corresponding to each candidate position and the similarity parameter corresponding to each candidate position to obtain the distance evaluation parameter of each candidate position; and select the candidate position with the highest distance evaluation parameter as the playback target position at each time point according to the distance evaluation parameter of each candidate position.

[0167] In a possible implementation example, the training module 905 is also used to calculate the target reward data of multiple time points respectively according to the updated multiple transfer data; update the reward data of each time point in the updated multiple transfer data to the target reward data of each time point to obtain multiple target transfer data; train the initial robotic arm control model according to the multiple target transfer data to generate a target robotic arm control model.

[0168] In a possible implementation example, the training device of the robotic arm control model also includes a calculation module, which is used to calculate a first distance parameter at a first time point based on the completed target position at a second time point before the first time point in the updated multiple transfer data and the target position of the same interactive task; calculate the second distance parameter at the first time point based on the completed target position at the first time point in the updated multiple transfer data and the target position of the same interactive task; calculate the target reward data at the first time point based on the first distance parameter and the second distance parameter.

[0169] In a possible implementation example, the calculation module is also used to calculate the progress reward data at the first time point based on the difference between the first distance parameter and the second distance parameter; calculate the target reward data at the first time point based on the progress reward data at the first time point and the reward data at the first time point after the updated multiple transfer data.

[0170] In a possible implementation example, the training device for the robotic arm control model also includes a testing module, which is used to update the parameters of the initial robotic arm control model according to multiple target transfer data; using the initial robotic arm control model after parameter update, controlling the virtual model of the preset robotic arm to perform interactive tasks on the virtual model of the preset workpiece, and obtaining the control success rate of the initial robotic arm control model; if the success rate does not meet the preset threshold, re-controlling the virtual model of the initial robotic arm to perform multiple interactive tasks on the virtual model of the preset workpiece until the control success rate of the target robotic arm control model meets the preset threshold.

[0171] The above-mentioned device is used to execute the method provided by the aforementioned embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0172] The above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASICs), or one or more microprocessors, or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0173] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0174] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0175] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0176] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to perform some steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (English: Read-Only Memory, abbreviated: ROM), random access memory (English: Random Access Memory, abbreviated: RAM), disk or optical disk and other media that can store program codes.

[0177] Figures 10(a) and 10(b) are schematic diagrams of the structure of a robotic arm control system provided in an embodiment of the present application. As shown in Figures 10(a) and 10(b), the robotic arm control system includes: a computer device 1, a preset robotic arm 2, a controller 3 and an environment acquisition module 4.

[0178] Among them, the computer device is pre-integrated with a target robotic arm control model and a virtual working environment, wherein the virtual working environment has a virtual model of a preset robotic arm and a virtual model of a preset workpiece, and the target robotic arm control model is a model trained by the training method of the robotic arm control model of the above-mentioned embodiment.

[0179] The environment acquisition module can be, for example, a camera with a structured light illumination function, which can be used to collect point cloud environment parameters of the actual working environment. The computer device is connected to the environment acquisition module in communication. Based on the communication connection, the computer device can obtain the point cloud environment parameters collected by the environment acquisition module, and generate a virtual working environment based on the point cloud environment parameters and a preset target position. The computer device can also use the target robot arm control model to generate robot arm control parameters. The robot arm control parameters are used to control the virtual model of the preset robot arm in the virtual working environment, so that the virtual model of the preset robot arm performs multiple interactive tasks on the virtual model of the preset workpiece in the virtual working environment. In this embodiment, the robot arm control parameters can be the rotation angle of each joint of the robot arm, or the torque of each joint of the robot arm.

[0180] The computer device is also communicatively connected to the controller, and the controller is connected to the preset robotic arm, so that when the computer device controls the virtual model of the preset robotic arm in the virtual working environment to perform interactive operations on the virtual model of the preset workpiece based on the robotic arm control parameters, the preset robotic arm in the actual working environment is controlled by the controller to perform interactive operations on the preset workpiece. That is, by setting the controller, the computer device can simultaneously control the virtual model of the preset robotic arm in the virtual working environment and the preset robotic arm in the actual working environment.

[0181] Optionally, the robot control system may further include preset workpieces 5 and storage boxes 6. The storage boxes may be two or other numbers, and the number of storage boxes may be determined according to actual needs. In this embodiment, the number of storage boxes is two, and the preset robot can sort the two workpieces and place the two workpieces in different storage boxes.

[0182] The robotic arm control system may also include a power supply 7 and a transformer 8. The power supply is connected to the computer device, the preset robotic arm, the controller, and the environment acquisition module to supply power to the computer device, the preset robotic arm, the controller, and the environment acquisition module. The transformer is used to convert the power supply voltage into voltages of different values ​​that can be used by the computer device, the preset robotic arm, the controller, and the environment acquisition module.

[0183] In summary, this embodiment provides a robotic arm control system, including: a computer device, a preset robotic arm, a controller and an environment acquisition module. The environment acquisition module can collect point cloud data of the actual working environment and send the point cloud data to the computer device. The computer device can generate a virtual working environment according to the point cloud data, and according to the trained target robotic arm control model, control the virtual model of the preset robotic arm in the virtual working environment to perform interactive operations on the virtual model of the preset workpiece. The computer device can also send control information to the controller to control the preset robotic arm in the actual working environment through the controller to perform interactive operations on the preset workpiece.

[0184] Fig.11 A schematic diagram of the structure of a computer device provided in an embodiment of the present application is shown in FIG. Fig.11 As shown, the computer device includes: a processor 100, a storage medium 200 and a bus 300. The storage medium stores program instructions executable by the processor. When the computer device is running, the processor and the storage medium communicate through the bus, and the processor executes the program instructions to perform the steps of the training method of the above-mentioned robot arm control model.

[0185] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the training method for the above-mentioned robot arm control model are executed.

[0186] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A training method for a robot arm control model, characterized in that: include: Using a preset initial robot arm control model, controlling a virtual model of a preset robot arm in a virtual working environment to perform multiple interactive tasks on a virtual model of a preset workpiece in the virtual working environment, and generating multiple interactive trajectory data; wherein each interactive trajectory data is trajectory data of the virtual model of the preset robot arm performing one interactive task; Sampling a plurality of transfer data from each of the interaction trajectory data, the plurality of transfer data comprising: process data of a plurality of time points at which the virtual model of the preset robot arm executes the same interaction task; Determine, from the plurality of transfer data, candidate playback target sets for the plurality of time points, wherein the candidate playback target set for each time point includes: a completed target position of the virtual model of the preset workpiece at at least one time point after each time point; According to the candidate playback target set at each time point, updating the completed target position in the transfer data corresponding to each time point to obtain an updated transfer data; Training the initial robot arm control model according to the updated multiple transfer data to generate a target robot arm control model; The sampling of a plurality of transfer data from each interaction trajectory data comprises: Sampling the interaction observation data of the virtual model of the preset robot arm at the multiple time points from each interaction trajectory data; Sampling the motion data of the virtual model of the preset robot arm at the multiple time points from each of the interaction trajectory data; wherein the motion data includes: the position and posture of the virtual model of the preset workpiece and the position and posture of the end of the virtual model of the preset robot arm; Sampling reward data for completing the same interactive task at the multiple time points from each of the interactive trajectory data; Sampling the target position of the same interactive task at the multiple time points from each of the interactive trajectory data; Sampling completed target positions at the plurality of time points from each interaction trajectory data; The multiple transfer data are generated respectively according to the interaction observation data, action data, reward data, target position and completed target position at the multiple time points.

2. The method according to claim 1, characterized in that The updating of the completed target position in the transfer data corresponding to each time point according to the candidate playback target set at each time point to obtain an updated transfer data includes: Determine the playback target position at each time point from each candidate position in the candidate playback target set at each time point; According to the playback target position at each time point, the completed target position in the transfer data corresponding to each time point is updated to obtain the updated transfer data.

3. The method according to claim 2, characterized in that Determining the playback target position at each time point from each candidate position in the candidate playback target set at each time point includes: According to each candidate position at each time point and the target position of the same interactive task, calculating the distance parameter corresponding to each candidate position; Calculate, according to each candidate position at each time point and the completed target position in the transfer data at each time point, a similarity parameter corresponding to each candidate position; Performing weighted processing according to the distance parameters corresponding to the candidate positions and the similarity parameters corresponding to the candidate positions to obtain the distance evaluation parameters of the candidate positions; According to the distance evaluation parameters of the candidate positions, the candidate position with the highest distance evaluation parameter is selected as the playback target position at each time point.

4. The method according to claim 1, characterized in that: The training of the initial robot arm control model according to the updated multiple transfer data to generate a target robot arm control model includes: Calculating the target reward data at the multiple time points respectively according to the updated multiple transfer data; Updating the reward data at each time point in the updated multiple transfer data to the target reward data at each time point to obtain multiple target transfer data; The initial robot arm control model is trained according to the multiple target transfer data to generate the target robot arm control model.

5. The method according to claim 4, characterized in that The step of calculating the target reward data at the plurality of time points respectively according to the updated plurality of transfer data comprises: Calculate a first distance parameter at the first time point according to the completed target position at a second time point before the first time point in the updated plurality of transfer data and the target position of the same interactive task; Calculating a second distance parameter at the first time point according to the completed target positions at the first time point in the updated plurality of transfer data and the target position of the same interactive task; Calculate target reward data at the first time point according to the first distance parameter and the second distance parameter.

6. The method according to claim 5, characterized in that The step of calculating target reward data at the first time point according to the first distance parameter and the second distance parameter includes: Calculating the progress reward data at the first time point according to the difference between the first distance parameter and the second distance parameter; The target reward data at the first time point is calculated based on the progress reward data at the first time point and the reward data at the first time point after the updated plurality of transfer data.

7. The method according to claim 4, characterized in that The step of training the initial robot arm control model according to the plurality of target transfer data to generate the target robot arm control model comprises: updating the parameters of the initial robot arm control model according to the plurality of target transfer data; Using the initial robot arm control model after parameter update, controlling the virtual model of the preset robot arm to perform an interactive task on the virtual model of the preset workpiece, and obtaining a control success rate of the initial robot arm control model; If the success rate does not meet the preset threshold, the virtual model of the initial robot arm is re-controlled to perform multiple interactive tasks on the virtual model of the preset workpiece until the control success rate of the target robot arm control model meets the preset threshold.

8. A training device for a robot arm control model, characterized in that: include: A control module, used to use a preset initial robot arm control model to control a virtual model of a preset robot arm in a virtual working environment to perform multiple interactive tasks on a virtual model of a preset workpiece in the virtual working environment, and generate multiple interactive trajectory data; wherein each interactive trajectory data is trajectory data of the virtual model of the preset robot arm performing one interactive task; A collection module, used for sampling a plurality of transfer data from each interaction trajectory data, wherein the plurality of transfer data include: process data of a plurality of time points at which the virtual model of the preset robot arm executes the same interaction task; A determination module, configured to determine, from the plurality of transfer data, candidate playback target sets at the plurality of time points, wherein the candidate playback target set at each time point comprises: a completed target position of the virtual model of the preset workpiece at at least one time point after each time point; An updating module, configured to update the completed target position in the transfer data corresponding to each time point according to the candidate playback target set at each time point, to obtain an updated transfer data; A training module, used for training the initial robot arm control model according to the updated multiple transfer data to generate a target robot arm control model; The acquisition module is also used to sample the interaction observation data of the virtual model of the preset robot arm at the multiple time points from each interaction trajectory data; sample the action data of the virtual model of the preset robot arm at the multiple time points from each interaction trajectory data; wherein the action data includes: the posture of the virtual model of the preset workpiece and the posture of the end of the virtual model of the preset robot arm; sample the reward data for completing the same interaction task at the multiple time points from each interaction trajectory data; sample the target position of the same interaction task at the multiple time points from each interaction trajectory data; sample the completed target position at the multiple time points from each interaction trajectory data; and generate the multiple transfer data respectively according to the interaction observation data, action data, reward data, target position and completed target position at the multiple time points.

9. A robotic arm control system, characterized in that: The robot control system comprises: a computer device, a preset robot, a controller and an environment acquisition module; the target robot control model and a virtual working environment are pre-integrated on the computer device, wherein the virtual working environment has a virtual model of the preset robot and a virtual model of the preset workpiece; the target robot control model is a model trained by the robot control model training method according to any one of claims 1 to 7; The computer device is in communication connection with the environment acquisition module to obtain environmental parameters in the actual working environment where the preset robot arm is located, and based on the environmental parameters and the preset target position, the target robot arm control model is used to generate robot arm control parameters; The computer device is communicatively connected to the controller, and the controller is connected to the robotic arm, so that when the computer device controls the virtual model of the preset robotic arm in the virtual working environment to perform interactive operations on the virtual model of the preset workpiece based on the robotic arm control parameters, the preset robotic arm in the actual working environment is controlled by the controller to perform interactive operations on the preset workpiece.

Images