Robot low-cost simulation training method and system and electronic equipment

Abstract

The invention relates to the technical field of robot training scheme design, in particular to a robot low-cost simulation training method and system and electronic equipment. The target object is generated at the random position of the preset area of the simulation environment, the preset motion strategy of the mechanical arm in the simulation environment is configured, the mechanical arm is controlled to grab the target object according to the preset motion strategy, and in the grabbing process, the object information of the mechanical arm is obtained; obtaining object information obtained in the capturing process and recording the object information as a training data set; and the steps are repeated, and a first preset number of training data sets are obtained and used for training of the mechanical arm automatic training model. According to the method, the training data does not need to be collected manually, the working efficiency is improved to a great extent, more efficient data support is provided for model training, the application effect of the model is remarkably improved, and the method has wide application prospects and huge practical value.

Description

Technical Field

[0001] The present invention relates to the technical field of robot training scheme design, and particularly to a low-cost simulation training method and system for robots, and an electronic device. Background Art

[0002] In the current era of rapid technological development, model training, as a core link in many fields (such as artificial intelligence, machine learning, deep learning, data analysis and prediction, image recognition, natural language processing, etc.), plays a crucial role in promoting technological progress and practical applications. However, the existing technologies face many severe challenges in this key process, and the problem of the huge human cost and time cost consumed in collecting training data sets is particularly prominent.

[0003] In the initial stage of model training, it is necessary to construct a high-quality and representative training data set so that the model can learn sufficient features and patterns from it to achieve accurate and efficient performance. However, obtaining such a data set is not easy and involves multiple complex and cumbersome processes, and each link requires a large amount of human and time investment.

[0004] With the development of technology and the continuous growth of application requirements, the requirements for the scale and diversity of the data set in model training are also getting higher and higher. In order to improve the generalization ability and adaptability of the model, it is often necessary to collect and process a large amount of data. However, the existing data collection and preparation methods are difficult to meet this requirement. The traditional manual collection and annotation methods are not only inefficient but also prone to errors and inconsistencies when facing large-scale data, which affect the data quality and model performance.

[0005] In summary, in the existing technology, in the process of model training, the problem of the huge human cost and time cost required to collect training data sets seriously restricts the training efficiency and application effect of the model and becomes one of the key factors hindering the further development of fields such as artificial intelligence. Therefore, there is an urgent need for a more efficient, accurate and low-cost training data set collection and preparation method to solve the above problems existing in the existing technology and provide better support for model training.

[0006] Therefore, the existing technology still needs to be further developed. Summary of the Invention

[0007] The purpose of the present invention is to overcome the above technical deficiencies and provide a low-cost simulation training method and system for robots, and an electronic device to solve the problems existing in the existing technology.

[0008] To achieve the above technical purpose, according to the first aspect of the present invention, the present invention provides a low-cost simulation training method for robots, and the method includes: S100. Initialize the state of the robotic arm, generate a target object at a random position in a preset area of the simulation environment, configure a preset motion strategy for the robotic arm in the simulation environment, control the robotic arm to grasp the target object according to the preset motion strategy, and during this grasping process, obtain the object information of the robotic arm at each preset time point corresponding to the preset time point list, and record the object information obtained during this grasping process as a training data set; S200. Repeat the execution of S100 until the first preset number of training data sets are obtained. Divide the obtained training data sets into a training data set and a validation data set according to the first preset ratio; use the training data set to train the automatic training model of the robotic arm; S200. Use the validation data set to optimize the trained automatic training model of the robotic arm, optimize the model parameters, and output the optimized model parameter file.

[0009] Specifically, the object information includes at least one of the following: The position information of the robotic arm, the speed information of the robotic arm, the force information of the robotic arm, the angle information of each joint of the robotic arm, and the image information of the virtual camera set on the robotic arm.

[0010] Specifically, the robotic arm includes a left robotic arm and a right robotic arm. Grippers are respectively provided on the left robotic arm and the right robotic arm. The grippers are used to grasp the target object. Configuring the preset motion strategy of the robotic arm in the simulation environment includes: Preset the positions that the left robotic arm needs to reach at each preset time point in the preset time point list; Preset the action states that the gripper of the left robotic arm needs to execute at each preset time point in the preset time point list; Preset the positions that the right robotic arm needs to reach at each preset time point in the preset time point list; Preset the action states that the gripper of the right robotic arm needs to execute at each preset time point in the preset time point list.

[0011] Specifically, the method further includes: Perform the grasping path planning of the left robotic arm according to the positions that the preset left robotic arm needs to reach at each preset time point in the preset time point list; Perform the grasping path planning of the right robotic arm according to the positions that the preset right robotic arm needs to reach at each preset time point in the preset time point list.

[0012] Specifically, controlling the robotic arm to grasp the target object according to the preset motion strategy includes: Control the right robotic arm to grasp the target object. After the right robotic arm grasps the target object, control the right robotic arm to transfer the target object to the left robotic arm, and control the left robotic arm to place the target object at the target position.

[0013] Specifically, the target object includes a first target object and a second target object. The first target object is a plug-in of the second target object. The control of the robotic arm to grasp the target object according to a preset motion strategy includes: Controlling the right robotic arm to grasp the first target object, controlling the left robotic arm to grasp the second target object, and controlling the right robotic arm to plug the first target object onto the second target object.

[0014] Specifically, the control of the right robotic arm to grasp the target object, after the right robotic arm grasps the target object, controlling the right robotic arm to transfer the target object to the left robotic arm, and controlling the left robotic arm to place the target object at the target position, includes the following processes: Controlling the right robotic arm to grasp the target object. After the right robotic arm grasps the target object, controlling the right robotic arm to reach the first preset meeting position at the first preset time point; Controlling the right robotic arm to rotate at the first preset meeting position until it reaches the preset transfer posture; Controlling the left robotic arm to reach the second preset meeting position at the second preset time point; Controlling the gripper of the left robotic arm to close; Controlling the gripper of the right robotic arm to open; Completing the transfer of the target object; Controlling the left robotic arm to place the target object at the target position.

[0015] Specifically, the initialization of the state of the robotic arm includes: Controlling the robotic arm to move to the preset initial position and controlling the joint angles of the robotic arm to move to the preset initial state.

[0016] According to the second aspect of the present invention, a low-cost simulation training system for a robot is provided, including: An acquisition module, configured to obtain the object information of the robotic arm at each preset time point corresponding to the preset time point list during the process of grasping the target object, and record the object information obtained in each grasping process as a training data set; and configured to obtain a first preset number of training data sets; A control module, configured to initialize the state of the robotic arm, generate a target object at a random position in a preset area of the simulation environment, configure a preset motion strategy of the robotic arm in the simulation environment; configured to divide the obtained training data set into a training data set and a validation data set according to a first preset ratio; use the training data set to train an automatic training model of the robotic arm; and configured to optimize the trained automatic training model of the robotic arm using the validation data set, optimize the model parameters, and output an optimized model parameter file.

[0017] According to a third aspect of the present invention, there is provided an electronic device, comprising: a memory; and a processor, wherein computer-readable instructions are stored on the memory, and when the computer-readable instructions are executed by the processor, the above-mentioned low-cost simulation training method for a robot is implemented.

[0018] Beneficial effects: The present invention realizes that training data does not need to be collected manually, greatly improves work efficiency, can collect and train data in multiple environments and scenarios, enhances the robustness of the model. At the same time, the present invention can enable the server to continuously collect training data sets, and the efficiency of collecting data sets by teleoperation in the past has increased exponentially. The simulation environment and motion strategy in the present invention can be customized or made according to user needs to meet the personalized needs of customers. The present invention provides more efficient data support for model training, significantly improves the application effect of the model, and has broad application prospects and great practical value. Description of the drawings

[0019] Figure 1 is a flowchart of the low-cost simulation training method for a robot provided in a specific embodiment of the present invention; Figure 2 is a schematic diagram of the system composition of the low-cost simulation training system for a robot provided in a specific embodiment of the present invention; Figure 3 is a schematic diagram of the specific process of configuring the robot simulation environment provided in a specific embodiment of the present invention; Figure 4 is a schematic diagram of the process of generating a target object in the simulation environment provided in a specific embodiment of the present invention; Figure 5 is a schematic diagram of the structure of the robotic arm automatic training model provided in a specific embodiment of the present invention; Figure 6 is a schematic diagram of the forward propagation of the robotic arm automatic training model provided in a specific embodiment of the present invention; Figure 7 is a schematic diagram of the design process of the motion strategy provided in a specific embodiment of the present invention; Figure 8 is a schematic diagram of the installation position of the virtual camera provided in a specific embodiment of the present invention. Specific embodiments

[0020] To enable those skilled in the art to better understand the technical solution of the present invention, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the present invention. Based on the embodiments in this application, other similar embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of this application. In addition, the directional terms mentioned in the following embodiments, such as "up", "down", "left", "right", etc., are only with reference to the directions in the accompanying drawings. Therefore, the directional terms used are for illustration rather than to limit the present invention.

[0021] The present invention will be further described below in conjunction with the accompanying drawings and preferred embodiments.

[0022] First of all, it should be noted that the present invention is directed to the intelligent training of a dual-arm robot. Since the labor cost and time cost required to collect a data set in the real environment are huge, an action strategy and environmental parameter configuration are designed through a simulation environment, and the dual-arm robot is trained in the simulation environment to perform target actions in different environments. Finally, it is directly applied to the dual-arm robot in the real display environment, omitting the data collection process and greatly improving the efficiency. The specific process is as Figure 1 shown; the construction of the simulation environment is also involved in the process, that is Figure 3 shown; the construction of the target object to be operated in the simulation environment, that is Figure 4 shown; the compilation of the motion strategy, that is Figure 7 shown: Please refer to Figures 1 - 8 , the present invention provides a low-cost simulation training method for a robot, including: S100. Initialize the state of the robotic arm, generate a target object at a random position in a preset area of the simulation environment, configure a preset motion strategy of the robotic arm in the simulation environment, control the robotic arm to grasp the target object according to the preset motion strategy, and during this grasping process, obtain the object information of the robotic arm at each preset time point corresponding to the preset time point list, and record the object information obtained during this grasping process as a training data set.

[0023] Specifically, the object information includes at least one of the following: The position information of the robotic arm, the speed information of the robotic arm, the force information of the robotic arm, the angle information of each joint of the robotic arm, and the image information of the virtual camera set on the robotic arm.

[0024] Specifically, the robotic arm includes a left robotic arm and a right robotic arm, and grippers are respectively arranged on the left robotic arm and the right robotic arm. The grippers are used to grasp the target object. The configuration of the preset motion strategy of the robotic arm in the simulation environment includes: Preset the positions that the left robotic arm needs to reach at each preset time point in the preset time point list; The action states required for the gripper of the preset left robotic arm at each preset time point in the preset time point list.

[0025] The positions that the preset right robotic arm needs to reach at each preset time point in the preset time point list; The action states required for the gripper of the preset right robotic arm at each preset time point in the preset time point list.

[0026] In a preferred embodiment, refer to Figure 8 , the camera includes a total of 4 cameras: a top camera, a front camera, a left-arm camera, and a right-arm camera (the top and front cameras can be reduced according to requirements). When the camera includes a first camera 9 provided at the front end of the left robotic arm 1, a second camera 8 provided at the front end of the right robotic arm 2, a third camera 5 provided at the edge of the chassis 3, and a fourth camera 7 provided in the middle of the bracket 4, a total of four cameras, the fourth camera 7 provided in the middle of the bracket 4 and the third camera 5 provided at the edge of the chassis 3 can be selected to be deleted according to the actual needs of the user. The specific camera settings can be reduced or increased according to the user's needs, which can effectively identify the position of the target object 6 and help predict the next position during the forward propagation of the model.

[0027] Specifically, the code defining the motion strategy of the left robotic arm includes: self.left_trajectory = {"t": 0, "xyz": init_mocap_pose_left[:3], "quat": init_mocap_pose_left[3:], "gripper": 0} {"t": 100, "xyz": meet_xyz + np.array([-0.1, 0, -0.02]), "quat":meet_left_quat.elements, "gripper": 1} {"t": 260, "xyz": meet_xyz + np.array([0.02, 0, -0.02]), "quat":meet_left_quat.elements, "gripper": 1} {"t": 310, "xyz": meet_xyz + np.array([0.02, 0, -0.02]), "quat":meet_left_quat.elements, "gripper": 0} {"t": 360, "xyz": meet_xyz + np.array([-0.1, 0, -0.02]), "quat":np.array([1, 0, 0, 0]), "gripper": 0} {"t": 400, "xyz": meet_xyz + np.array([-0.1, 0, -0.02]), "quat":np.array([1, 0, 0, 0]), "gripper": 0}] It can be understood that the above code defines the following actions of the left robotic arm: 1. t = 0: Initial position, gripper closed (`gripper = 0`).

[0028] 2. t = 100: Approach the second preset meeting position, open the gripper (`gripper = 1`).

[0029] 3. t = 260: Move to the second preset meeting position, keep the gripper open.

[0030] 4. t = 310: Close the gripper at the second preset meeting position, prepare to grasp the object.

[0031] 5. t = 360 & t = 400: Move to a position slightly to the left and stay, the gripper remains closed.

[0032] Specifically, the code defining the motion strategy of the right robotic arm includes: self.right_trajectory = {"t": 0, "xyz": init_mocap_pose_right[:3], "quat": init_mocap_pose_right[3:], "gripper": 0} {"t": 90, "xyz": box_xyz + np.array([0, 0, 0.08]), "quat": gripper_pick_quat.elements, "gripper": 1} {"t": 130, "xyz": box_xyz + np.array([0, 0, -0.015]), "quat":gripper_pick_quat.elements, "gripper": 1} {"t": 170, "xyz": box_xyz + np.array([0, 0, -0.015]), "quat":gripper_pick_quat.elements, "gripper": 0} {"t": 200, "xyz": meet_xyz + np.array([0.05, 0, 0]), "quat": gripper_pick_quat.elements, "gripper": 0} {"t": 220, "xyz": meet_xyz, "quat": gripper_pick_quat.elements, "gripper": 0} {"t": 310, "xyz": meet_xyz, "quat": gripper_pick_quat.elements, "gripper": 1} {"t": 360, "xyz": meet_xyz + np.array([0.1, 0, 0]), "quat": gripper_pick_quat.elements, "gripper": 1} {"t": 400, "xyz": meet_xyz + np.array([0.1, 0, 0]), "quat": gripper_pick_quat.elements, "gripper": 1} It can be understood that the above code defines the following actions for the right robotic arm: 1. t = 0: Initial position, gripper closed.

[0033] 2. t = 90: Approach the object, adjust above the object and slightly raise (increase in z-axis), open the gripper.

[0034] 3. t = 130: Lower to slightly below the object, prepare to grasp.

[0035] 4. t = 170: Close the gripper at the target position, perform the grasping action.

[0036] 5. t = 200: Indicate approaching the first preset meeting position, gripper closed.

[0037] 6. t = 220: Indicate moving to the first preset meeting position, gripper closed. ​

[0038] 7. t = 310: It means to open the gripper at the first preset meeting position.

[0039] 8. t = 360: It means to move to the right and the gripper remains open.

[0040] 9. t = 400: It means to stay at the position after moving to the right and the gripper is open.

[0041] Specifically, the method further includes: Performing the grasping path planning of the left robotic arm according to the positions that the preset left robotic arm needs to reach at each preset time point in the preset time point list; Performing the grasping path planning of the right robotic arm according to the positions that the preset right robotic arm needs to reach at each preset time point in the preset time point list.

[0042] Specifically, controlling the robotic arm to grasp the target according to the preset motion strategy includes: Controlling the right robotic arm to grasp the target. After the right robotic arm grasps the target, controlling the right robotic arm to transfer the target to the left robotic arm, and controlling the left robotic arm to place the target at the target position.

[0043] Specifically, controlling the right robotic arm to grasp the target, after the right robotic arm grasps the target, controlling the right robotic arm to transfer the target to the left robotic arm, and controlling the left robotic arm to place the target at the target position includes the following processes: Controlling the right robotic arm to grasp the target. After the right robotic arm grasps the target, controlling the right robotic arm to reach the first preset meeting position at the first preset time point; Controlling the right robotic arm to rotate at the first preset meeting position until it reaches the preset handover posture; Controlling the left robotic arm to reach the second preset meeting position at the second preset time point; Controlling the gripper of the left robotic arm to close; Controlling the gripper of the right robotic arm to open; Completing the handover of the target; Controlling the left robotic arm to place the target at the target position.

[0044] Specifically, initializing the state of the robotic arm includes: Controlling the robotic arm to move to the preset initial position and controlling the joint angles of the robotic arm to move to the preset initial state.

[0045] Specifically, controlling the robotic arm to move to the preset initial position includes: Controlling the right robotic arm to move to the first preset initial position; Controlling the left robotic arm to move to the second preset initial position.

[0046] In a preferred embodiment, the target object includes a first target object and a second target object. The first target object is a plug-in of the second target object. Controlling the robotic arm to grasp the target object according to a preset motion strategy includes: Controlling the right robotic arm to grasp the first target object, controlling the left robotic arm to grasp the second target object, and controlling the right robotic arm to plug the first target object onto the second target object.

[0047] Specifically, the code defining the motion strategy of the left robotic arm includes: self.left_trajectory = {"t": 0, "xyz": init_mocap_pose_left[:3], "quat": init_mocap_pose_left[3:], "gripper": 0} {"t": 120, "xyz": socket_xyz + np.array([0, 0, 0.08]), "quat":gripper_pick_quat_left.elements, "gripper": 1} {"t": 170, "xyz": socket_xyz + np.array([0, 0, -0.03]), "quat":gripper_pick_quat_left.elements, "gripper": 1} {"t": 220, "xyz": socket_xyz + np.array([0, 0, -0.03]), "quat":gripper_pick_quat_left.elements, "gripper": 0} {"t": 285, "xyz": meet_xyz + np.array([-0.1, 0, 0]), "quat": gripper_pick_quat_left.elements, "gripper": 0} {"t": 340, "xyz": meet_xyz + np.array([-0.05, 0, 0]), "quat":gripper_pick_quat_left.elements,"gripper": 0} {"t": 400, "xyz": meet_xyz + np.array([-0.05, 0, 0]), "quat":gripper_pick_quat_left.elements, "gripper": 0} It can be understood that the above code defines the following actions for the left robotic arm: 1. t = 0: Initial position, representing the initial state, gripper: 0, indicating that the gripper is open.

[0048] 2. t = 120: The left arm moves to a position 0.08 units above socket_xyz, preparing to grasp.

[0049] 3. t = 170: At the 170th time unit, the left arm moves down 0.03 units, keeping the gripper closed to grasp the object.

[0050] 4. t = 220: The left arm remains in the same position, but the gripper opens.

[0051] 5. t = 285: The left arm moves to a position 0.1 unit to the left of meet_xyz, preparing to meet the right arm.

[0052] 6. t = 340 - 400: At the 340th and 400th time units, the left arm remains in a position 0.05 units to the left of meet_xyz, performing the insertion action.

[0053] Specifically, the code defining the motion strategy of the right robotic arm includes: self.right_trajectory = {"t": 0, "xyz": init_mocap_pose_right[:3], "quat": init_mocap_pose_right[3:], "gripp {"t": 120, "xyz": peg_xyz + np.array([0, 0, 0.08]), "quat": gripper_pick_quat_right.elements, "gripper": 1}, # approach the cube ​{"t": 170, "xyz": peg_xyz + np.array([0, 0, -0.03]), "quat": gripper_pick_quat_right.elements, "gripper": 1}, # go down {"t": 220, "xyz": peg_xyz + np.array([0, 0, -0.03]), "quat": gripper_pick_quat_right.elements, "gripper": 0}, # close gripper {"t": 285, "xyz": meet_xyz + np.array([0.1, 0, lift_right]), "quat":gripper_pick_quat_right.elements, "gripper": 0}, # approach meet position {"t": 340, "xyz": meet_xyz + np.array([0.05, 0, lift_right]), "quat":gripper_pick_quat_right.elements, "gripper": 0}, # insertion {"t": 400, "xyz": meet_xyz + np.array([0.05, 0, lift_right]), "quat":gripper_pick_quat_right.elements, "gripper": 0}, # insertion It can be understood that the above code defines the following actions of the right robotic arm: 1. t = 0: Initial position, representing the initial state, gripper: 0, indicating that the gripper is open.

[0054] 2. t = 120: The right arm moves to a position 0.08 units above socket_xyz, preparing to grasp.

[0055] 3. t = 170: At the 170 - time unit, the right arm moves down 0.03 units, keeping the gripper closed to grasp the object.

[0056] 4. t = 220: The right arm remains in the same position, but the gripper opens. ​

[0057] 5. t = 285: At time unit 285, the right arm moves 0.1 unit to the right of meet_xyz and lifts by lift_right units in height, preparing to meet the left arm.

[0058] 6. t = 340 - 400: At time units 340 and 400, the right arm remains 0.05 unit to the right of meet_xyz and lifts by lift_right units in height, performing an insertion action or maintaining the position and waiting.

[0059] It should be noted here that before step S100, it includes: In the control module, a first preset quantity, a first preset ratio, a first preset number of rounds, a second preset number of rounds, a preset motion strategy, a first preset time point, a second preset time point, a list of preset time points, a first preset meeting position, a second preset meeting position, a preset initial position, a first preset initial position, and a second preset initial position are preset in advance.

[0060] It can be understood that the present invention does not limit the specific values of the first preset quantity, the first preset ratio, the first preset number of rounds, the second preset number of rounds, the preset motion strategy, the first preset time point, the second preset time point, the list of preset time points, the first preset meeting position, the second preset meeting position, the preset initial position, the first preset initial position, and the second preset initial position, as long as they can be applied to the robotic arm automatic training method based on deep learning proposed by the present invention.

[0061] Preferably, the present invention sets the first preset quantity to 50, sets the first preset ratio to 9:1, sets the first preset number of rounds to 20000 times, and sets the second preset number of rounds to 200. The above settings can further improve the performance of the model and the efficiency of model training, and greatly improve the accuracy and reliability of the prediction of the robotic arm automatic training model of the present invention.

[0062] S200. Repeat S100 until a first preset quantity of training data sets are obtained, and divide the obtained training data sets into training data sets and validation data sets according to the first preset ratio; use the training data sets to train the robotic arm automatic training model.

[0063] Specifically, using the training data sets to train the robotic arm automatic training model includes: The training data set is input into a preset network layer in batches for training. The preset network layer includes a ResNet network layer and a Transformer network layer. The ResNet network layer is used to identify the position of the target object through the image information of the camera in the training data set and the validation data set. The Transformer network layer is used to predict the next position through forward propagation based on the current position, and then obtain the predicted loss value; Calculate the loss value of the preset network layer and input it into the optimizer for optimization to determine the direction in which the parameter gradient of the robotic arm automatic training model drops fastest; The robotic arm automatic training model performs backpropagation based on the loss value and the parameters of the model to optimize the parameters of the robotic arm automatic training model.

[0064] It should be noted here that ResNet is a deep convolutional neural network. It solves the problem of gradient disappearance in the training of deep networks by introducing residual connections. The core component of ResNet is the residual block. These blocks usually contain two or three convolutional layers and a skip connection, which allows the gradient to directly bypass these convolutional layers. The network structure of ResNet can be very deep. These networks extract image features by stacking multiple residual blocks and perform classification tasks through global average pooling and fully connected layers; Transformer is a deep learning model based on the self-attention mechanism, composed of a multi-head attention mechanism and a feed-forward neural network, which can process sequence data and capture long-range dependencies. The Transformer model does not contain traditional convolutional layers, but processes each element in the input sequence through attention layers. The typical structure of Transformer includes an encoder and a decoder. The encoder is stacked by multiple identical layers, and the decoder adds a self-attention layer on the basis of the encoder to process sequence generation tasks. In the present invention, the ResNet network layer and the Transformer network layer are combined. ResNet is good at extracting local features in images, while Transformer can supplement global context information. This combination can strengthen the model's ability in feature extraction and global understanding, greatly improve the performance of the model, and significantly reduce the computational cost.

[0065] It can be understood that the ResNet network layer is used to identify the position of the target object through the image information of the camera in the training data set and the validation data set. The specific steps are as follows: (1) First, put the image information into the Backbones network layer of the model. Backbone refers to a series of convolutional layers that form the skeleton of a neural network. The main function of these layers is to extract the features of the input data. The Backbone network usually consists of multiple convolutional layers, pooling layers, and activation functions, and can extract meaningful feature representations from the original data. Backbones are composed of multiple Backbones, and the core network layer in Backbone is the Resnet network layer. After the dataset is input into the model, the image part will be input into Backbones for training; (2) When constructing the Backbone, positional encoding and the Resnet network layer are added. After the image is input, Resnet starts training layer by layer, extracts the features in the image, and assists the Transformer in predicting actions. The extracted features obtain the position information of the image features with the help of positional encoding.

[0066] Specifically, the method of training the robotic arm automatic training model using the training dataset and the validation dataset further includes: After each training, the validation dataset is input into the preset network layer of the previous training in batches for model parameter verification, and cyclic training is performed. The total number of training rounds is set to the first preset round; Record the loss value of the preset network layer, determine whether the loss value meets the first preset condition, and determine whether to end the training and output the model parameters according to the judgment result.

[0067] Specifically, the step of determining whether the loss value meets the first preset condition and determining whether to end the training and output the model parameters according to the judgment result includes: If the loss value meets the first preset condition, end the training and output the parameters of the current robotic arm automatic training model; If the loss value does not meet the first preset condition, continue the training.

[0068] Specifically, the first preset condition is: after the second preset round of training, the loss values obtained in the next round of training are all greater than or equal to the loss values that appeared during the second preset round of training.

[0069] It can be understood that the maximum number of rounds of cyclic training in the present invention is preferably 20,000 times, which can effectively ensure the accuracy of the model prediction of the present invention. The loss value of model training is recorded. The second preset number of rounds in the present invention is set to 200, that is, when the loss value of training no longer decreases within 200 rounds of training, the round with the smallest loss value in 200 rounds of training is saved as the parameters of the best verification round, and the best model file is generated. The above settings enable the number of training rounds to no longer rely on manual judgment, but to use a deep learning model for automatic training, improving the accuracy of model prediction. And when the accuracy reaches the requirement, the training is automatically ended, greatly saving the training time, and can also effectively prevent overfitting, greatly improving the intelligence level of the present invention and the efficiency of model training.

[0070] Further, the specific process of the robotic arm automatic training model of the present invention is as follows: (1) Obtain the mean mu of the latent vector and the logarithmic variance logvar during the forward propagation of the robotic arm automatic training model; (2) Calculate the reasonable loss value loss representing the training effect: Calculate the divergence: k1 = -0.5 * (1 + logvar - mu^2 - (e^logvar)); Calculate the loss value loss between the position information robotic arm ion of the robotic arm and the predicted value: loss = L1 + k1 * k1_weight; Among them, loss is the loss value, k1 is the divergence, k1_weight is the proportion of k1, with a value of 10, and L1 is the absolute difference between the predicted value and the true value. The calculation formula of L1 is as follows: ; Among them, A is the L1 loss in pytorch, which calculates the average or sum of the absolute differences between two tensors; the shape of A is N * C * H * W, which indicates that it is a four-dimensional tensor. N represents the number of samples (Batch Size), indicating that there are N data points. C represents the number of channels (Channels), and H and W respectively represent the height and width, which are usually the two-dimensional spatial dimensions of the input features. P represents the padding value, which is a boolean type. The shape of P is C * H * W * 1, and the dimensions of P in channels, height, and width are the same as those of A, but there is no number of samples N, which means that P is the mask or padding information for a certain feature map of each sample. The boolean value of P may be used to mark whether the elements in A are filled (True) or valid (False).

[0071] (3) Use the validation data set to obtain the loss value during the verification of the current model network; (4) If the verification loss in this round is less than that in the previous round, the best model parameters are replaced with the training model of this time; (5) Put the round number and the loss value loss into a dictionary one by one; (6) Compare the loss value of the current round with the loss values of the previous 200 rounds (parameters are provided and can be set according to actual needs, with a default of 200) of this round, that is, traverse the loss values from epoch - 200 to epoch - 1 (epoch is the training round number, and the comparison is made when epoch is greater than 200). If the loss value of the current round of epoch is greater than the loss values from the (epoch - 200)th to the (epoch - 1)th, stop training.

[0072] (7) Save the best model parameters in the cyclic verification as a ckpt model file.

[0073] It can be understood that the above method can be summarized into the following process: Specifically, the above method includes: Step 1: Build a training model, input the training data set, and select parameters such as batch size and file path to train the model; Step 2: Create a training model. The model mainly uses the ResNet network layer and the Transformer network layer. The specific structure of the model is as Figure 3 shown; Step 3: Put the training data set into the network model in batches. ResNet is responsible for identifying the position of the target object through the image information in the data set, and Transformer then predicts the next position through forward propagation based on the current position to obtain the prediction loss value. The forward propagation is as Figure 4 shown; Step 4: After obtaining the loss value, perform backpropagation, calculate the gradient, and put it into the optimizer to optimize along the gradient direction; Step 5: After completing one training, perform model parameter verification, that is, put the verification data set into the parameter network layer obtained from the previous training in batches, and perform forward propagation to obtain the loss value; Step 6: Perform cyclic training. The training round number can be set to 20,000 times, record the training loss value loss. When the training loss no longer decreases within 200 rounds of training, save the parameters of the best verification round to generate the best model file.

[0074] S200. Use the verification data set to optimize the trained robotic arm automatic training model, optimize the model parameters, and output the optimized model parameter file.

[0075] Specifically, the method for optimizing the trained robotic arm automatic training model using the validation dataset, optimizing the model parameters, and outputting the optimized model parameter file includes: Set the custom robotic arm automatic training model parameters, input the validation dataset into the preset network layer for model parameter verification, and optimize the model parameters according to the output loss value; Perform cyclic training based on the new model parameters until the loss value meets the first preset condition, that is, complete the model optimization, and output the optimized model parameter file.

[0076] It can be understood that when the number of training times is insufficient and model optimization is desired, since the original model can only be trained from scratch, therefore, the present invention develops a method to continue model optimization based on the previously trained model parameter file, set custom parameters, and can choose to continue training based on the training parameters at a certain time according to user needs, obtain the best model, and complete the model optimization, solving the problem in the prior art that the results of previous training cannot be used when model optimization is required, greatly improving the efficiency and accuracy of model training.

[0077] Specifically, the above method includes the following process: (1) Provide custom parameters, and choose to continue training based on the training parameters at a certain time according to user needs; (2) Load the parameters of the custom model file; (3) Put the validation dataset into the custom parameter network layer for verification, obtain the loss value and record the best model parameters; (4) Perform forward propagation training, backpropagation, and optimizer parameter optimization based on this set of model parameters to obtain a new set of data; (5) Put the validation data into the new data network layer for inference to obtain the loss value; (6) Perform cyclic training to obtain the best model and complete the model optimization.

[0078] It can be understood that the process of the low-cost simulation training method for robots proposed by the present invention includes the following steps: P1. Simulation environment configuration; Motion strategy configuration.

[0079] P2. Build the Mujoco environment according to the simulation environment configuration file.

[0080] P3. Call the Mujoco environment and reset the Mujoco simulation environment (initialization).

[0081] P4. Call the physical environment to obtain the initial state information of the physical environment.

[0082] P5. Return the time of the robotic arm in the physical environment, and the observed value (obtain the initial state value of the robotic arm).

[0083] P6. Set the size and number of the collected data sets according to your own requirements. The default number of times is 50. Set the motion frequency of the robotic arm, with the default being episode_len = 400 times; set the fixed quantity DT of the simulation environment, with the default being 0.05. Then, the default time for each task can be calculated as t = DT * episode_len = 0.05 * 400 = 20s.

[0084] P7. Capture the motion strategy, put the initial state value of the robotic arm into the motion strategy, and obtain the joint position and angle information of the robotic arm at each time point.

[0085] P8. Capture the simulation environment, put the position and state information of the robotic arm at each time into the simulation environment, and obtain the environmental information. Here, only the information of the camera is taken.

[0086] P9. Retrieve the robotic arm information and camera information obtained from 'Capture the motion strategy' and 'Capture the simulation environment' and update them to the observed value variable. Loop episode_len, put the observed value into the steps of 'Capture the motion strategy' and 'Capture the simulation environment', and repeat 400 times to obtain the final complete action.

[0087] P10. Save the position, speed, and state information of the robotic arm in the complete action, as well as the camera information, to an hd5 file to complete the collection of a data set; set the size and number of the collected data sets according to your own requirements (the default is 50), and repeat the above steps in a loop to obtain the final training data set (the default is 50 action groups).

[0088] P11. Save the position, speed, and state information of the robotic arm in the complete action, as well as the camera information, to an hd5 file to complete the collection of a data set; set the size and number of the collected data sets according to your own requirements (the default is 50), and repeat the above steps in a loop to obtain the final training data set (the default is 50 action groups).

[0089] P12. Build an ACT (a deep learning network architecture) training model, and select parameters such as the batch size and file path for model training.

[0090] P13. Create a training model. The model mainly uses the ResNet network layer and the transformer network layer. The specific structure of the model is as Figure 5 shown.

[0091] P14. The training data set is put into the network model in batches. ResNet is responsible for identifying the target object position through the image information in the data set, while the transformer predicts the next position through forward propagation based on the current position, and obtains the prediction loss value. The forward propagation is as Figure 6 shown.

[0092] P15. After obtaining the loss value, backpropagation is performed to calculate the gradient, which is then put into the optimizer for optimization along the gradient direction.

[0093] P16. After one training is completed, model parameter verification is carried out. That is, the verification data is put into the parameter network layer obtained from the previous training in batches, and forward propagation is performed to obtain the loss value.

[0094] P17. Loop training: The number of training rounds can be set larger, such as 20000. Record the training loss. When the training loss no longer decreases within 200 training rounds, the parameters of the best verification round are saved to generate the best model file.

[0095] P18. After the model training is completed, the model can be used for automatic inference of the robotic arm in the real environment. That is, the action information of the next step of the current action is obtained according to the above network layer prediction (forward propagation diagram). Then, a ros system is built to transmit the action to the dual-arm robot, enabling it to complete the environmental intelligent processing perception pre-trained in the simulation environment.

[0096] It can be understood that the present invention realizes that the training data does not need to be collected manually, greatly improves the work efficiency, can collect and train data in multiple environments and scenarios, enhances the robustness of the model. At the same time, the present invention allows the server to continuously collect the training data set, and the efficiency of collecting the data set by remote operation in the past increases exponentially. The simulation environment and motion strategy in the present invention can be customized or made according to user needs to meet the personalized needs of customers. The present invention provides more efficient data support for model training, significantly improves the application effect of the model, and has broad application prospects and great practical value.

[0097] Please refer to Figure 2 , the present invention provides another embodiment. The present embodiment provides a low-cost simulation training system for a robot. The low-cost simulation training system for a robot includes: An acquisition module 100, which is used to acquire the object information of the robotic arm at each preset time point corresponding to the preset time point list during the process of grasping the target object, and record the object information obtained in each grasping process as a training data set; and is used to acquire the first preset number of training data sets; The control module 200 is used to initialize the state of the robotic arm, generate a target object at a random position in a preset area of the simulation environment, and configure a preset motion strategy for the robotic arm in the simulation environment; it is used to divide the obtained training data set into a training data set and a validation data set according to a first preset ratio; use the training data set to train the automatic training model of the robotic arm; and use the validation data set to optimize the trained automatic training model of the robotic arm, optimize the model parameters, and output an optimized model parameter file.

[0098] It should be noted here that the present invention realizes that training data does not need to be collected manually, greatly improving work efficiency. It can collect and train data in multiple environments and scenarios, enhancing the robustness of the model. At the same time, the present invention allows the server to continuously collect training data sets, and the efficiency of collecting data sets by remote operation in the past has increased exponentially. The simulation environment and motion strategy in the present invention can be customized or made according to user needs to meet the personalized needs of customers. The present invention provides more efficient data support for model training, significantly improving the application effect of the model, and has broad application prospects and great practical value.

[0099] In a preferred embodiment, the present application also provides an electronic device, which includes: A memory; and a processor, where a computer-readable instruction is stored on the memory, and when the computer-readable instruction is executed by the processor, the low-cost simulation training method of the robot as described above is implemented. This computer device can generally be a server, a terminal, or any other electronic device with necessary computing and / or processing capabilities. In one embodiment, this computer device may include a processor, a memory, a network interface, a communication interface, etc. connected through a system bus. The processor of this computer device can be used to provide necessary computing, processing, and / or control capabilities. The memory of this computer device may include a non-volatile storage medium and an internal memory. An operating system, a computer program, etc. may be stored in or on the non-volatile storage medium. The internal memory can provide an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface and communication interface of this computer device can be used to connect and communicate with external devices through a network. The computer program, when executed by the processor, executes the steps of the method of the present invention.

[0100] The present invention can be implemented as a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, causes the steps of the method of the embodiments of the present invention to be performed. In one embodiment, the computer program is distributed across a plurality of network-coupled computer devices or processors such that the computer program is stored, accessed, and executed by one or more computer devices or processors in a distributed manner. A single method step / operation, or two or more method steps / operations, can be performed by a single computer device or processor or by two or more computer devices or processors. One or more method steps / operations can be performed by one or more computer devices or processors, and one or more other method steps / operations can be performed by one or more other computer devices or processors. One or more computer devices or processors can perform a single method step / operation, or perform two or more method steps / operations.

[0101] Those of ordinary skill in the art will understand that the method steps of the present invention can be completed by a computer program instructing related hardware such as a computer device or a processor. The computer program can be stored in a non-transitory computer-readable storage medium, and when the computer program is executed, the steps of the present invention are caused to be performed. Depending on the circumstances, any reference herein to a memory, storage, database, or other medium may include non-volatile and / or volatile memory. Examples of non-volatile memory include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), flash memory, magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk, etc. Examples of volatile memory include random access memory (RAM), external cache memory, etc.

[0102] It can be understood that the present invention realizes that training data does not need to be collected manually, greatly improves work efficiency, can collect and train data in multiple environments and scenarios, enhances the robustness of the model. At the same time, the present invention can enable the server to continuously collect training data sets, and the efficiency of collecting data sets by remote operation in the past has increased exponentially. The simulation environment and motion strategy in the present invention can be customized or made according to user needs to meet the personalized needs of customers. The present invention provides more efficient data support for model training, significantly improves the application effect of the model, and has broad application prospects and great practical value.

[0103] The technical features described above can be combined arbitrarily. Although not all possible combinations of these technical features are described, any combination of these technical features should be considered to be covered by this specification as long as such a combination does not exist in contradiction.

[0104] The specific embodiments of the present invention described above do not limit the protection scope of the present invention. Any other corresponding changes and modifications made according to the technical concept of the present invention shall be included in the protection scope of the claims of the present invention.

Claims

1. A low-cost simulation training method for robots, characterized in that: The method comprises: S100, initializing the state of the robotic arm, generating a target object at a random position in a preset area of ​​the simulation environment, configuring a preset motion strategy of the robotic arm in the simulation environment, controlling the robotic arm to grasp the target object according to the preset motion strategy, and obtaining object information of the robotic arm at each preset time point corresponding to the preset time point list during the grasping process, and recording the object information obtained during the grasping process as a training data set; S200, repeatedly executing S100 until a first preset number of training data sets are obtained, dividing the obtained training data sets into a training data set and a verification data set according to a first preset ratio; and using the training data sets to train the automatic training model of the robotic arm; S200, using the verification data set to optimize the trained robotic arm automatic training model, optimize the model parameters, and output the optimized model parameter file.

2. The low-cost simulation training method for robots according to claim 1, characterized in that: The object information includes at least one of the following: The position information of the robotic arm, the speed information of the robotic arm, the force information of the robotic arm, the angle information of each joint of the robotic arm, and the image information of the virtual camera set on the robotic arm.

3. The low-cost simulation training method for robots according to claim 1, characterized in that: The robotic arm comprises a left robotic arm and a right robotic arm, wherein the left robotic arm and the right robotic arm are respectively provided with grippers, and the grippers are used to grasp a target object. The preset motion strategy of the configuration robotic arm in the simulation environment comprises: Preset the position that the left robotic arm needs to reach at each preset time point in the preset time point list; Preset the action state that the gripper of the left robot arm needs to perform at each preset time point in the preset time point list; Preset the position that the right robotic arm needs to reach at each preset time point in the preset time point list; Preset the action state that the gripper of the right robot arm needs to perform at each preset time point in the preset time point list.

4. The low-cost simulation training method for robots according to claim 3, characterized in that: The method further comprises: According to the positions that the left robotic arm needs to reach at each preset time point in the preset time point list, the grasping path of the left robotic arm is planned; The grasping path planning of the right robotic arm is performed according to the positions that the right robotic arm needs to reach at each preset time point in the preset time point list.

5. The low-cost simulation training method for robots according to claim 4, characterized in that: The controlling the robot arm to grasp the target object according to a preset motion strategy includes: The right robotic arm is controlled to grasp the target object. After the right robotic arm grasps the target object, the right robotic arm is controlled to hand over the target object to the left robotic arm, and the left robotic arm is controlled to place the target object at the target position.

6. The low-cost simulation training method for robots according to claim 4, characterized in that: The target object includes a first target object and a second target object, the first target object is a plug-in of the second target object, and the controlling robot arm to grasp the target object according to a preset motion strategy includes: The right robotic arm is controlled to grab the first target object, the left robotic arm is controlled to grab the second target object, and the right robotic arm is controlled to plug the first target object onto the second target object.

7. The low-cost simulation training method for robots according to claim 5, characterized in that: The method of controlling the right robotic arm to grasp the target object, and after the right robotic arm grasps the target object, controlling the right robotic arm to hand over the target object to the left robotic arm, and controlling the left robotic arm to place the target object at the target position includes the following processes: Control the right robotic arm to grasp the target object, and after the right robotic arm grasps the target object, control the right robotic arm to reach a first preset meeting position at a first preset time point; Control the right robotic arm to rotate at the first preset meeting position to a preset handover posture; Controlling the left robotic arm to reach a second preset meeting position at a second preset time point; Control the gripper of the left robotic arm to close; Control the gripper of the right robotic arm to open; Complete the handover of the target object; Control the left robotic arm to place the target object at the target location.

8. The low-cost simulation training method for robots according to claim 1, characterized in that: The initializing the state of the robotic arm includes: Control the robot arm to move to a preset initial position, and control the angles of each joint of the robot arm to move to a preset initial state.

9. A low-cost robot simulation training system, characterized in that: include: An acquisition module, used for acquiring object information of the robot arm at each preset time point corresponding to the preset time point list during the process of grasping the target object, and recording the object information acquired during each grasping process as a training data set; and used for acquiring a first preset number of training data sets; A control module is used to initialize the state of the robot arm, generate a target object at a random position in a preset area of ​​the simulation environment, and configure a preset motion strategy of the robot arm in the simulation environment; Used to divide the acquired training data set into a training data set and a verification data set according to a first preset ratio; and use the training data set to train the automatic training model of the robotic arm; It is used to optimize the trained robotic arm automatic training model using the verification data set, optimize the model parameters, and output the optimized model parameter file.

10. An electronic device, characterized in that: include: Memory; and a processor, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the low-cost robot simulation training method according to any one of claims 1 to 8 is implemented.

Images