Robot simulation training method

Abstract

The invention relates to an artificial intelligence training method, in particular to a robot simulation training method which comprises the steps that initial environment information is collected, and a visual simulator constructs a live-action three-dimensional model according to the initial environment information; the dynamic simulator generates a live-action twinborn digital model, and binds the live-action twinborn digital model with the live-action three-dimensional model; the dynamic simulator transmits the motion data to the visual simulator in real time, so that the visual simulator renders the motion of the live-action three-dimensional model in real time, and training is executed according to image data rendered by the live-action three-dimensional model under different poses and dynamic data generated by the corresponding live-action twin digital model; when the visual simulator renders the motion of the live-action three-dimensional model, different light simulation vectors are changed. By means of the method, the adjustable light rays can be fused in rendering of the live-action three-dimensional model, on-site light ray change interference is simulated, and a large number of data sets needed by model training are efficiently generated at low cost.

Description

Technical Field

[0001] The invention relates to an artificial intelligence training method, and in particular to a robot simulation training method. Background Art

[0002] The description in this section merely provides background information related to the present disclosure and does not constitute prior art.

[0003] The robot control model (for example, the ACT (Action Chunking With Transformer) algorithm) is an end-to-end deep learning model that can perform real-time robot control based on visual information and joint encoder information as observation values, that is, it can map from images to actions, allowing the robot to learn and imitate specific actions from visual input. End-to-end deep learning models usually need to be trained with the help of simulation environments (such as Mujoco and Gazebo) and their performance is verified in simulation. However, the model trained in simulation is difficult to apply directly to actual scenarios and requires additional data collection and training. In other words, the migration efficiency from simulation to reality is low, and the data collection and training costs of real scenarios are high, which limits the practical application and promotion of robot intelligent models.

[0004] At the same time, further, during training, due to the complexity of the robot's working scenarios, the diversity of data processing dimensions, the robot's structural errors, and the interference caused by changes in on-site lighting conditions, the robot's intelligent model needs to rely on a larger amount of data samples for training, and data collection based on real scenarios requires more manpower / material resources, resulting in expensive training costs.

[0005] It should be noted that the above introduction to the technical background is only for the convenience of providing a clear and complete description of the technical solutions of the present invention and for the convenience of understanding by those skilled in the art. It cannot be considered that the above technical solutions are well known to those skilled in the art simply because these solutions are described in the background technology section of the present invention. Summary of the invention

[0006] The purpose of the present invention is to provide a robot simulation training method, which can generate a large amount of data sets required for model training efficiently and at low cost by integrating adjustable light into the rendering of a real-life three-dimensional model to simulate the interference of on-site light changes.

[0007] In order to achieve the above object, the present invention discloses the following robot simulation training method, which comprises:

[0008] Providing a visual simulator and collecting initial environmental information, wherein the visual simulator constructs a real-scene three-dimensional model according to the initial environmental information;

[0009] Providing a dynamics simulator, the dynamics simulator generates a real-scene twin digital model, and binding the real-scene twin digital model with the real-scene three-dimensional model;

[0010] The dynamics simulator simulates different movements of the simulated robot in the real-scene twin digital model, and in the process of simulating the movement of the simulated robot, the dynamics simulator transmits the movement data to the visual simulator in real time, so that the visual simulator renders the movement of the real-scene three-dimensional model in real time, and performs training according to the image data rendered by the real-scene three-dimensional model in different postures and the corresponding dynamics data generated by the real-scene twin digital model;

[0011] When the visual simulator renders the motion of the real-life three-dimensional model according to the initial environmental information and / or the motion data provided by the dynamic simulator, different light simulation vectors are used in the rendering to make multiple simulated environmental lighting scenes in the image data rendered by the real-life three-dimensional model in different postures for training, wherein the simulated environmental information in any of the simulated environmental lighting scenes is not completely the same as the initial environmental information.

[0012] As a further description of the above technical solution, in the step of "providing a visual simulator and collecting initial environmental information, and the visual simulator constructing a real-scene three-dimensional model based on the initial environmental information", the real-scene three-dimensional model includes a Gaussian model based on a point cloud; in the step of "changing different light simulation vectors in rendering", a light field surrounding the Gaussian model is set, and after some points in the point cloud of the Gaussian model each receive an equal amount of preset light collection, each maps out a corresponding RGB point in the image to be rendered of the real-scene three-dimensional model, so as to form a complete frame of the image rendered by the real-scene three-dimensional model.

[0013] As a further description of the above technical solution, in one action of the simulated robot, the light field is set to be unique.

[0014] As a further description of the above technical solution, in one action of the simulated robot, the light field is set to have multiple different light rays.

[0015] As a further description of the above technical solution, the light field is configured to be a hemispherical shape surrounding the top of the Gaussian model in the horizontal direction.

[0016] As a further description of the above technical solution, it also includes a controller, which is used to make judgments and decisions based on the image output generated by the visual simulator and input the decision results to the dynamic simulator; the dynamic simulator transmits motion data to the visual simulator in real time, and the visual simulator synchronously renders the motion of the real-life three-dimensional model.

[0017] As a further description of the above technical solution, the controller is configured as at least one of a keyboard, a VR, a mouse and a simulation object.

[0018] As a further description of the above technical solution, in the step of "performing training according to the image data rendered by the real-life three-dimensional model in different postures and the corresponding dynamic data generated by the real-life twin digital model", a convolutional neural network structure is used to extract features of the image data rendered by the real-life three-dimensional model in different postures, and the features are aligned with the dynamic data generated by the real-life twin digital model.

[0019] As a further description of the above technical solution, based on the alignment relationship between the features and the dynamic data generated by the real-life twin digital model, the operating movements of the simulated robot are predicted.

[0020] The present invention also discloses a robot simulation control device, wherein the device includes a memory and a processor, the memory stores at least one program instruction, and the processor implements any of the methods described in the above-mentioned robot simulation training by loading and executing the at least one program instruction.

[0021] By means of the above technical solution, the beneficial effects of the present invention are as follows:

[0022] The robot simulation training method of the present invention collects the initial environmental information in the real scene, and then uses the visual simulator to establish a real-scene three-dimensional model, and binds the real-scene twin digital model that matches it, simulates the movement of the robot in the dynamic simulator, and synchronously feeds back to the visual simulator, timely generates the real-scene three-dimensional model of the movement, and then re-inputs the real-scene three-dimensional model of the movement into the dynamic simulator to generate the real-scene twin digital model. In the above process, only the initial environmental information in the real scene needs to be collected once to achieve long-term training, and there is no need to manually collect data in large quantities, which is more cost-effective and efficient.

[0023] At the same time, in this application, the fusion of adjustable light is introduced in the rendering link of the visual simulator to generate the real-scene 3D model, simulating the interference of on-site light changes, and efficiently and low-costly generating a large amount of data sets required for model training.

[0024] To further understand the features and technical contents of the present invention, please refer to the following detailed description and drawings of the present invention. However, the drawings provided are only for reference and description and are not intended to limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0026] Figure 1 It is a schematic diagram of the overall process of a robot simulation training method provided in an embodiment of this specification;

[0027] Figure 2 It is a schematic diagram of the rendering process of a robot simulation training method provided in an embodiment of this specification. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0029] The following is an explanation of the embodiments of the present invention through specific embodiments. Those skilled in the art can understand the advantages and effects of the present invention from the contents disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and the details in this specification can also be modified and changed in various ways based on different viewpoints and applications without departing from the concept of the present invention. In addition, the drawings of the present invention are only simple schematic illustrations and are not depicted according to actual sizes. It is stated in advance. The following embodiments will further explain the relevant technical contents of the present invention in detail, but the disclosed contents are not intended to limit the scope of protection of the present invention.

[0030] It should be understood that, although the terms "first", "second", "third", etc. may be used herein to describe various components or signals, these components or signals should not be limited by these terms. These terms are mainly used to distinguish one component from another component, or one signal from another signal. In addition, the term "or" used herein may include any one or more combinations of the associated listed items depending on the actual situation.

[0031] See also Figure 1 , is a robot simulation training method of this embodiment, wherein the robot simulation training method includes:

[0032] Providing a visual simulator and collecting initial environmental information, the visual simulator constructs a real-scene three-dimensional model based on the initial environmental information;

[0033] Provide a dynamics simulator, which generates a real-scene twin digital model and binds the real-scene twin digital model to the real-scene 3D model;

[0034] The dynamics simulator is used to simulate different movements of the simulation robot in the real-scene twin digital model. In the process of simulating the movement of the simulation robot, the dynamics simulator transmits the movement data to the visual simulator in real time, so that the visual simulator renders the movement of the real-scene three-dimensional model in real time, and performs training according to the image data rendered by the real-scene three-dimensional model in different postures and the dynamics data generated by the corresponding real-scene twin digital model.

[0035] Among them, when the visual simulator renders the movement of the real-life three-dimensional model according to the initial environment information and / or the motion data provided by the dynamic simulator, different light simulation vectors are used in the rendering to make multiple simulated environment lighting scenes in the image data rendered by the real-life three-dimensional model in different postures used for training, wherein the simulated environment information in any of the simulated environment lighting scenes is not completely the same as the initial environment information.

[0036] In the above process, first, the operator only needs to obtain the initial environment information, and generate the initial real-scene three-dimensional model in the visual simulator for the initial environment information. Based on the initial real-scene three-dimensional model, the size information and coordinate information in the real-scene three-dimensional model are bound to the real-scene twin mathematical model. Then, the real-scene twin digital model is manipulated with the help of the dynamics simulator, the movement of the simulation robot is simulated in the real-scene twin digital model, and the real-scene twin digital model undergoes real-time interference changes, and the data of the changed real-scene twin digital model is packaged and resent to the visual simulator, so that the visual simulator generates a moving real-scene three-dimensional model for the changed data, and the image data of the moving real-scene three-dimensional model is reprocessed, bound to the corresponding real-scene twin digital model, and continued to be processed in the dynamics simulator, so as to cycle. That is to say, in the above process, the operator only needs to obtain the initial environment information once, generate an initial real-scene three-dimensional model for a field task, and then use the visual simulator and the dynamics simulator to process the dynamic graphic data and dynamics data in a loop, respectively, so that a large amount of training data can be obtained on the basis of avoiding a large amount of on-site manual data collection.

[0037] At the same time, in the present invention, during the process of rendering and generating any real-life three-dimensional model by a visual simulator, light processing is also introduced in the rendering link. By changing different light simulation vectors, the real-life three-dimensional model has various light source environments that are different from the actual initial environmental information, and the above-mentioned real-life three-dimensional model with a variety of different light source environments is bound to the real-life twin digital model and then processed. That is to say, the training of data in this embodiment takes into account the changes in the actual on-site light environment, so that the problem of light interference is taken into account and solved during training. Therefore, the ability of the simulated robot to cope with different light scenes when predicting actions is solved (light scenes include but are not limited to cloudy days, nighttime, under lights, etc.).

[0038] For the specific method of rendering the first modeling in this embodiment, please refer to Figure 2 First, the pose estimation and multi-view 3D reconstruction are performed on the original image (which can be a captured video) obtained from the initial environment information to obtain a dense point cloud, and the original image is segmented to obtain the mask of the target object. Then, these three are input into the visual simulator. Through back propagation, based on Gaussian splashing and BRDF rendering formula, the rendering parameters are continuously optimized to achieve 1:1 reconstruction of the Gaussian model corresponding to the real 3D model. The light scene of the real 3D model is changed through different light simulation vectors, and finally, combined with the pose of the simulated robot, a rendering image with a light scene different from the actual initial environment information is rendered through the GPU, and then put into the dynamic simulator for continued use.

[0039] For the specific method of rendering after the first modeling in this embodiment, please refer to Figure 2 The dynamic data generated by the dynamic simulator after simulating the specified simulated robot action based on the previous frame of the real-scene 3D model is processed to obtain a new Gaussian model, and a new image is rendered through the GPU in combination with the current posture of the simulated robot, and then put into the dynamic simulator for continued use, and this cycle is repeated.

[0040] In the above process of collecting initial environmental information, the initial environmental information includes the posture information of the simulated robot at the application site, the object information of the simulated robot in contact with the object, and the scene information around the simulated robot. The posture information includes the posture information of the body and limbs of the simulated robot at the application site. In actual implementation, when acquiring the initial environmental information, a calibration plate can be placed in the application scene, and the initial environmental information including the calibration plate can be collected. Subsequently, the translation and rotation matrix of the current camera coordinate system can be obtained by identifying the calibration mark of the calibration plate.

[0041] In the above process, binding the real-scene twin digital model with the real-scene three-dimensional model refers to binding the size information and coordinate information in the real-scene three-dimensional model with the real-scene twin mathematical model.

[0042] In actual implementation, this step may include:

[0043] First, obtain the first coordinate set of the initial environment information in the calibration plate coordinate system;

[0044] The initial environment information may include multiple sets of pictures, which include calibration plates. This step can obtain the coordinate sets A{(x1, y1, z1), (x2, y2, z2), (x3, y3, z3)....} of the multiple sets of cameras in the calibration plate coordinate system.

[0045] Second, when constructing the Gaussian model, a second coordinate set including initial environmental information of the calibration plate is obtained;

[0046] When constructing the Gaussian model, the camera pose of each frame is calculated through SFM, and the image coordinates including the calibration plate are obtained from each image to form the second coordinate set S{(x1, y1, z1), (x2, y2, z2), (x3, y3, z3)....}.

[0047] Third, converting the coordinates in the second coordinate set into a third coordinate set that is consistent with the coordinate size in the first coordinate set;

[0048] (1) Calculate the first distance between adjacent coordinate points in the first coordinate system;

[0049] Calculate the distance La{l1,l2,l3....} between adjacent points in the coordinate set A.

[0050] (2) calculating a second distance between adjacent coordinate points in the second coordinate system;

[0051] The distance between adjacent points in the set S is Ls{l1,l2,l3....}.

[0052] (3) calculating a distance ratio coefficient k according to the first distance and the second distance;

[0053] Calculate the distance ratios {k1, k2, k3, ...} of the corresponding distances in the set A and the distances in the set S, and calculate the proportionality coefficient k of each distance ratio.

[0054] (4) Scale each coordinate point in the second coordinate set by k times to obtain the third coordinate set.

[0055] That is, after scaling the coordinates in the set S by k times, we get Sk, which is the third coordinate set.

[0056] Fourth, obtain the rotation and translation matrices of the first coordinate set and the third coordinate set;

[0057] (1) Calculate the centroid coordinates of the third coordinate set and the first coordinate set respectively;

[0058] The centroid coordinates are the average values ​​of the coordinates on each coordinate axis.

[0059] (2) Obtain the coordinates of the third coordinate set and the decentralized first coordinate set;

[0060] For the first coordinate set and the third coordinate set, the coordinates after decentralization are calculated respectively. That is, for the first coordinate set A and the third coordinate set Sk, the coordinates after decentralization are

[0061] (3) Calculate the covariance matrix based on the decentralized third coordinate set and the first coordinate set;

[0062] The covariance matrix H is:

[0063] (4) Obtain the rotation matrix based on the covariance matrix;

[0064] Perform singular value decomposition on the covariance matrix to obtain the rotation matrix R, H = U · ∑ · V T . Where U and V are orthogonal matrices, Σ is a diagonal matrix. The rotation matrix R is composed of U and V.

[0065] (5) Obtain the translation vector according to the center of mass of the first coordinate set, the center of mass of the third coordinate set, and the rotation matrix.

[0066] The translation vector t is:

[0067] The rotation matrix and translation vector are combined to form the rotation and translation matrix Mk.

[0068] After that, the Gaussian model is bound to the mathematical model. The mathematical model is the model in the dynamics simulator. The mathematical model sets the parameters such as the mass and friction coefficient of the model. After the binding is completed, the Gaussian model in the visual simulator and the mathematical model in the dynamics simulator are unified in the coordinate system.

[0069] The above can build a complete simulation environment. Later, the simulation robot can be simulated in the simulation environment through the interaction of the visual simulator and the dynamic simulator.

[0070] Further, in the step of "providing a visual simulator and collecting initial environmental information, and the visual simulator constructing a real-scene three-dimensional model according to the initial environmental information", the real-scene three-dimensional model includes a Gaussian model based on a point cloud; in the step of "changing different light simulation vectors in rendering", a light field surrounding the Gaussian model is set, and after some points in the point cloud of the Gaussian model each receive an equal amount of preset light collection, each maps a corresponding RGB point in the image to be rendered of the real-scene three-dimensional model, so as to form a complete frame of the image rendered by the real-scene three-dimensional model. Specifically, the setting of the light field can be adjusted accordingly according to the size of the actual scene. In rendering, the light field emits a number of light rays to a specified area in the scene. After the light rays collide with the physical model in the scene, they respectively collide with each individual point in the scene that has a volume and can receive light collisions. After receiving the specified number of collisions, the collection of the preset collection amount is completed. For example, in one embodiment, each corresponding individual point can complete the collection after receiving 300 light collisions, and has sufficient light collection quality. In this embodiment, each collected point is mapped into a 360-degree panoramic image, and RGB points with corresponding colors are generated, so that a complete image scene is formed by the aggregation of all points.

[0071] In this embodiment, the light field is set to be a hemispherical shape that surrounds the top of the Gaussian model in the horizontal direction. This is mainly because in real robot usage scenarios, the main light changes are from the top area, not the bottom surface, so it can save calculations and be close to real physical scenarios. Of course, in some other embodiments, the specific coverage range of the light field can be set accordingly as needed without departing from the processing logic of this embodiment.

[0072] In the above embodiment, the light field can be set to be unique, that is, the light emitted by the light field is always consistent, which is required for simulating a stable light usage scenario training. In another embodiment, the light field can be set to have multiple different light sources, that is, in the rendering of multiple frames, the light of the scene of the real scene 3D model can change dynamically, which is required for simulating a scene training where the light may change.

[0073] Furthermore, it also includes a controller, which is used to make judgments and decisions based on the image output generated by the visual simulator and input the decision results to the dynamic simulator; the dynamic simulator transmits motion data to the visual simulator in real time, and the visual simulator synchronously renders the motion of the real-life three-dimensional model. Specifically, the controller can be at least one of a keyboard, VR, a mouse, and a simulator. The operator can generate dynamic instructions for the simulation robot in the real-life twin digital model by operating the controller to complete the corresponding actions, such as controlling the simulation robot to perform displacement, grasping, turning, etc., so that the change information of the camera posture, simulation robot posture and object posture in the real-life twin digital model is generated, and the above-mentioned dynamic data is further sent to the visual simulator, so that the visual simulator renders the current real-life three-dimensional model in real time.

[0074] Furthermore, in the step of "executing training according to the image data rendered by the real-life 3D model in different postures and the corresponding dynamic data generated by the real-life twin digital model", a convolutional neural network structure is used to extract the features of the image data rendered by the real-life 3D model in different postures, and align the features with the dynamic data generated by the real-life twin digital model. Specifically, based on the alignment relationship between the features and the dynamic data generated by the real-life twin digital model, the operation of the simulated robot is predicted. Among them, the convolutional neural network is used to extract feature information such as edges, textures, and shapes from the image data. After extracting the image features, the fully connected network is used to align the dimensions with the joint features in the dynamic data, and the two are projected into the same feature space for subsequent processing, so as to achieve effective fusion of the two types of information, which is used to more accurately predict the motion actions or other tasks of the simulated robot.

[0075] Specifically, after the alignment operation, the global relationship between the image features and the joint features after alignment is extracted based on the self-attention mechanism, and the translation / rotation invariance of the convolutional network (avoiding the change of position or angle caused by the displacement of the simulated robot) and the sequence feature extraction ability of the self-attention mechanism are combined to further predict the operation of the simulated robot. Among them, the self-attention mechanism is a neural network mechanism used to capture the global relationship between data, and will assign weights according to the similarity between features to capture the correlation between image features and joint features.

[0076] The contents disclosed above are only preferred feasible embodiments of the present invention, and are not intended to limit the scope of the present invention. Therefore, all equivalent technical changes made using the contents of the present invention's specification and drawings are included in the scope of the present invention.

[0077] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other. Each embodiment focuses on the differences from other embodiments.

[0078] Although the present application has been described through embodiments, those skilled in the art will appreciate that there are many modifications and changes to the present application without departing from the spirit of the present application. It is intended that the attached embodiments include these modifications and changes without departing from the present application.

Claims

1. A robot simulation training method, characterized in that: The robot simulation training method comprises: Providing a visual simulator and collecting initial environmental information, wherein the visual simulator constructs a real-scene three-dimensional model according to the initial environmental information; Providing a dynamics simulator, the dynamics simulator generates a real-scene twin digital model, and binding the real-scene twin digital model with the real-scene three-dimensional model; The dynamics simulator simulates different movements of the simulated robot in the real-scene twin digital model, and in the process of simulating the movement of the simulated robot, the dynamics simulator transmits the movement data to the visual simulator in real time, so that the visual simulator renders the movement of the real-scene three-dimensional model in real time, and performs training according to the image data rendered by the real-scene three-dimensional model in different postures and the corresponding dynamics data generated by the real-scene twin digital model; When the visual simulator renders the motion of the real-life three-dimensional model according to the initial environmental information and / or the motion data provided by the dynamic simulator, different light simulation vectors are used in the rendering to make multiple simulated environmental lighting scenes in the image data rendered by the real-life three-dimensional model in different postures for training, wherein the simulated environmental information in any of the simulated environmental lighting scenes is not completely the same as the initial environmental information.

2. The robot simulation training method according to claim 1, characterized in that: In the step of "providing a visual simulator and collecting initial environmental information, wherein the visual simulator constructs a real-scene three-dimensional model based on the initial environmental information", the real-scene three-dimensional model includes a Gaussian model based on a point cloud; in the step of "changing different light simulation vectors in rendering", a light field surrounding the Gaussian model is set, and after some points in the point cloud of the Gaussian model each receive an equal amount of preset light collection, each maps out a corresponding RGB point in the image to be rendered of the real-scene three-dimensional model, so as to form a complete frame of the image rendered by the real-scene three-dimensional model.

3. The robot simulation training method according to claim 2, characterized in that: During an action of the simulated robot, the light field is set to be unique.

4. The robot simulation training method according to claim 2, characterized in that: In one action of the simulated robot, the light field is configured to have a plurality of different light rays.

5. The robot simulation training method according to claim 2, characterized in that: The light field is configured to be a hemispherical shape surrounding the top of the Gaussian model in a horizontal direction.

6. The robot simulation training method according to claim 1, characterized in that: It also includes a controller, which is used to make judgments and decisions based on the image output generated by the visual simulator and input the decision results to the dynamic simulator; the dynamic simulator transmits motion data to the visual simulator in real time, and the visual simulator synchronously renders the motion of the real-scene three-dimensional model.

7. The robot simulation training method according to claim 6, characterized in that: The controller is configured as at least one of a keyboard, a VR, a mouse, and an analog object.

8. The robot simulation training method according to claim 1, characterized in that: In the step of "performing training according to the image data rendered by the real-life three-dimensional model in different postures and the corresponding dynamic data generated by the real-life twin digital model", a convolutional neural network structure is used to extract features of the image data rendered by the real-life three-dimensional model in different postures, and the features are aligned with the dynamic data generated by the real-life twin digital model.

9. The robot simulation training method according to claim 8, characterized in that: Based on the alignment relationship between the features and the dynamic data generated by the real-scene twin digital model, the operating movements of the simulated robot are predicted.

10. A robot simulation control device, characterized in that: The device includes a memory and a processor, wherein at least one program instruction is stored in the memory, and the processor implements the method according to any one of claims 1 to 9 by loading and executing the at least one program instruction.

Images