Gaussian prediction model training method, robot reinforcement learning method and electronic device

Abstract

The present application provides a Gaussian prediction model training method, a robot reinforcement learning method, and an electronic device, relating to the field of artificial intelligence technology. The method includes: obtaining a first sample image of a first preset perspective and a second sample image of a first target perspective; using a pre-trained Gaussian prediction model to perform parameter prediction based on the first sample image to obtain first predicted Gaussian parameters and second predicted Gaussian parameters; generating a first predicted image based on the second predicted Gaussian parameters; training the pre-trained Gaussian prediction model based on the first training loss of the first predicted image and the second sample image and the second training loss of the first predicted Gaussian parameters and the second predicted Gaussian parameters to obtain a target Gaussian prediction model, which is used to be deployed on the robot, and reinforcement learning is performed by generating scene Gaussian parameters of the work scene to determine the robot's actions. The present application can generate high-quality environment representation parameters and improve the effect of reinforcement learning.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and more specifically, to a Gaussian prediction model training method, a robot reinforcement learning method, and an electronic device. Background Art

[0002] In reinforcement learning (RL), especially vision-based RL tasks, excellent environment representation parameters are crucial to learning effectiveness, and their quality directly affects the completion of learning tasks.

[0003] Existing vision-based RL typically uses explicit or implicit methods to represent the environment, such as images, points, voxels, and neural radiation fields. However, these representations have many defects. They are either unable to describe complex local geometric structures, or difficult to generalize to unseen scenes. Some also require foreground masks, resulting in the inability to provide excellent environment representation parameters for RL, reducing the effectiveness of reinforcement learning. Summary of the Invention

[0004] The purpose of this application is to address the deficiencies in the above-mentioned prior art and provide a Gaussian prediction model training method, a robot reinforcement learning method and an electronic device so as to generate high-quality environment representation parameters and improve the effect of reinforcement learning.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of the present application are as follows:

[0006] In a first aspect, an embodiment of the present application provides a Gaussian prediction model training method, the method comprising:

[0007] Acquire a first sample image of a training three-dimensional scene at a first preset perspective and a second sample image at a first target perspective;

[0008] Performing parameter prediction using a pre-trained Gaussian prediction model based on the first sample image to obtain first predicted Gaussian parameters and second predicted Gaussian parameters, where the second predicted Gaussian parameters are attribute parameters obtained by denoising the first predicted Gaussian parameters;

[0009] generating a first predicted image of the first target perspective according to the second predicted Gaussian parameters;

[0010] Calculating a first training loss based on the first predicted image and the second sample image;

[0011] Calculating a second training loss based on the first predicted Gaussian parameter and the second predicted Gaussian parameter;

[0012] According to the first training loss and the second training loss, the pre-trained Gaussian prediction model is trained to obtain a target Gaussian prediction model. The target Gaussian prediction model is used to be deployed on a robot so that the robot generates scene Gaussian parameters of the working scene according to the target Gaussian prediction model, and reinforcement learning is performed according to the scene Gaussian parameters to determine the action of the robot.

[0013] Optionally, the pre-trained Gaussian prediction model includes: a pre-trained parameter prediction model and an initial parameter optimization model; and performing parameter prediction using the pre-trained Gaussian prediction model according to the first sample image to obtain first predicted Gaussian parameters and second predicted Gaussian parameters includes:

[0014] Performing parameter prediction using the pre-trained parameter prediction model according to the first sample image to obtain the first predicted Gaussian parameters;

[0015] The first predicted Gaussian parameters are optimized using the initial parameter optimization model to obtain the second predicted Gaussian parameters.

[0016] Optionally, performing parameter prediction using the pre-trained parameter prediction model according to the first sample image to obtain the first predicted Gaussian parameter includes:

[0017] Obtaining a sample feature map of the first sample image using a depth prediction module in the pre-trained parameter prediction model, and performing depth estimation on the sample feature map to obtain a depth image;

[0018] Using the attribute prediction module in the pre-trained parameter prediction model to perform parameter prediction on the first sample image, the sample feature map, and the fused feature map corresponding to the depth image to obtain a predicted rotation parameter, a predicted scaling parameter, and a predicted transparency parameter;

[0019] Determining predicted three-dimensional position parameters based on the depth image;

[0020] determining a predicted color parameter according to the color value of the first sample image;

[0021] The first predicted Gaussian parameters include: the predicted three-dimensional position parameter, the predicted rotation parameter, the predicted scaling parameter, the predicted color parameter and the predicted transparency parameter.

[0022] Optionally, the using a depth prediction module in the pre-trained parameter prediction model to perform depth estimation on the sample feature map to obtain a depth image includes:

[0023] If the first sample image is a single sample image, a single image depth predictor is used to perform depth prediction on a sample feature map of the single sample image to obtain the depth image;

[0024] If the first sample image is two sample images, a disparity prediction network is used to perform disparity prediction on the sample feature maps of the two sample images to obtain the depth image.

[0025] Optionally, acquiring a first sample image of the training three-dimensional scene at a first preset perspective and a second sample image at a first target perspective includes:

[0026] Determine the second sample image from the sample image set of the training three-dimensional scene;

[0027] According to the first target viewing angle of the second sample image, two sample images with adjacent viewing angles closest to the first target viewing angle are determined from the sample image set as the first sample images.

[0028] Optionally, before performing parameter prediction using a pre-trained Gaussian prediction model according to the first sample image to obtain first predicted Gaussian parameters and second predicted Gaussian parameters, the method further includes:

[0029] Acquire a third sample image of a second preset viewing angle and a fourth sample image of a second target viewing angle;

[0030] Perform parameter prediction using an initial Gaussian prediction model according to the fourth sample image to obtain third predicted Gaussian parameters;

[0031] generating a second predicted image of the second target perspective according to the third predicted Gaussian parameters;

[0032] Calculating a third training loss based on the second predicted image and the fourth sample image;

[0033] The depth prediction module in the initial Gaussian prediction model is pre-trained according to the third training loss to obtain the pre-trained Gaussian prediction model.

[0034] Optionally, the training the pre-trained Gaussian prediction model according to the first training loss and the second training loss to obtain a target Gaussian prediction model includes:

[0035] According to the first training loss and the second training loss, other modules in the pre-trained Gaussian prediction model except the depth prediction module are trained to obtain a target Gaussian prediction model.

[0036] Optionally, the initial parameter optimization model is a graph neural network model, and optimizing the first predicted Gaussian parameters using the initial parameter optimization model to obtain the second predicted Gaussian parameters includes:

[0037] Generate a first predicted Gaussian point cloud according to the first predicted Gaussian parameters;

[0038] Optimizing the first predicted Gaussian point cloud using the graph neural network model to obtain a second predicted Gaussian point cloud, where the second predicted Gaussian point cloud is a point cloud represented by the second predicted Gaussian parameters;

[0039] Generating a first predicted image of the first target perspective according to the second predicted Gaussian parameters includes:

[0040] Rendering is performed according to the second predicted Gaussian point cloud to generate the first predicted image.

[0041] In a second aspect, an embodiment of the present application further provides a robot reinforcement learning method based on a Gaussian prediction model, the method comprising:

[0042] Obtaining a scene image of a preset working scene;

[0043] Using a pre-trained target Gaussian prediction model to perform parameter prediction on the scene image to obtain scene Gaussian parameters of the preset working scene, the target Gaussian prediction model being pre-trained using the Gaussian prediction model training method according to any one of the first aspects;

[0044] A reinforcement learning network is used to predict the robot's movements based on the Gaussian parameters of the scene to control the robot to operate in the preset working scene.

[0045] In a third aspect, an embodiment of the present application provides a Gaussian prediction model training device, the device comprising:

[0046] A first image acquisition module is used to acquire a first sample image of a training three-dimensional scene at a first preset perspective and a second sample image at a first target perspective;

[0047] a first parameter prediction module, configured to perform parameter prediction based on the first sample image using a pre-trained Gaussian prediction model to obtain first predicted Gaussian parameters and second predicted Gaussian parameters, where the second predicted Gaussian parameters are attribute parameters obtained by denoising the first predicted Gaussian parameters;

[0048] an image generation module, configured to generate a first predicted image of the first target perspective according to the second predicted Gaussian parameters;

[0049] a loss calculation module, configured to calculate a first training loss based on the first predicted image and the second sample image;

[0050] The loss calculation module is further configured to calculate a second training loss based on the first predicted Gaussian parameter and the second predicted Gaussian parameter;

[0051] A model training module is used to train the pre-trained Gaussian prediction model according to the first training loss and the second training loss to obtain a target Gaussian prediction model. The target Gaussian prediction model is used to be deployed on a robot so that the robot generates scene Gaussian parameters of the working scene according to the target Gaussian prediction model, performs reinforcement learning according to the scene Gaussian parameters, and determines the action of the robot.

[0052] Optionally, the pre-trained Gaussian prediction model includes: a pre-trained parameter prediction model and an initial parameter optimization model; the first parameter prediction module includes:

[0053] a first parameter prediction unit, configured to perform parameter prediction based on the first sample image using the pre-trained parameter prediction model to obtain the first predicted Gaussian parameter;

[0054] The second parameter prediction unit is configured to optimize the first predicted Gaussian parameters using the initial parameter optimization model to obtain the second predicted Gaussian parameters.

[0055] Optionally, the first parameter prediction unit is specifically used to use the depth prediction module in the pre-trained parameter prediction model to obtain the sample feature map of the first sample image, and perform depth estimation on the sample feature map to obtain a depth image; use the attribute prediction module in the pre-trained parameter prediction model to perform parameter prediction on the first sample image, the sample feature map and the fusion feature map corresponding to the depth image to obtain predicted rotation parameters, predicted scaling parameters and predicted transparency parameters; determine the predicted three-dimensional position parameters based on the depth image; determine the predicted color parameters based on the color value of the first sample image; wherein, the first predicted Gaussian parameters include: the predicted three-dimensional position parameters, the predicted rotation parameters, the predicted scaling parameters, the predicted color parameters and the predicted transparency parameters.

[0056] Optionally, the first parameter prediction unit is specifically used to, if the first sample image is a single sample image, use a single image depth predictor to perform depth prediction on the sample feature map of the single sample image to obtain the depth image; if the first sample image is two sample images, use a disparity prediction network to perform disparity prediction on the sample feature maps of the two sample images to obtain the depth image.

[0057] Optionally, the first image acquisition module is specifically used to determine the second sample image from the sample image set of the training three-dimensional scene; based on the first target perspective of the second sample image, determine from the sample image set two sample images with adjacent perspectives closest to the first target perspective as the first sample image.

[0058] Optionally, the first image acquisition module is further configured to acquire a third sample image of a second preset viewing angle and a fourth sample image of a second target viewing angle;

[0059] The first parameter prediction module is further configured to perform parameter prediction based on the fourth sample image using an initial Gaussian prediction model to obtain third predicted Gaussian parameters;

[0060] The image generation module is further configured to generate a second predicted image of the second target perspective according to the third predicted Gaussian parameters;

[0061] The loss calculation module is further configured to calculate a third training loss based on the second predicted image and the fourth sample image;

[0062] The model training module is further used to pre-train the depth prediction module in the initial Gaussian prediction model according to the third training loss to obtain the pre-trained Gaussian prediction model.

[0063] Optionally, the model training module is specifically used to train other modules in the pre-trained Gaussian prediction model except the depth prediction module according to the first training loss and the second training loss to obtain a target Gaussian prediction model.

[0064] Optionally, the initial parameter optimization model is a graph neural network model, and the second parameter prediction unit is specifically used to generate a first predicted Gaussian point cloud based on the first predicted Gaussian parameters; the first predicted Gaussian point cloud is optimized using the graph neural network model to obtain a second predicted Gaussian point cloud, where the second predicted Gaussian point cloud is a point cloud represented by the second predicted Gaussian parameters;

[0065] The image generation module is specifically used to perform rendering according to the second predicted Gaussian point cloud to generate the first predicted image.

[0066] In a fourth aspect, an embodiment of the present application further provides a robot reinforcement learning device based on a Gaussian prediction model, the device comprising:

[0067] A second image acquisition module is used to acquire a scene image of a preset working scene;

[0068] a second parameter prediction module, configured to perform parameter prediction on the scene image using a pre-trained target Gaussian prediction model to obtain scene Gaussian parameters of the preset working scene, wherein the target Gaussian prediction model is pre-trained using the Gaussian prediction model training method according to any one of the first aspects;

[0069] The control module is used to predict the robot's movements using a reinforcement learning network based on the Gaussian parameters of the scene, so as to control the robot to operate in the preset working scene.

[0070] In the fifth aspect, an embodiment of the present application also provides an electronic device, comprising: a processor, a storage medium and a bus, wherein the storage medium stores program instructions executable by the processor. When the electronic device is running, the processor and the storage medium communicate through the bus, and the processor executes the program instructions to perform the steps of the Gaussian prediction model training method as described in any one of the first aspects, or the steps of the robot reinforcement learning method based on the Gaussian prediction model as described in the second aspect.

[0071] In a sixth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the Gaussian prediction model training method as described in any one of the first aspects, or the robot reinforcement learning method based on the Gaussian prediction model as described in the second aspect, is executed.

[0072] The beneficial effects of this application are:

[0073] The Gaussian prediction model training method, robot reinforcement learning method and electronic device provided in the present application train a Gaussian prediction model through sample images of a three-dimensional scene, so that the scene Gaussian parameters of the scene can be generated through the Gaussian prediction model, and the three-dimensional local geometric structure of the scene can be described in detail through the scene Gaussian parameters, which facilitates the robot to perform reinforcement learning based on the scene Gaussian parameters, improves the robot's reinforcement learning effect, and ensures the robot's operation accuracy; and based on the Gaussian prediction model, Gaussian parameter prediction can be provided for robots in different scenes, which is generalizable and has a wide range of application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0075] Figure 1 Schematic diagram of the process of the Gaussian prediction model training method provided in the embodiment of the present application Figure 1 ;

[0076] Figure 2 A model architecture diagram provided for an embodiment of the present application;

[0077] Figure 3 Schematic diagram of the process of the Gaussian prediction model training method provided in the embodiment of the present application Figure 2 ;

[0078] Figure 4 Schematic diagram of the process of the Gaussian prediction model training method provided in the embodiment of the present application Figure 3 ;

[0079] Figure 5 A schematic diagram of a process flow of a robot reinforcement learning method based on a Gaussian prediction model provided in an embodiment of the present application;

[0080] Figure 6 A schematic diagram of the structure of a Gaussian prediction model training device provided in an embodiment of the present application;

[0081] Figure 7 A schematic diagram of the structure of a robot reinforcement learning device based on a Gaussian prediction model provided in an embodiment of the present application;

[0082] Figure 8 A schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0083] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0084] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for protection, but merely represents selected embodiments of the present application. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments in the present application without making any creative efforts shall fall within the scope of protection of the present application.

[0085] In addition, the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0086] It should be noted that, in the absence of conflict, the features in the embodiments of this application can be combined with each other.

[0087] Figure 1 Schematic diagram of the process of the Gaussian prediction model training method provided in the embodiment of the present application Figure 1 ,like Figure 1 As shown, the method may include:

[0088] S101: Acquire a first sample image of a training three-dimensional scene at a first preset viewing angle and a second sample image at a first target viewing angle.

[0089] In this embodiment, when the robot is working in a three-dimensional scene, it needs to perform reinforcement learning based on the environmental representation parameters of the three-dimensional scene to determine the working actions. In order to provide the robot with accurate environmental representation parameters that can describe the three-dimensional scene, a Gaussian prediction model is planned to be trained.

[0090] Image acquisition is performed on the training three-dimensional scene to obtain sample images of the training three-dimensional scene at multiple perspectives to form a sample image set, and a first sample image of a first preset perspective and a second sample image of a first target perspective are obtained from the sample image set, wherein the first target perspective is the perspective of the predicted image to be generated, the second sample image of the first target perspective is used to supervise the training of the Gaussian prediction model, the first preset perspective is an adjacent perspective of the first target perspective, and the first sample image of the first preset perspective is used as a training sample of the Gaussian prediction model.

[0091] S102 , performing parameter prediction using a pre-trained Gaussian prediction model according to the first sample image to obtain first predicted Gaussian parameters and second predicted Gaussian parameters, where the second predicted Gaussian parameters are attribute parameters obtained by denoising the first predicted Gaussian parameters.

[0092] In this embodiment, the pre-trained Gaussian prediction model is a Gaussian prediction model that has been partially trained in advance. Some model parameters of the pre-trained Gaussian prediction model are frozen in this training, that is, some model parameters of the pre-trained Gaussian prediction model do not participate in the training during this training process. In this way, pre-training is performed before training, which can improve the training efficiency of the model.

[0093] Part of the model parameters in the Gaussian prediction model that are common to different three-dimensional scenes may be pre-trained.

[0094] The first sample image is input into the pre-trained Gaussian prediction model. The pre-trained Gaussian prediction model generates first predicted Gaussian parameters by performing feature extraction, encoding, decoding and other processes on the first sample image. The first predicted Gaussian parameters are used to represent the Gaussian attribute parameters of the training three-dimensional scene.

[0095] After generating the first predicted Gaussian parameters, since this scheme needs to generate the first predicted image of the first target perspective based on the first sample image of the first preset perspective, in order to reduce the high-frequency noise caused by the differences between adjacent perspectives, the pre-trained Gaussian prediction model also optimizes the first predicted Gaussian parameters, filters the noise in the first predicted Gaussian parameters, and obtains smoothed second predicted Gaussian parameters.

[0096] S103 : Generate a first predicted image of a first target perspective according to the second predicted Gaussian parameters.

[0097] In this embodiment, the second predicted Gaussian parameters are environment representation parameters of the training three-dimensional scene, and rasterization rendering is performed based on the second predicted Gaussian parameters to obtain a first predicted image of the first target perspective.

[0098] Specifically, a first predicted image of the first target perspective may be generated according to the second predicted Gaussian parameters and the camera intrinsic parameters and posture of the second sample image of the first target perspective.

[0099] S104: Calculate a first training loss based on the first predicted image and the second sample image.

[0100] In this embodiment, the first training loss is calculated according to the difference between each pixel point between the first predicted image and the second sample image.

[0101] For example, the first training loss may be calculated based on the square of the Euclidean distance between the first predicted image and the second sample image.

[0102] S105. Calculate a second training loss according to the first predicted Gaussian parameter and the second predicted Gaussian parameter.

[0103] In this embodiment, in order to supervise the optimization process of the pre-trained Gaussian prediction model generating the second predicted Gaussian parameters based on the first predicted Gaussian parameters, it is also necessary to calculate the second training loss between the first predicted Gaussian parameters and the second predicted Gaussian parameters.

[0104] For example, the second training loss may be calculated based on the square of the Euclidean distance between the first predicted Gaussian parameter and the second predicted Gaussian parameter.

[0105] S106. The pre-trained Gaussian prediction model is trained according to the first training loss and the second training loss to obtain a target Gaussian prediction model. The target Gaussian prediction model is used to be deployed on the robot so that the humanoid robot generates scene Gaussian parameters of the working scene according to the target Gaussian prediction model, and reinforcement learning is performed according to the scene Gaussian parameters to determine the actions of the humanoid robot.

[0106] In this embodiment, a weighted calculation is performed on the first training loss and the second training loss, and model parameters other than the frozen parameters in the pre-trained prediction model are optimized based on the weighted training loss. Multiple rounds of training are performed using the above steps, and training is terminated when the number of training rounds reaches a preset number or when the weighted training loss converges, thereby obtaining a target Gaussian prediction model.

[0107] After the training is completed, the target Gaussian prediction model is deployed in the robot. By obtaining images of the robot's working scene from any perspective, the target Gaussian prediction model is used to generate the scene Gaussian parameters of the robot's working scene, that is, the environment representation parameters, so that the reinforcement learning network in the robot can perform reinforcement learning based on the scene Gaussian parameters to determine the robot's next action.

[0108] The Gaussian prediction model training method provided in the above embodiment trains the Gaussian prediction model through sample images of the three-dimensional scene, so that the scene Gaussian parameters of the scene can be generated through the Gaussian prediction model, and the three-dimensional local geometric structure of the scene is described in detail through the scene Gaussian parameters, which facilitates the robot to perform reinforcement learning based on the scene Gaussian parameters, improves the robot's reinforcement learning effect, and ensures the robot's operation accuracy; and based on the Gaussian prediction model, Gaussian parameter prediction can be provided for robots in different scenes, which is generalizable and has a wide range of application scenarios.

[0109] In one possible implementation, Figure 2 The model architecture diagram provided in the embodiment of this application is as follows: Figure 2 As shown, the pre-trained Gaussian prediction model may include: a pre-trained parameter prediction model and an initial parameter optimization model. The above S102 uses the pre-trained Gaussian prediction model to perform parameter prediction based on the first sample image to obtain the first predicted Gaussian parameter and the second predicted Gaussian parameter, which may include:

[0110] A pre-trained parameter prediction model is used to perform parameter prediction based on the first sample image to obtain first predicted Gaussian parameters; and an initial parameter optimization model is used to perform optimization based on the first predicted Gaussian parameters to obtain second predicted Gaussian parameters.

[0111] In this embodiment, the pre-trained Gaussian prediction model is divided into two parts: a pre-trained parameter prediction model and an initial parameter optimization model, wherein some model parameters in the pre-trained parameter prediction model are frozen after pre-training.

[0112] The first sample image is input into the pre-trained parameter prediction model, and the pre-trained parameter prediction model generates a first predicted Gaussian parameter by performing feature extraction, encoding, decoding and other processes on the first sample image.

[0113] The first preset Gaussian parameter is input into the initial parameter optimization model, and the initial parameter optimization model encodes and decodes the first predicted Gaussian parameter to reduce the high-frequency noise caused by the difference between adjacent views in the first predicted Gaussian parameter, and obtains the smoothed second predicted Gaussian parameter.

[0114] The Gaussian prediction model training method provided in the above embodiment optimizes the predicted Gaussian parameters output by the parameter prediction model based on the parameter optimization model, reduces the high-frequency noise caused by the differences between adjacent views, and improves the smoothness of the predicted Gaussian parameters, so that the robot's reinforcement learning network can control the robot's movements based on precise scene Gaussian parameters, thereby improving the robot's operation accuracy.

[0115] In one possible implementation, Figure 3 Schematic diagram of the process of the Gaussian prediction model training method provided in the embodiment of the present application Figure 2 ,like Figure 3 As shown, the process of performing parameter prediction using the pre-trained parameter prediction model according to the first sample image to obtain the first predicted Gaussian parameter may include:

[0116] S201: Use a depth prediction module in a pre-trained parameter prediction model to obtain a sample feature map of a first sample image, and perform depth estimation on the sample feature map to obtain a depth image.

[0117] S202: Use the attribute prediction module in the pre-trained parameter prediction model to perform parameter prediction on the first sample image, the sample feature map, and the fused feature map corresponding to the depth image to obtain predicted rotation parameters, predicted scaling parameters, and predicted transparency parameters.

[0118] S203: Determine predicted three-dimensional position parameters according to the depth image.

[0119] S204: Determine predicted color parameters according to the color value of the first sample image.

[0120] The first predicted Gaussian parameters include: predicted three-dimensional position parameters, predicted rotation parameters, predicted scaling parameters, predicted color parameters and predicted transparency parameters.

[0121] In this embodiment, if Figure 2 As shown, the pre-trained parameter prediction model may include: a depth prediction module, a feature fusion module and an attribute prediction module, wherein the depth prediction model includes an encoder and a decoder, the encoder is a feature extractor, and the feature extractor is used to extract features of the first sample image to obtain a sample feature map of the first sample image.

[0122] The sample feature map of the first sample image is input into the decoder of the depth prediction module for depth estimation, and the depth value of each pixel in the first sample image is determined to obtain a depth image of the first sample image.

[0123] The first sample image, the sample feature map of the first sample image and the depth image are connected through a feature fusion module to obtain a fused feature map, wherein the feature fusion module can use an encoder to generate a fused feature map with full image resolution.

[0124] The fused feature map is input into the attribute prediction module, which includes multiple parameter decoders, namely a rotation parameter decoder, a scaling parameter decoder and a transparency parameter decoder. The fused feature map is decoded by the multiple parameter decoders to obtain predicted rotation parameters, predicted scaling parameters and predicted transparency parameters.

[0125] In some embodiments, multiple parameter decoders are each composed of a full convolutional layer with a 1*1 convolution kernel.

[0126] In addition to the predicted rotation parameters, the predicted scaling parameters, and the predicted transparency parameters, the first predicted Gaussian parameters also include predicted three-dimensional position parameters and predicted color parameters, wherein the predicted three-dimensional position parameters can be determined based on the depth image and the camera posture of the first sample image taken from the first target perspective, and the predicted color parameters can be directly determined based on the RGB color values of each pixel in the first sample image.

[0127] The predicted rotation parameters and predicted scaling parameters represent the rotation information and scaling information of the Gaussian points corresponding to the object structure in the training three-dimensional scene. The predicted transparency parameters are used to represent the occlusion of light by the Gaussian points corresponding to the object structure in the training three-dimensional scene. The predicted three-dimensional position parameters are used to represent the position information of the Gaussian points corresponding to the object structure in the training three-dimensional scene. The predicted color parameters are used to represent the color information of the Gaussian points corresponding to the object structure in the training three-dimensional scene.

[0128] It should be noted that the above-mentioned prediction parameters are all pixel-by-pixel prediction parameters, and the predicted parameters are actually parameter maps.

[0129] In one possible implementation, the process of performing depth estimation on the sample feature map using the depth prediction module in the pre-trained parameter prediction model in S202 to obtain the depth image may include:

[0130] If the first sample image is a single sample image, a single image depth predictor is used to perform depth prediction on the sample feature map of the single sample image to obtain a depth image; if the first sample image is two sample images, a disparity prediction network is used to perform disparity prediction on the sample feature maps of the two sample images to obtain a depth image.

[0131] In this embodiment, Gaussian parameters can be predicted using a single sample image or using multi-view images. If the first sample image is a single sample image, the depth prediction module is a pre-trained single-image depth predictor, which is used to perform depth prediction based on the sample feature map of the single sample image to obtain a depth image. If the first sample image is two sample images, the depth prediction module is a pre-trained disparity prediction network, which uses two identical feature extractors to extract features from the two sample images to obtain two sample feature maps. The disparity prediction network is used to predict the disparity of each pixel in the two sample feature maps, and the depth value of each pixel is determined based on the maximum disparity value in the disparity data set of each pixel to obtain a depth image.

[0132] In some embodiments, if the first sample image is two sample images, the process of obtaining the first sample image of the training 3D scene at the first preset perspective and the second sample image at the first target perspective in S101 may include:

[0133] A second sample image is determined from a sample image set of a training three-dimensional scene; and according to a first target perspective of the second sample image, two sample images of adjacent perspectives closest to the first target perspective are determined from the sample image set as first sample images.

[0134] In this embodiment, the sample image set for training three-dimensional scenes includes sample images of multiple perspectives. First, the second sample image of the first target perspective is determined from the sample image set. Then, based on the distance between the first target perspective and the perspectives of other sample images, the two sample images of the two adjacent perspectives closest to the first target perspective are determined as the first sample images.

[0135] Furthermore, if the first sample image is two sample images, the process of determining the predicted color parameter according to the color value of the first sample image may be determining the predicted color parameter according to the average value of the color values of the two sample images.

[0136] In one possible implementation, Figure 4 Schematic diagram of the process of the Gaussian prediction model training method provided in the embodiment of the present application Figure 3 ,like Figure 4 As shown, before performing parameter prediction using a pre-trained Gaussian prediction model according to the first sample image in S102 to obtain first predicted Gaussian parameters and second predicted Gaussian parameters, the method may further include:

[0137] S301: Acquire a third sample image of a second preset viewing angle and a fourth sample image of a second target viewing angle.

[0138] S302 : Perform parameter prediction using an initial Gaussian prediction model according to the fourth sample image to obtain third predicted Gaussian parameters.

[0139] S303: Generate a second predicted image of a second target perspective according to the third predicted Gaussian parameters.

[0140] S304: Calculate a third training loss based on the second predicted image and the fourth sample image.

[0141] S305 . Pre-train the depth prediction module in the initial Gaussian prediction model according to the third training loss to obtain a pre-trained Gaussian prediction model.

[0142] In this embodiment, the depth prediction module in the initial Gaussian prediction model needs to be pre-trained. The pre-training steps are similar to the training steps in S101-S106 above, except that the training loss in the pre-training steps is only the third training loss between the fourth sample image and the second predicted image, and does not include the training loss between the two predicted Gaussian parameters. The third predicted Gaussian parameter is the optimized Gaussian parameter.

[0143] In some embodiments, an L1 loss function may be used to calculate the third training loss of the second predicted image and the fourth sample image.

[0144] The Gaussian prediction model training method provided in the above embodiment can improve the training efficiency of the model by pre-training the depth prediction module in the Gaussian prediction model.

[0145] In one possible implementation, the initial parameter optimization model is a graph neural network model. The process of optimizing the first predicted Gaussian parameters using the initial parameter optimization model to obtain the second predicted Gaussian parameters may include:

[0146] A first predicted Gaussian point cloud is generated according to the first predicted Gaussian parameters; the first predicted Gaussian point cloud is optimized using a graph neural network model to obtain a second predicted Gaussian point cloud, where the second predicted Gaussian point cloud is a point cloud represented by the second predicted Gaussian parameters.

[0147] The process of generating the first predicted image of the first target perspective according to the second predicted Gaussian parameters in S103 may include:

[0148] Rendering is performed according to the second predicted Gaussian point cloud to generate a first predicted image.

[0149] In this embodiment, the initial position of the point cloud is determined according to the predicted three-dimensional position parameters in the first predicted Gaussian parameters, the orientation of the initial position of the point cloud is adjusted according to the predicted rotation parameters, the scale of the point cloud is adjusted according to the predicted scaling parameters to obtain the position of the point cloud, and the color and opacity of the power source are defined according to the predicted color parameters and the predicted transparency parameters to obtain the first predicted Gaussian point cloud.

[0150] The first predicted Gaussian point cloud is input into the graph neural network model for optimization, the positions of each point cloud in the first predicted Gaussian point cloud are smoothed, and the high-frequency noise caused by the differences between different views is removed to obtain the second predicted Gaussian point cloud. Rasterization rendering is performed based on the second predicted Gaussian point cloud to generate the first predicted image.

[0151] In some embodiments, the graph neural network model may include an encoder and a decoder, the encoder may include multiple multi-layer perceptrons for outputting a multi-dimensional node feature map, and the number of layers of the decoder is the same as that of the encoder.

[0152] The Gaussian prediction model training method provided in the above embodiment uses a graph neural network model to optimize the first predicted Gaussian parameters, and utilizes the smoothing characteristics of the graph neural network model to effectively reduce the high-frequency noise caused by view differences, thereby improving the accuracy of the second predicted Gaussian parameters.

[0153] In addition to the Gaussian prediction model training method provided in the above embodiment, the embodiment of the present application also provides a robot reinforcement learning method based on the Gaussian prediction model. Figure 5 A flowchart of a robot reinforcement learning method based on a Gaussian prediction model provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, the method may include:

[0154] S401: Acquire a scene image of a preset working scene.

[0155] S402: Use a pre-trained target Gaussian prediction model to perform parameter prediction on the scene image to obtain scene Gaussian parameters of a preset working scene.

[0156] S403: Using a reinforcement learning network to predict the robot's movements based on the Gaussian parameters of the scene, so as to control the robot to operate in a preset working scene.

[0157] In this embodiment, the target Gaussian prediction model is deployed on the robot. The robot is also provided with a camera, through which the scene image of the working scene of the robot can be obtained. The target Gaussian prediction model can generate scene Gaussian parameters of the working scene based on the scene image, and generate a scene Gaussian point cloud based on the scene Gaussian parameters. The scene Gaussian point cloud is used to predict the robot's motion through the reinforcement learning network deployed on the robot to control the robot to perform operations in the working scene.

[0158] The robot reinforcement learning method based on the Gaussian prediction model provided in the above embodiment uses the Gaussian prediction model trained by Yuxuan to generate scene Gaussian parameters, so that the reinforcement learning network controls the robot to perform actions based on the scene Gaussian parameters. The high-quality scene Gaussian parameters improve the accuracy of the reinforcement learning network in action prediction.

[0159] Based on the above method embodiments, an embodiment of the present application provides a Gaussian prediction model training device. Figure 6 A schematic diagram of the structure of the Gaussian prediction model training device provided in the embodiment of the present application is shown in FIG. Figure 6 As shown, the device may include:

[0160] The first image acquisition module 501 is configured to acquire a first sample image of a training three-dimensional scene at a first preset viewing angle and a second sample image at a first target viewing angle;

[0161] A first parameter prediction module 502 is configured to perform parameter prediction based on the first sample image using a pre-trained Gaussian prediction model to obtain first predicted Gaussian parameters and second predicted Gaussian parameters, where the second predicted Gaussian parameters are attribute parameters obtained by denoising the first predicted Gaussian parameters;

[0162] An image generation module 503 is configured to generate a first predicted image of a first target perspective according to the second predicted Gaussian parameters;

[0163] A loss calculation module 504 is configured to calculate a first training loss based on the first predicted image and the second sample image;

[0164] The loss calculation module 504 is further configured to calculate a second training loss based on the first predicted Gaussian parameter and the second predicted Gaussian parameter;

[0165] The model training module 505 is used to train the pre-trained Gaussian prediction model according to the first training loss and the second training loss to obtain a target Gaussian prediction model. The target Gaussian prediction model is used to be deployed on the robot so that the robot generates scene Gaussian parameters of the working scene according to the target Gaussian prediction model, performs reinforcement learning according to the scene Gaussian parameters, and determines the robot's actions.

[0166] Optionally, the pre-trained Gaussian prediction model includes: a pre-trained parameter prediction model and an initial parameter optimization model; the first parameter prediction module 502 includes:

[0167] A first parameter prediction unit, configured to perform parameter prediction based on the first sample image using a pre-trained parameter prediction model to obtain a first predicted Gaussian parameter;

[0168] The second parameter prediction unit is used to optimize the first predicted Gaussian parameters using an initial parameter optimization model to obtain second predicted Gaussian parameters.

[0169] Optionally, the first parameter prediction unit is specifically used to use the depth prediction module in the pre-trained parameter prediction model to obtain the sample feature map of the first sample image, and perform depth estimation on the sample feature map to obtain a depth image; use the attribute prediction module in the pre-trained parameter prediction model to perform parameter prediction on the fusion feature map corresponding to the first sample image, the sample feature map and the depth image to obtain predicted rotation parameters, predicted scaling parameters and predicted transparency parameters; determine the predicted three-dimensional position parameters based on the depth image; determine the predicted color parameters based on the color value of the first sample image; wherein the first predicted Gaussian parameters include: predicted three-dimensional position parameters, predicted rotation parameters, predicted scaling parameters, predicted color parameters and predicted transparency parameters.

[0170] Optionally, the first parameter prediction unit is specifically used to, if the first sample image is a single sample image, use a single image depth predictor to perform depth prediction on the sample feature map of the single sample image to obtain a depth image; if the first sample image is two sample images, use a disparity prediction network to perform disparity prediction on the sample feature maps of the two sample images to obtain a depth image.

[0171] Optionally, the first image acquisition module 501 is specifically used to determine a second sample image from a sample image set of a training three-dimensional scene; based on the first target perspective of the second sample image, two sample images of adjacent perspectives closest to the first target perspective are determined from the sample image set as first sample images.

[0172] Optionally, the first image acquisition module 501 is further configured to acquire a third sample image of a second preset viewing angle and a fourth sample image of a second target viewing angle;

[0173] The first parameter prediction module 502 is further configured to perform parameter prediction based on the fourth sample image using the initial Gaussian prediction model to obtain third predicted Gaussian parameters;

[0174] The image generation module 503 is further configured to generate a second predicted image of a second target perspective according to the third predicted Gaussian parameter;

[0175] The loss calculation module 504 is further configured to calculate a third training loss based on the second predicted image and the fourth sample image;

[0176] The model training module 505 is further used to pre-train the depth prediction module in the initial Gaussian prediction model according to the third training loss to obtain a pre-trained Gaussian prediction model.

[0177] Optionally, the model training module 505 is specifically used to train other modules in the pre-trained Gaussian prediction model except the depth prediction module according to the first training loss and the second training loss to obtain a target Gaussian prediction model.

[0178] Optionally, the initial parameter optimization model is a graph neural network model, and the second parameter prediction unit is specifically used to generate a first predicted Gaussian point cloud based on the first predicted Gaussian parameters; the first predicted Gaussian point cloud is optimized using the graph neural network model to obtain a second predicted Gaussian point cloud, where the second predicted Gaussian point cloud is a point cloud represented by the second predicted Gaussian parameters;

[0179] The image generation module 503 is specifically configured to perform rendering according to the second predicted Gaussian point cloud to generate a first predicted image.

[0180] An embodiment of the present application also provides a robot reinforcement learning device based on a Gaussian prediction model. Figure 7 A schematic diagram of the structure of a robot reinforcement learning device based on a Gaussian prediction model provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, the device may include:

[0181] The second image acquisition module 601 is used to acquire a scene image of a preset working scene;

[0182] A second parameter prediction module 602 is configured to perform parameter prediction on the scene image using a pre-trained target Gaussian prediction model to obtain scene Gaussian parameters of a preset working scene, wherein the target Gaussian prediction model is pre-trained using the Gaussian prediction model training method according to any one of the first aspects;

[0183] The control module 603 is used to predict the robot's movements using a reinforcement learning network based on the scene Gaussian parameters, so as to control the robot to operate in a preset working scene.

[0184] The above-mentioned device is used to execute the method provided in the above-mentioned embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0185] The above modules can be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), one or more microprocessors, or one or more field programmable gate arrays (FPGAs). For example, when a module is implemented by scheduling program code through a processing element, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0186] Figure 8 This is a schematic diagram of an electronic device provided in an embodiment of the present application. The electronic device 700 may include: a processor 701, a storage medium 702, and a bus. The storage medium 702 stores program instructions executable by the processor 701. When the electronic device 700 is running, the processor 701 and the storage medium 702 communicate via the bus, and the processor 701 executes the program instructions to perform the above-mentioned method embodiment. The specific implementation methods and technical effects are similar and will not be repeated here.

[0187] Optionally, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is run by a processor, the above method embodiment is executed.

[0188] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0189] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0190] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0191] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) or a processor (English: processor) to perform some steps of the methods of each embodiment of the present application. The aforementioned storage medium includes: USB flash drives, mobile hard drives, read-only memory (English: Read-Only Memory, abbreviated: ROM), random access memory (English: Random Access Memory, abbreviated: RAM), magnetic disks or optical disks, and other media that can store program code.

[0192] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited to them. Any changes or substitutions that can be easily conceived by any person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A Gaussian prediction model training method, characterized in that: The method comprises: Acquire a first sample image of a training 3D scene at a first preset perspective and a second sample image at a first target perspective, where the first target perspective is the perspective of a predicted image to be generated, and the first preset perspective is an adjacent perspective to the first target perspective; Performing parameter prediction using a pre-trained Gaussian prediction model based on the first sample image to obtain first predicted Gaussian parameters and second predicted Gaussian parameters, where the second predicted Gaussian parameters are attribute parameters obtained by denoising the first predicted Gaussian parameters; generating a first predicted image of the first target perspective according to the second predicted Gaussian parameters; Calculating a first training loss based on the first predicted image and the second sample image; Calculating a second training loss based on the first predicted Gaussian parameter and the second predicted Gaussian parameter; According to the first training loss and the second training loss, the pre-trained Gaussian prediction model is trained to obtain a target Gaussian prediction model. The target Gaussian prediction model is used to be deployed on a robot so that the robot generates scene Gaussian parameters of the working scene according to the target Gaussian prediction model, and reinforcement learning is performed according to the scene Gaussian parameters to determine the action of the robot.

2. The method according to claim 1, wherein The pre-trained Gaussian prediction model includes: a pre-trained parameter prediction model and an initial parameter optimization model; the parameter prediction is performed using the pre-trained Gaussian prediction model according to the first sample image to obtain the first predicted Gaussian parameter and the second predicted Gaussian parameter, including: Performing parameter prediction using the pre-trained parameter prediction model according to the first sample image to obtain the first predicted Gaussian parameters; The first predicted Gaussian parameters are optimized using the initial parameter optimization model to obtain the second predicted Gaussian parameters.

3. The method according to claim 2, wherein The step of performing parameter prediction using the pre-trained parameter prediction model according to the first sample image to obtain the first predicted Gaussian parameter includes: Obtaining a sample feature map of the first sample image using a depth prediction module in the pre-trained parameter prediction model, and performing depth estimation on the sample feature map to obtain a depth image; Using the attribute prediction module in the pre-trained parameter prediction model to perform parameter prediction on the first sample image, the sample feature map, and the fused feature map corresponding to the depth image to obtain a predicted rotation parameter, a predicted scaling parameter, and a predicted transparency parameter; Determining predicted three-dimensional position parameters based on the depth image; determining a predicted color parameter according to the color value of the first sample image; The first predicted Gaussian parameters include: the predicted three-dimensional position parameter, the predicted rotation parameter, the predicted scaling parameter, the predicted color parameter and the predicted transparency parameter.

4. The method according to claim 3, wherein The step of using the depth prediction module in the pre-trained parameter prediction model to perform depth estimation on the sample feature map to obtain a depth image includes: If the first sample image is a single sample image, a single image depth predictor is used to perform depth prediction on a sample feature map of the single sample image to obtain the depth image; If the first sample image is two sample images, a disparity prediction network is used to perform disparity prediction on the sample feature maps of the two sample images to obtain the depth image.

5. The method according to claim 4, wherein The step of acquiring a first sample image of a training three-dimensional scene at a first preset viewing angle and a second sample image at a first target viewing angle includes: Determine the second sample image from the sample image set of the training three-dimensional scene; According to the first target viewing angle of the second sample image, two sample images with adjacent viewing angles closest to the first target viewing angle are determined from the sample image set as the first sample images.

6. The method according to claim 3, wherein Before performing parameter prediction using a pre-trained Gaussian prediction model according to the first sample image to obtain first predicted Gaussian parameters and second predicted Gaussian parameters, the method further includes: Acquire a third sample image of a second preset viewing angle and a fourth sample image of a second target viewing angle; Perform parameter prediction using an initial Gaussian prediction model according to the fourth sample image to obtain third predicted Gaussian parameters; generating a second predicted image of the second target perspective according to the third predicted Gaussian parameters; Calculating a third training loss based on the second predicted image and the fourth sample image; The depth prediction module in the initial Gaussian prediction model is pre-trained according to the third training loss to obtain the pre-trained Gaussian prediction model.

7. The method according to claim 6, wherein The step of training the pre-trained Gaussian prediction model according to the first training loss and the second training loss to obtain a target Gaussian prediction model includes: According to the first training loss and the second training loss, other modules in the pre-trained Gaussian prediction model except the depth prediction module are trained to obtain a target Gaussian prediction model.

8. The method according to claim 2, wherein The initial parameter optimization model is a graph neural network model, and the optimization is performed using the initial parameter optimization model according to the first predicted Gaussian parameter to obtain the second predicted Gaussian parameter, including: Generate a first predicted Gaussian point cloud according to the first predicted Gaussian parameters; Optimizing the first predicted Gaussian point cloud using the graph neural network model to obtain a second predicted Gaussian point cloud, where the second predicted Gaussian point cloud is a point cloud represented by the second predicted Gaussian parameters; Generating a first predicted image of the first target perspective according to the second predicted Gaussian parameters includes: Rendering is performed according to the second predicted Gaussian point cloud to generate the first predicted image.

9. A robot reinforcement learning method based on a Gaussian prediction model, characterized in that: The method comprises: Obtaining a scene image of a preset working scene; Using a pre-trained target Gaussian prediction model to perform parameter prediction on the scene image to obtain scene Gaussian parameters of the preset working scene, the target Gaussian prediction model being pre-trained using the Gaussian prediction model training method according to any one of claims 1 to 8; A reinforcement learning network is used to predict the robot's movements based on the Gaussian parameters of the scene to control the robot to operate in the preset working scene.

10. An electronic device, characterized in that: include: A processor, a storage medium, and a bus, wherein the storage medium stores program instructions executable by the processor. When the electronic device is running, the processor and the storage medium communicate via the bus, and the processor executes the program instructions to perform the Gaussian prediction model training method according to any one of claims 1 to 8, or the robot reinforcement learning method based on the Gaussian prediction model according to claim 9.

Images