Differential simulation method and device based on three-dimensional Gaussian sputtering and electronic equipment

Abstract

The invention provides a differentiable simulation method and device based on three-dimensional Gaussian sputtering and electronic equipment, a three-dimensional Gaussian model is constructed based on image data and attitude data of a working area of a robot, and based on the three-dimensional Gaussian model, the internal motion state of the robot and the action output by a strategy network, the differentiable simulation of the robot is realized. And performing three-dimensional Gaussian sputtering differential rendering to generate a high-fidelity image, and training a strategy network based on the high-fidelity image and the internal motion state of the robot so as to finely adjust the three-dimensional Gaussian model based on the high-fidelity image and the real visual observation image of the robot in the process that the robot moves based on the output action of the strategy network. By the adoption of the method, strategy optimization can be more efficient, the cost and time of simulation environment construction are reduced, and therefore the applicability of the intelligent robot in the complex dynamic environment is further improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot motion simulation, and in particular to a differentiable simulation method and device based on three-dimensional Gaussian sputtering and an electronic device. BACKGROUND

[0002] In robot motion control, strategy training using a simulation environment is a mainstream method. However, the core challenge of this method is the "simulation-to-reality (Sim2Real) gap".

[0003] The method of strategy training using a simulation environment has the following characteristics: (1) the image realism (such as lighting, texture, sensor noise) of a traditional simulation environment (such as MuJoCo, IsaacGym) has a significant difference from the real world. This low-fidelity visual input makes it difficult for a strategy trained in simulation to generalize to real-world camera images, that is, the visual Sim2Real gap. Moreover, the cost of manually modeling, adjusting textures, and lighting is extremely high, and it is difficult to quickly reproduce complex real scenes; (2) neural rendering technology (such as NeRF) can reconstruct a high-fidelity, photo-level scene from multi-view images. However, the rendering speed is slow, and the representation based on voxels or implicit functions makes the calculation cost of gradient penetration through the rendering layer for backpropagation (i.e., differentiable rendering) extremely high, making it difficult to integrate into a reinforcement learning training loop that requires high throughput and real-time feedback; (3) domain randomization (DR) is a brute-force method, which is inefficient and cannot guarantee that the parameter space of randomization can completely cover the complexity of the real world; (4) differentiable simulation (Differentiable Simulation) technology realizes differentiability through a physics engine, allowing gradient flow through the physical model. However, the physics engine mainly focuses on dynamics and is usually unable to capture and construct a photo-level differentiable visual environment from real scenes.

[0004] It can be seen that the prior art has the following deficiencies in constructing a high-fidelity Sim2Real environment: (1) Although neural rendering technology can collect data from the real world and generate a high-fidelity visual environment, the complex voxel or implicit representation leads to slow rendering speed and complex gradient calculation, which cannot meet the demand for high throughput and high frequency sampling in embodied intelligent reinforcement learning training, which makes high fidelity and training efficiency a pair of contradictions that are difficult to reconcile; (2) In the traditional reinforcement learning (RL) training process, the visual renderer is usually regarded as a black box, and the loss gradient of the policy network can only propagate to the visual input layer, and cannot penetrate the rendering layer to directly connect the visual input and the policy weight. This mechanism defect hinders the end-to-end differentiable optimization of vision-based control, and limits the sensitivity and efficiency of policy learning to visual features; (3) The existing RL training process lacks an efficient Real-to-Sim-to-Real closed-loop mechanism. When the policy is deployed in the real world and fails, it cannot efficiently propagate the real-world feedback (such as visual differences and light changes) back to the simulation environment for online correction and policy iteration, resulting in a one-way and passive reduction of the Sim2Real gap; (4) Whether through manual modeling, complex texture adjustment, domain randomization, or through neural rendering, a large amount of real data collection, complex manual intervention, and a long reconstruction period are involved. The existing technology lacks the ability to quickly and cost-effectively convert a real scene into a high-fidelity, trainable digital twin model that can be used for RL training. SUMMARY

[0005] Therefore, the present application aims to provide a differentiable simulation method and device based on three-dimensional Gaussian sputtering and electronic equipment to alleviate the above-mentioned problems in the prior art.

[0006] In a first aspect, the embodiments of the present application provide a differentiable simulation method based on three-dimensional Gaussian sputtering, comprising: constructing a three-dimensional Gaussian model based on image data and pose data of a robot working area; performing three-dimensional Gaussian sputtering differentiable rendering based on the three-dimensional Gaussian model, the internal motion state of the robot, and the action output by the policy network to generate a high-fidelity image; training the policy network based on the high-fidelity image and the internal motion state of the robot; wherein the gradient of the first loss of the policy network is backpropagated during the training process of the policy network; and fine-tuning the three-dimensional Gaussian model based on the high-fidelity image and the real visual observation image of the robot during the motion of the robot based on the action output by the policy network.

[0007] In a second aspect, the embodiments of the present application also provide a differentiable simulation device based on three-dimensional Gaussian sputtering, comprising: a construction module configured to construct a three-dimensional Gaussian model based on image data and pose data of a robot working area; a rendering module configured to perform three-dimensional Gaussian sputtering differentiable rendering based on the three-dimensional Gaussian model, internal motion state of the robot, and actions output by a policy network to generate high-fidelity images; a training module configured to train the policy network based on the high-fidelity images and the internal motion state of the robot; wherein gradients of a first loss of the policy network are back-propagated in a training process of the policy network; and a fine-tuning module configured to fine-tune the three-dimensional Gaussian model based on the high-fidelity images and real visual observation images of the robot in a process in which the robot moves based on the actions output by the policy network.

[0008] In a third aspect, the embodiments of the present application also provide an electronic device, comprising a processor and a memory, wherein the memory stores computer executable instructions capable of being executed by the processor, and the processor executes the computer executable instructions to implement the three-dimensional Gaussian sputtering based differentiable simulation method of the first aspect.

[0009] The three-dimensional Gaussian sputtering based differentiable simulation method, device, and electronic device provided by the embodiments of the present application first construct a three-dimensional Gaussian model based on image data and pose data of a robot working area, then perform three-dimensional Gaussian sputtering differentiable rendering based on the three-dimensional Gaussian model, internal motion state of the robot, and actions output by a policy network to generate high-fidelity images, and then train the policy network based on the high-fidelity images and the internal motion state of the robot, and fine-tune the three-dimensional Gaussian model based on the high-fidelity images and real visual observation images of the robot in a process in which the robot moves based on the actions output by the policy network. By using the above technology, a three-dimensional Gaussian model can be constructed by using image data and pose data of a robot working area, a high-fidelity simulation environment can be quickly and low-costly reconstructed by three-dimensional Gaussian sputtering differentiable rendering in combination with internal motion state of the robot and actions output by a policy network, and then the policy network can be trained and the three-dimensional Gaussian model can be fine-tuned in combination with the internal motion state of the robot and real visual observation images, so that end-to-end optimization from visual feedback to parameters of the policy network is realized, the policy optimization becomes more efficient, the cost and time for constructing a simulation environment are reduced, and the applicability of embodied intelligence in a complex dynamic environment is further improved.

[0010] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent from the description, or can be learned by practice of the present application. The objects and other advantages of the present application will be realized and achieved by means of the structures particularly pointed out in the description and the claims.

[0011] In order to make the above objectives, characteristics and advantages of the present application more obvious and easy to understand, the following preferred embodiments are specifically described below, and the accompanying drawings are described in detail as follows. BRIEF DESCRIPTION OF DRAWINGS

[0012] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings without creative labor based on these drawings.

[0013] Figure 1 A flowchart of a differentiable simulation method based on three-dimensional Gaussian sputtering in an embodiment of the present application; Figure 2 A flowchart of scene reconstruction and object physical property registration in an embodiment of the present application; Figure 3 A schematic diagram of differentiable rendering and policy gradient backpropagation principle in an embodiment of the present application; Figure 4 A Sim2Real online self-adaptation and model fine-tuning closed-loop flowchart in an embodiment of the present application; Figure 5 A whole architecture diagram of a differentiable simulation system in an embodiment of the present application; Figure 6 A structure schematic diagram of a differentiable simulation device based on three-dimensional Gaussian sputtering in an embodiment of the present application; Figure 7 A structure schematic diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0014] In order to make the objectives, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions of the present application will be described clearly and completely in combination with embodiments. Obviously, the described embodiments are some embodiments of the present application, but not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0015] At present, in robot motion control, the method of using simulation environment for policy training is difficult to reconcile the contradiction between high-fidelity visual simulation and reinforcement learning training efficiency, and lacks an efficient visual feedback differentiable mechanism, and there are problems of high cost and low efficiency in digital twin model construction.

[0016] Based on this, the differentiable simulation method, device and electronic device based on three-dimensional Gaussian sputtering provided by the embodiments of the present application can alleviate the above problems existing in the prior art.

[0017] For the convenience of understanding the present embodiment, first, a kind of differentiable simulation method based on three-dimensional Gaussian sputtering disclosed in the embodiment of the application is introduced in detail, see Figure 1 As shown in the figure, the method can include the following steps: Step S102, based on the image data and pose data of the robot working area, three-dimensional Gaussian model is constructed.

[0018] Exemplarily, RGB-D camera array can be arranged to collect the image data and pose data of the robot working area, and then three-dimensional Gaussian model is constructed based on the image data and pose data collected by the RGB-D camera array.

[0019] Step S104, based on three-dimensional Gaussian model, internal motion state of robot and action output by policy network, three-dimensional Gaussian sputtering differentiable rendering is carried out to generate high-fidelity image.

[0020] Step S106, based on high-fidelity image and internal motion state of robot, policy network is trained.

[0021] Wherein, the gradient of the first loss of the policy network can be back propagated in the training process of the policy network.

[0022] Step S108, in the process that the robot moves based on the action output by the policy network, three-dimensional Gaussian model is fine-tuned based on high-fidelity image and real visual observation image of the robot.

[0023] Wherein, the real visual observation image can be collected in real time in the process that the robot moves.

[0024] The embodiment of the application provides a differentiable simulation method based on three-dimensional Gaussian splatting, which comprises the following steps: firstly, constructing a three-dimensional Gaussian model based on image data and pose data of a robot working area; secondly, performing three-dimensional Gaussian splatting differentiable rendering based on the three-dimensional Gaussian model, internal motion state of the robot and an action output by a policy network to generate a high-fidelity image; thirdly, training the policy network based on the high-fidelity image and the internal motion state of the robot; and fourthly, fine-tuning the three-dimensional Gaussian model based on the high-fidelity image and a real visual observation image of the robot during the motion of the robot based on the action output by the policy network. By using the operation mode, the three-dimensional Gaussian model can be constructed by using the image data and the pose data of the robot working area, the high-fidelity simulation environment can be quickly and low-costly reconstructed by combining the internal motion state of the robot and the action output by the policy network through three-dimensional Gaussian splatting differentiable rasterization rendering, and then the policy network training and the three-dimensional Gaussian model fine-tuning are performed by combining the internal motion state of the robot and the real visual observation image, so that the end-to-end optimization from the visual feedback to the policy network parameters is realized, the policy optimization becomes more efficient, the cost and the time of the simulation environment construction are reduced, and the applicability of embodied intelligence in a complex dynamic environment is further improved.

[0025] As a possible implementation, the step S104 (i.e., performing three-dimensional Gaussian splatting differentiable rendering based on the three-dimensional Gaussian model, the internal motion state of the robot and the action output by the policy network to generate a high-fidelity image) can comprise: dividing the three-dimensional Gaussian model into a first Gaussian point set representing a background and a second Gaussian point set representing an interactive object; creating an initial physical collision body at a geometric center of each second Gaussian point in the second Gaussian point set, and assigning a physical parameter to each initial physical collision body to obtain a corresponding physical collision body; and performing three-dimensional Gaussian splatting differentiable rendering based on the physical collision body, the three-dimensional Gaussian model, the internal motion state of the robot and the action output by the policy network to generate a high-fidelity image.

[0026] Exemplarily, referring to FIG. 1, Figure 2 the scene reconstruction and the object physical property registration can be performed in the following manner: collecting image data and pose data of a real robot working area by using a multi-view RGB-D camera array; reconstructing a three-dimensional Gaussian model containing position , covariance , color and opacity attributes by using the image data and the pose data of the robot working area through a three-dimensional Gaussian splatting (3D Gaussian Splatting, 3DGS) optimization algorithm; performing semantic segmentation on the three-dimensional Gaussian model , and dividing the three-dimensional Gaussian model into a background Gaussian point set (for rendering only) and an interactive object Gaussian point set ; will be registered into the physics engine, specifically in the geometry center of each interactive object Gaussian point to create a simplified physics collider (such as a bounding box or convex hull) and assign physical parameters (such as mass , friction coefficient , etc.) for the physics engine to recognize and process, so that the simplified physics collider assigned with physical parameters is the interactive digital twin model.

[0027] As a possible implementation, the step of performing three-dimensional Gaussian sputtering differentiable rendering to generate high-fidelity images based on the physical colliders, the three-dimensional Gaussian model, the internal motion state of the robot, and the action output by the policy network can include: performing rendering by a three-dimensional Gaussian sputtering differentiable rasterization algorithm to generate high-fidelity images based on the physical colliders, the three-dimensional Gaussian model, the internal motion state of the robot, and the action output by the policy network; wherein the three-dimensional Gaussian sputtering differentiable rasterization algorithm has a gradient backpropagation interface, which can be used to backpropagate the gradient of the first loss of the policy network during the training process of the policy network.

[0028] As a possible implementation, the step of performing rendering by a three-dimensional Gaussian sputtering differentiable rasterization algorithm to generate high-fidelity images based on the physical colliders, the three-dimensional Gaussian model, the internal motion state of the robot, and the action output by the policy network can include: Step A1, using the physical colliders, the internal motion state of the robot, and the action output by the policy network, the physics engine calculates the pose of the robot and the interactive object respectively; Step A2, based on the three-dimensional Gaussian model and the pose, rendering is performed by a three-dimensional Gaussian sputtering differentiable rasterization algorithm to generate high-fidelity images.

[0029] Exemplarily, the three-dimensional Gaussian model has color and opacity; based on this, the above-mentioned step A2 can calculate the color value of each pixel in the high-fidelity image by the three-dimensional Gaussian sputtering differentiable rasterization algorithm using the following formula:

[0030] wherein, is the pixel color value, is the color of the first Gaussian point on the three-dimensional Gaussian model, and are the opacity of the first Gaussian point and the second Gaussian point on the three-dimensional Gaussian model, respectively; The three-dimensional Gaussian model can also have position and covariance, and the above-mentioned step A2 can also update the position and covariance of the second Gaussian point in the second Gaussian point set using the pose.

[0031] In a specific application, in the process of training the policy network, the physical engine can use the simplified physical collision body endowed with physical parameters and the proprioceptive state of the robot and the action output by the policy network to calculate the accurate pose of the robot and the interactive object in real time ; wherein the pose is used to update the position and covariance of the Gaussian point in real time .

[0032] The proprioceptive state mentioned above can refer to information about the internal state of the robot obtained by the robot without relying on external sensors (such as cameras), and can specifically include: (1) joint state, i.e. the angle, angular velocity, angular acceleration, etc. of each joint of the robot; (2) end effector state, i.e. the position, pose, velocity of the robot gripper or tool end in the robot base coordinate system; (3) inertial measurement unit (IMU) data, i.e. linear acceleration and angular velocity, etc. of the robot base; (4) force / torque sensor data, i.e. force or torque information measured at the joint or wrist, etc. These data are usually directly measured by the robot's built-in encoders, IMUs, and force / torque sensors, etc. and are bottom-layer data that can be directly accessed by the robot control system. When calculating the pose , the physical engine will integrate and solve according to the action output by the policy network (such as joint torque, etc.) and in combination with the dynamics model of the robot (which includes proprioceptive information such as joint state, etc.); therefore, the proprioceptive state is one of the inputs for the physical engine to calculate the pose, and it is also provided as an independent input channel to the policy network together with the visual information, enabling it to make more accurate decisions.

[0033] The updating process of the position and covariance of the Gaussian point can be completed by constructing and applying a transformation matrix, and the specific implementation is as follows: 1) Pose representation: the new pose of the interactive object calculated by the physical engine is composed of a rotation matrix (representing the pose) and a translation vector (representing the position), which can be combined into a rigid body transformation matrix as .

[0034] 2) Update of Gaussian point position (mean value): for each Gaussian point belonging to the interactive object, the original position (mean value) is a three-dimensional vector, and the new position after updating is calculated by the following affine transformation:

[0035] The affine transformation calculation transforms the position coordinates of the Gauss points in their local coordinate system (or initial world coordinate system) to the position coordinates in the new world coordinate system calculated by the physics engine.

[0036] 3) Update of the Gauss point covariance: Covariance matrix The shape and direction of the Gauss points are described, and when the interactive object rotates, the covariance matrix must also rotate accordingly; the interactive object Gauss point covariance update process is as follows:

[0037] wherein, is the transpose of the rotation matrix , and and are the original covariance and the new covariance of each Gauss point belonging to the interactive object, respectively; This operation ensures that the shape of the Gauss ellipsoid remains unchanged, but the orientation of the Gauss ellipsoid in space is consistent with the rotation of the interactive object.

[0038] By applying the rigid body transformation matrix output by the physics engine, the position and orientation of each associated Gauss point can be accurately and efficiently updated, thereby realizing real-time synchronization of the visual model and the physical model.

[0039] In specific applications, a 3DGS differentiable rendering unit can be pre-configured, which can generate high-fidelity visual images observed by a virtual camera in real time through a CUDA-accelerated differentiable rasterization algorithm after receiving the updated three-dimensional Gauss model .

[0040] The implementation principle of the CUDA-accelerated differentiable rasterization algorithm is as follows: 1) Preprocessing and sorting: in the CUDA kernel, all three-dimensional Gauss points are projected onto the 2D camera view plane in parallel; in order to process efficiently, the 2D camera view plane is divided into multiple tiles, and the algorithm quickly calculates which tiles each Gauss point covers and assigns the index of the Gauss point to the list of corresponding tiles, avoiding traversal of all Gauss points for each pixel.

[0041] 2) Parallel rasterization and depth sorting: a thread block is started in each tile, and each thread block will load all Gauss points covering each pixel in the corresponding tile and quickly sort these Gauss points based on their depth (i.e., the distance of the Gauss point from the camera) (usually using an efficient parallel sorting algorithm such as Radix Sort).

[0042] ​3) Forward Accumulation and Color Computation: After sorting, traverse the Gaussians from front to back (from the closest to the farthest from the camera); for each pixel, compute the color and opacity according to the rendering formula above Sequential accumulation, i.e., compute the color and opacity from the first Gaussian, then blend the color and opacity of the first Gaussian with the color and opacity of the second Gaussian, and so on (i.e., blend the color and opacity of the current Gaussian with the color and opacity of the next Gaussian) until the opacity of the last Gaussian or the accumulated opacity reaches 1. This process is done completely on the GPU in parallel, thus achieving extremely high rendering speed.

[0043] 4) Differentiability: Since each step of the above forward computation has an explicit mathematical definition of its contribution to the final pixel color, when backpropagating, the gradient of the final loss function with respect to each pixel color can be computed according to the chain rule, and then this gradient can be losslessly and analytically backpropagated to all the properties (including position, covariance, color, and opacity, etc.) of each Gaussian involved in the computation of this pixel color. Based on this, a custom CUDA Kernel can be specially written for 3DGS to ensure that the gradients of the entire rasterization pipeline (including sorting and accumulation) can be efficiently computed and backpropagated.

[0044] The above differentiable rasterization algorithm is a complex, highly optimized parallel computation process on the GPU, which skillfully combines sorting and accumulation operations to ensure the mathematical differentiability of the entire process, thus achieving real-time high-fidelity differentiable rendering.

[0045] As a possible implementation, the above step S106 (i.e., training the policy network based on the high-fidelity image and the internal motion state of the robot) can include: in the training process of the policy network, inputting the high-fidelity image and the internal motion state into the policy network to output a corresponding action through the policy network, and calculating a first loss based on the high-fidelity image, the internal motion state, and the corresponding action output by the policy network, and then backpropagating the gradient of the first loss through a gradient backpropagation interface to update the parameters of the policy network.

[0046] The above differentiable rasterization algorithm ensures that on the basis of rendering output high-fidelity visual images , a gradient backpropagation interface can be provided, which allows the gradient of any loss function to be backpropagated to the corresponding input through the rendering process.

[0047] Following the previous example, referring to Figure 3 , the principle of differentiable rendering and policy gradient backpropagation can include: the above differentiable rasterization algorithm renders and outputs high-fidelity visual images ; the policy network receives And the robot's proprioceptive state (i.e., the robot's internal motion state). As input and output action Policy-based networks Given the input and output, calculate the reinforcement learning loss function. The gradient backpropagation interface provided by the aforementioned 3DGS differentiable rendering unit will... gradient Backpropagation to the policy network Update its weight parameters This completes end-to-end optimization from visual feedback to policy network parameters.

[0048] Differentiable mechanisms enable policy networks to directly learn how to extract the most effective and sensitive features for control tasks from high-fidelity visual inputs.

[0049] As one possible implementation, the fine-tuning of the three-dimensional Gaussian model based on the high-fidelity image and the real visual observation image of the robot acquired in real time in step S108 above may include: during the fine-tuning process of the three-dimensional Gaussian model, calculating the second loss based on the high-fidelity image and the real visual observation image, and backpropagating the gradient of the second loss through the gradient backpropagation interface to fine-tune the color and opacity of the three-dimensional Gaussian model.

[0050] For example, the above steps for calculating the second loss based on the high-fidelity image and the real visual observation image may include: extracting features from the high-fidelity image and the real visual observation image respectively through a pre-trained feature extraction network to obtain the first feature corresponding to the high-fidelity image and the second feature corresponding to the real visual observation image; and calculating the Euclidean distance between the first feature and the second feature as the second loss.

[0051] For specific applications, see the previous example. Figure 4 As shown, real visual observation images of the robot can be collected during actual robot deployment and operation. and calculate in real time With simulated images Visual domain adaptation loss The gradient backpropagation interface provided by the aforementioned 3DGS differentiable rendering unit is used for... gradient Perform online backpropagation, and then fine-tune the 3D Gaussian model online. Visual attributes such as color and opacity.

[0052] Real visual observation images It can refer to real-world images captured by cameras deployed on real physical robots; it is a two-dimensional matrix of pixels (e.g., a single image). RGB images), are external perception information. Real visual observation images for comparison with the simulation images to correct the visual performance of the simulation environment.

[0053] Computing the visual domain adaptation loss A pre-trained feature extraction network is introduced to calculate the perceptual loss . The specific calculation method is as follows: 1) Select a feature extractor: select a deep convolutional neural network pre-trained on a large image dataset (such as ImageNet), such as VGG-16 or ResNet. This network is used as a fixed feature extractor, and its weights are not updated when calculating the loss.

[0054] 2) Feature extraction: input the simulation image and the real image into the feature extractor for feature extraction.

[0055] 3) Calculate the L2 loss: select one or more intermediate layers (such as , , etc.) in the feature extractor to output the simulation image and the real image each corresponding feature map, calculate the Euclidean distance (i.e. L2 loss) between the two sets of feature maps (i.e. the simulation image and the real image each corresponding feature map):

[0056] where, denotes the feature extraction operation of the feature extractor, denotes the feature map corresponding to the real image extracted from the feature extractor, denotes the feature map corresponding to the simulation image extracted from the feature extractor.

[0057] The purpose of introducing the above visual domain adaptation loss calculation method is to require the simulation image and the real image to be as similar as possible in the more important deep features (such as texture, style, object parts, etc.) of human visual perception, so as to realize visual domain adaptation.

[0058] Color and opacity are the most direct properties of 3D Gaussian models that affect the rendering appearance, and the SH (Spherical Harmonics) coefficients and covariance of 3D Gaussian models can be adjusted when fine-tuning the visual properties of 3D Gaussian models.

[0059] In 3DGS, the color of a 3D Gaussian model is actually represented by a set of SH coefficients, which makes the Gaussian point exhibit color effects that change with the viewing angle (view-dependent), so fine-tuning the color can actually be achieved by fine-tuning these SH coefficients, which is a higher-dimensional and more detailed adjustment than directly fine-tuning the color.

[0060] In some advanced applications, if the shape or texture details of a real-world object differ from the simulation results, the covariance matrix of the 3D Gaussian model can also be fine-tuned to change the shape and size of the Gaussian ellipsoid to better match the real geometry, but fine-tuning the covariance matrix of the 3D Gaussian model has high computational cost and may affect the stability of physical interaction.

[0061] The Real-to-Sim feedback correction mechanism introduced by the above step S108 can ensure that the visual fidelity of the simulation environment dynamically adapts to the subtle differences in real-world lighting, camera white balance, and other time-varying factors, forming a continuous and automatic closing loop that reduces the Sim2Real gap.

[0062] For ease of understanding, the implementation of the above-mentioned differentiable simulation method based on 3D Gaussian sputtering is described below as an example.

[0063] The above-mentioned differentiable simulation method based on 3D Gaussian sputtering can mainly include the following steps: Step 1, high-fidelity reconstruction of real scene and accurate registration of physical properties (such as Figure 2 as shown).

[0064] This step aims to build a digital twin environment from the real world that has both photo-level visual details and meets the needs of physical engine interaction; this step mainly includes: 3D Gaussian model reconstruction, accurate registration of physical properties (dividing the 3D Gaussian model into a background Gaussian point set and an interactive object Gaussian point set , and registering them to the physical engine).

[0065] Step 2, integration of 3DGS differentiable renderer (including the above-mentioned 3DGS optimization algorithm) and dynamics.

[0066] This step aims to bridge the gap between high-fidelity vision and high-efficiency dynamics, enabling real-time, differentiable rendering output. This step primarily includes configuring a pose synchronization mechanism (i.e., during policy network training, the physics engine calculates the accurate poses of the robot and interactive objects in real time). and application Real-time updates The calculation process of the position and covariance of the second Gaussian point in the middle), differentiable rendering output (pre-configured 3DGS differentiable renderer supports real-time generation of high-fidelity visual images via CUDA-accelerated differentiable rasterization algorithm). The introduction of a gradient backpropagation mechanism (providing a gradient backpropagation interface through the 3DGS differentiable renderer, allowing the gradient of any loss function to be backpropagated) (Perform reverse propagation).

[0067] Step 3, end-to-end optimization of the strategy based on differentiable visual feedback (e.g.) Figure 3 (As shown).

[0068] This step utilizes a differentiable mechanism to directly optimize the visual input into policy weights; this step mainly includes: policy input and loss calculation (policy network). take over And the robot's proprioceptive state As input and output action Based on policy network The input and output are used to calculate the reinforcement learning loss function. End-to-end optimization (through the gradient backpropagation interface provided by the 3DGS differentiable renderer) gradient Backpropagation to the policy network Update its weight parameters ).

[0069] Step 4, Sim2Real online adaptation and model fine-tuning closed loop (e.g.) Figure 4 (As shown).

[0070] This step aims to address the significant discrepancy between Sim2Real and real-world data after the deployment of a real robot, dynamically correcting the simulation environment through feedback from the real world. This step primarily includes: real data acquisition and loss calculation (acquiring real-world visual observation images of the robot during deployment and operation). And calculate in real time With simulated images Visual domain adaptation loss Online model fine-tuning (via the gradient backpropagation interface provided by the 3DGS differentiable renderer) gradient Backpropagation, online fine-tuning visual attributes of the robot and the interactive object.

[0071] To implement the above-mentioned differentiable simulation method based on three-dimensional Gaussian sputtering, a differentiable simulation system can be designed, as shown in Figure 5 The architecture of the differentiable simulation system mainly includes the following units: (1) Acquisition and reconstruction unit, for: acquiring real data (i.e. image data and pose data of the real robot working area) and reconstructing a three-dimensional Gaussian model , then dividing the three-dimensional Gaussian model into a background Gaussian point set and an interactive object Gaussian point set by semantic segmentation, and registering to the physics engine.

[0072] (2) Physical interaction unit, for: calculating the kinematics and dynamics of the robot and the interactive object respectively through the physics engine, and outputting the pose of the robot and the interactive object respectively .

[0073] (3) 3DGS differentiable rendering unit, for: receiving the pose and the three-dimensional Gaussian model , using the differentiable rasterization algorithm to generate high-fidelity visual images in real time, and providing a gradient backpropagation interface.

[0074] (4) Reinforcement learning training unit, including a policy network and an optimizer; the reinforcement learning training unit is used for: the policy network receives the high-fidelity visual image and the body perception state of the robot , the optimizer calculates the reinforcement learning loss function and uses its gradient backpropagation to perform policy end-to-end optimization based on differentiable visual feedback.

[0075] (5) Sim2Real adaptive unit, for: receiving real-world feedback data (i.e. real visual observation image ), calculating , and dynamically fine-tuning the three-dimensional Gaussian model .

[0076] The specific embodiments of the above-mentioned differentiable simulation method based on three-dimensional Gaussian sputtering are described in detail below in conjunction with Figure 5 .

[0077] In terms of hardware configuration, at least 4 high-resolution RGB-D cameras (such as RealSense D455, etc.) are used for data acquisition, and high-performance GPUs (such as NVIDIA RTX 4090, NVIDIA RTX A100, etc.) are used as computing units; in terms of software environment, a physical engine supporting GPU acceleration (such as NVIDIA Isaac Gym, MuJoCo, etc.) is used, and a 3DGS library is used to ensure that the rendering speed reaches at a resolution of and a secondary development of a differentiable rasterization interface is carried out, and an RL framework (such as PyTorch, TensorFlow, etc.) is used to realize the policy network and optimization algorithm (such as PPO, SAC, etc.).

[0078] The specific implementation steps of the above-mentioned differentiable simulation method based on three-dimensional Gaussian sputtering are as follows: Step one, high-fidelity digital twin model construction and physical property assignment.

[0079] This step mainly includes 3DGS model reconstruction and physical property assignment.

[0080] 3DGS model reconstruction: using 3DGS reconstruction algorithm, a three-dimensional Gaussian model with position, covariance, color and opacity attributes is obtained . In order to balance accuracy and speed, the number of Gaussian spheres of the three-dimensional Gaussian model is controlled to be ; the reconstruction loss function can use the weighted combination of L1 loss and SSIM (structural similarity) loss, and the weight value range of L1 loss and SSIM loss respectively can be and , the sum of the respective weights of L1 loss and SSIM loss is 1, for example, the respective weights of L1 loss and SSIM can be 0.8 and 0.2. L1 loss guarantees pixel-level color accuracy; SSIM loss pays more attention to the structure, brightness and contrast of the image, and can preserve more high-frequency details, making the reconstruction result more natural in vision. The greater the value of L1 loss indicates the greater emphasis on pixel accuracy, and the greater the value of SSIM loss indicates the greater emphasis on structural similarity.

[0081] Physical property assignment: divide the Gaussian points in the three-dimensional Gaussian model into a background point set and an interactive object point set ; in , create a physical collision body (such as a bounding box or convex hull) at the geometric center of each point and assign it physical parameters (such as mass , friction coefficient , etc.) and register it to the physical engine, Keep static. Assigning physical parameters is a key step to turn the pure visual 3DGS model into an entity that can be interacted with, operated in the simulation world. The physical parameters assigned are the physical basis for all subsequent dynamic calculations and reinforcement learning processes.

[0082] Step two, 3DGS differentiable renderer integration and rendering.

[0083] This step mainly includes pose synchronization and Gaussian point transformation, 3DGS differentiable rasterization.

[0084] Pose synchronization and Gaussian point transformation: the physical engine calculates the accurate pose of the robot and the interactive object respectively , which can be applied to real-time update the position and covariance of the corresponding interactive object Gaussian point.

[0085] 3DGS differentiable rasterization: using 3DGS differentiable rasterization algorithm, calculate the color of each pixel according to the rendering formula above.

[0086] The specific process of 3DGS differentiable rasterization mainly includes blocking, sorting and accumulation. Sorting is based on the depth value of Gaussian point projection to 2D plane, which can ensure correct occlusion relationship and mixing relationship; accumulation is in the order of sorting results from front to back (from the closest to the farthest from the camera) to apply the rendering formula for accumulation. The whole process of 3DGS differentiable rasterization is realized through self-defined CUDA Kernel, which ensures that the contribution of each operation to the final pixel color is traceable and differentiable, so as to realize efficient gradient back propagation. The core of realizing 3DGS differentiable rasterization lies in that the sorting operation and accumulation operation are differentiable, allowing the gradient of the loss function to penetrate the rendering process and back propagate to the Gaussian point attribute.

[0087] Step three, end-to-end reinforcement learning training.

[0088] This step mainly includes policy network input, gradient back propagation.

[0089] Policy network input: the policy network receives the visual observation image generated by 3DGS differentiable renderer rendering and the body perception state of the robot .

[0090] Gradient back propagation: calculate the RL loss function , through the gradient back propagation interface provided by 3DGS differentiable renderer, the gradient of is back propagated to , and then the weight parameters of the policy network are updated​ This enables end-to-end optimization of the policy network parameters, allowing the policy network to directly learn how to extract the most effective features for the control task from high-fidelity visual input, thus avoiding the disconnect between the visual feature extraction process and the control training process.

[0091] Step 4: Sim2Real online adaptive testing.

[0092] This step mainly includes visual loss calculation and model fine-tuning.

[0093] Visual loss calculation: Acquiring real visual observation images ,calculate and Visual domain adaptation loss (For example, using L2 loss in the VGG feature space).

[0094] Model fine-tuning: This is achieved through the gradient backpropagation interface provided by the 3DGS differentiable renderer. gradient Backpropagation to the 3D Gaussian model To fine-tune the 3D Gaussian model online Visual properties such as color and opacity of mid-Gaussian points; among them, online fine-tuning of the 3D Gaussian model. The color of the mid-Gaussian point can be achieved by fine-tuning the SH coefficient, through online fine-tuning of the 3D Gaussian model. The opacity of the mid-Gaussian point can correct visual differences caused by sensor noise or changes in the material of the object's surface (such as reflections), through online fine-tuning of the 3D Gaussian model. The covariance of mid-Gaussian points can further correct for minor geometric or textural differences.

[0095] By using the Real-to-Sim feedback correction mechanism in step four above to optimize the model in a closed loop, it is possible to ensure that the visual fidelity of the simulation environment remains consistent with that of the real world, thereby achieving a dynamic reduction in the Sim2Real gap.

[0096] The aforementioned differentiable simulation method based on 3D Gaussian sputtering achieves the construction and optimization of a high-fidelity, high-efficiency, and differentiable simulation environment through 3DGS, which can significantly reduce the generalization error of visual Sim2Real and significantly improve the training efficiency of RL compared with the method of using neural rendering technology.

[0097] In practical applications, four-dimensional Gaussian sputtering (4DGS) or a dynamic Gaussian field can be used to model a dynamic scene model containing moving objects or deformed components instead of the above-mentioned process of constructing a three-dimensional Gaussian model, and a dynamic scene model is used instead of the above-mentioned three-dimensional Gaussian model, thereby expanding the simulation capability of the system for dynamic work environments. Mip-Splatting or anti-aliasing 3DGS can be used to optimize rendering quality to eliminate aliasing errors, improve rendering consistency, and reduce high-frequency noise in backpropagation. Two-dimensional Gaussian sputtering (2DGS) can be introduced on the basis of 3DGS to strengthen the modeling of scene surface geometric information (such as the flatness of a plane), and 2DGS can provide better geometric normal constraints to more accurately perform collision detection and improve the physical simulation accuracy when objects are in contact.

[0098] In practical applications, in specific tasks where the rendering speed requirement is not high, NeRF (Neural Radiance Field), Instant NGP (Instant Neural Graphics Primitives), or TensoRF can be used instead of the above-mentioned 3DGS differentiable rendering unit to maintain the closed loop from real reconstruction to differentiable policy optimization; in particular, NeRF has an advantage in handling transparent objects or complex refraction scenes, and can ensure that the methodology of the present application can achieve end-to-end optimization based on visual control under different neural rendering frameworks.

[0099] In practical applications, the 3DGS differentiable rendering unit can also be combined with a gradient-based physics engine (such as DiffTaichi, Warp, Brax, etc.), so that not only is the rendering process differentiable, but the kinematics and dynamics of the robot arm (such as joint force and friction calculation) also have explicit gradients, enabling end-to-end differentiable training of the entire process from vision to physics to control, and the gradient can penetrate the visual rendering layer and the dynamics simulation layer to directly act on the underlying parameters of the controller, greatly shortening the search path of the policy in complex force control tasks.

[0100] In practical applications, a model prediction-based reinforcement learning architecture can also be used in the process of dynamically reducing the Sim2Real gap. Specifically, the 3DGS differentiable renderer can be used as a learnable environment prediction model, and the high-speed rendering capability of the 3DGS differentiable renderer can be used for forward prediction to predict the future visual state in the latent space, and real-time control can be performed in combination with the model prediction result. This approach significantly improves sample efficiency by converting 3DGS into a learnable environment predictor, enabling the robot to perform a preview with the help of the environment predictor's prediction result based on a small amount of real data, thereby enhancing the system's ability to handle unknown obstacles or dynamic disturbances.

[0101] In actual application, domain adaptation loss (such as maximum mean difference MMD, adversarial domain adaptation DANN) can be introduced into the bottom of the policy network in the Sim2Real adaptive process based on the fine-tuning of the 3DGS parameters through the visual loss, so as to map the simulation images and the real images into the same complementary feature space by using contrastive learning to force the policy network to extract environment-independent features. This way reduces the dependence of the system on the “absolute truth” of the simulation environment. Even if there are minor flaws in the three-dimensional Gaussian model reconstructed by 3DGS, the policy network can still accurately identify the core task features after the alignment of the feature space, thereby improving the robustness of the policy under extreme lighting or sensor drift.

[0102] In actual application, a diffusion model or a generative adversarial network can be introduced as a post-processing module of the 3DGS rendering output link to further render the 3DGS rendered images into images with real sensor noise and complex global lighting effects by training the post-processing module with real image data, and then training the policy. This way combines the geometric determinacy of 3DGS with the texture randomness of the generative model, and can simulate more diverse environmental changes (such as rain, fog, and reflection), thereby passively improving the generalization ability of the policy by increasing the diversity of the data.

[0103] The above-mentioned differentiable simulation method based on three-dimensional Gaussian sputtering provided by the present application realizes the perfect combination of high-fidelity vision and high efficiency, end-to-end training by introducing 3DGS differentiable rendering technology, and brings the following significant benefits: (1) Significantly reduce the visual Sim2Real gap: 3DGS can reconstruct a photo-level digital twin model, and its rendered images have highly realistic lighting, texture and details, so that the visual features learned by the policy can be directly generalized to the real world, which can significantly reduce the generalization error of visual Sim2Real.

[0104] (2) Realize efficient end-to-end differentiable optimization: using the high-speed differentiable rasterization technology of 3DGS, the end-to-end gradient backpropagation path from visual feedback to policy weight is opened up, which meets the high throughput demand of reinforcement learning, realizes the joint optimization of vision and control, and greatly improves the convergence speed and perception accuracy of the algorithm.

[0105] (3) More efficient policy optimization: policy gradient can directly flow through the differentiable renderer to realize end-to-end optimization from visual feedback to policy weight, accelerate convergence, and help the policy discover more sensitive and efficient motion patterns to visual features.

[0106] (Four) Quickly build a digital twin model: Compared with traditional CAD modeling or manual texture adjustment, 3DGS only needs a few minutes of multi-view image acquisition and a few hours of reconstruction to generate a high-fidelity scene, greatly reducing the construction cost and time of the simulation environment.

[0107] (Five) Improve the robustness of the strategy: The strategy network can learn a control strategy that is more sensitive and robust to minor changes in visual input. Through the Sim2Real adaptive closed-loop fine-tuning mechanism, it can dynamically adapt to minor changes in real-world lighting, sensor noise, etc., further improving the applicability of embodied intelligence in complex dynamic environments.

[0108] On the basis of the above-mentioned differentiable simulation method based on three-dimensional Gaussian sputtering, the embodiment of the application further provides a differentiable simulation device based on three-dimensional Gaussian sputtering, as shown in Figure 6 The device can include the following modules: The construction module 602 is configured to construct a three-dimensional Gaussian model based on image data and pose data of a robot working area.

[0109] The rendering module 604 is configured to perform three-dimensional Gaussian sputtering differentiable rendering based on the three-dimensional Gaussian model, the internal motion state of the robot, and the action output by the policy network to generate a high-fidelity image.

[0110] The training module 606 is configured to train the policy network based on the high-fidelity image and the internal motion state of the robot; wherein the gradient of the first loss of the policy network is backpropagated during the training process of the policy network.

[0111] The fine-tuning module 608 is configured to fine-tune the three-dimensional Gaussian model based on the high-fidelity image and the real visual observation image of the robot during the process of the robot moving based on the action output by the policy network.

[0112] The above-mentioned differentiable simulation device based on three-dimensional Gaussian sputtering can construct a three-dimensional Gaussian model using image data of a robot working area, and quickly and low-costly reconstruct a high-fidelity simulation environment through three-dimensional Gaussian sputtering differentiable rendering combined with the internal motion state of the robot and the action output by the policy network. Furthermore, the policy network training and three-dimensional Gaussian model fine-tuning are performed combined with the internal motion state of the robot and the real visual observation image, realizing end-to-end optimization from visual feedback to policy network parameters, making policy optimization more efficient, reducing the cost and time of simulation environment construction, and further improving the applicability of embodied intelligence in complex dynamic environments.

[0113] The differentiable simulation device based on three-dimensional Gaussian sputtering provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned differentiable simulation method based on three-dimensional Gaussian sputtering. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0114] This invention also provides an electronic device, such as... Figure 7 The diagram shows the structure of the electronic device, which includes a processor 71 and a memory 70. The memory 70 stores computer-executable instructions that can be executed by the processor 71. The processor 71 executes the computer-executable instructions to implement the above-mentioned differentiable simulation method based on three-dimensional Gaussian sputtering.

[0115] exist Figure 7 In the illustrated embodiment, the electronic device further includes a bus 72 and a communication interface 73, wherein the processor 71, the communication interface 73, and the memory 70 are connected via the bus 72.

[0116] The memory 70 may include high-speed random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device. Communication between this system network element and at least one other network element is achieved through at least one communication interface 73 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc. The bus 72 may be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 72 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0117] The processor 71 can be an integrated circuit chip with signal processing capability. In the implementation process, the steps of the above-described differentiable simulation method based on three-dimensional Gaussian sputtering can be completed by integrated logic circuits of hardware in the processor 71 or instructions in the form of software. The processor 71 described above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the differentiable simulation method based on three-dimensional Gaussian sputtering disclosed in combination with the embodiments of the present application can be directly embodied as hardware decoding processor execution completion, or executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, or other mature storage media in the art. The storage medium is located in the memory, and the processor 71 reads the information in the memory, and combines the hardware thereof to complete the steps of the differentiable simulation method based on three-dimensional Gaussian sputtering of the above-described embodiments.

[0118] Unless specifically stated, the relative steps, numerical expressions, and numerical values of the components and steps set forth in these embodiments do not limit the scope of the present application.

[0119] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present application or the part of the present application that essentially contributes to the prior art or the part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the differentiable simulation method based on three-dimensional Gaussian sputtering described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0120] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second", "third" are only for the purpose of description, and cannot be understood as indicating or implying relative importance.

[0121] Finally, it should be noted that the above-described embodiments are only specific embodiments of the present application, which are used to illustrate the technical solutions of the present application, and are not limiting. The protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily think of changes to the technical solutions recorded in the foregoing embodiments, or make equivalent replacements to some technical features, within the technical range disclosed by the present application. Such modifications, changes or replacements do not cause the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A differentiable simulation method based on three-dimensional Gaussian sputtering, characterized in that, include: A 3D Gaussian model is constructed based on image and pose data of the robot's working area; Based on the aforementioned 3D Gaussian model, the robot's internal motion state, and the actions output by the policy network, 3D Gaussian sputtering differentiable rendering is performed to generate a high-fidelity image. A policy network is trained based on the high-fidelity image and the robot's internal motion state; wherein, during the training of the policy network, the gradient of the first loss of the policy network is backpropagated. During the robot's movement based on the actions output by the policy network, the three-dimensional Gaussian model is fine-tuned based on the high-fidelity image and the robot's actual visual observation image.

2. The differentiable simulation method based on three-dimensional Gaussian sputtering according to claim 1, characterized in that, Based on the aforementioned 3D Gaussian model, the robot's internal motion state, and the actions output by the policy network, 3D Gaussian sputtering differentiable rendering is performed to generate a high-fidelity image, including: The three-dimensional Gaussian model is divided into a first Gaussian point set representing the background and a second Gaussian point set representing interactive objects. An initial physical collider is created at the geometric center of each second Gaussian point in the second Gaussian point set, and physical parameters are assigned to each initial physical collider to obtain the corresponding physical collider. Based on the physical collider, the three-dimensional Gaussian model, the robot's internal motion state, and the actions output by the policy network, three-dimensional Gaussian sputtering differentiable rendering is performed to generate a high-fidelity image.

3. The differentiable simulation method based on three-dimensional Gaussian sputtering according to claim 2, characterized in that, Based on the physical collider, the 3D Gaussian model, the robot's internal motion state, and the actions output by the policy network, 3D Gaussian sputtering differentiable rendering is performed to generate a high-fidelity image, including: Based on the physical collider, the three-dimensional Gaussian model, the robot's internal motion state, and the actions output by the policy network, a high-fidelity image is generated by rendering using a three-dimensional Gaussian sputtering differentiable rasterization algorithm; wherein, the three-dimensional Gaussian sputtering differentiable rasterization algorithm has a gradient backpropagation interface, which is used to backpropagate the gradient of the first loss of the policy network during the training process of the policy network.

4. The differentiable simulation method based on three-dimensional Gaussian sputtering according to claim 3, characterized in that, Based on the physical collider, the 3D Gaussian model, the robot's internal motion state, and the actions output by the policy network, a high-fidelity image is generated through rendering using a 3D Gaussian sputtering differentiable rasterization algorithm, including: The physics engine calculates the poses of the robot and the interactive object by using the physical collider, the robot's internal motion state, and the actions output by the policy network. Based on the three-dimensional Gaussian model and the pose, the high-fidelity image is generated by rendering using the three-dimensional Gaussian sputtering differentiable rasterization algorithm.

5. The differentiable simulation method based on three-dimensional Gaussian sputtering according to claim 4, characterized in that, The three-dimensional Gaussian model has color and opacity; based on the three-dimensional Gaussian model and the pose, the high-fidelity image is generated by rendering using the three-dimensional Gaussian sputtering differentiable rasterization algorithm, including: The color value of each pixel in the high-fidelity image is calculated using the following formula via the three-dimensional Gaussian sputtering differentiable rasterization algorithm: in, For pixel color values, For the three-dimensional Gaussian model, the first The color of a Gaussian point and The three-dimensional Gaussian model is respectively the first The Gaussian point and the first The transparency of each Gaussian point.

6. The differentiable simulation method based on three-dimensional Gaussian sputtering according to claim 5, characterized in that, The three-dimensional Gaussian model also has position and covariance; Based on the 3D Gaussian model and the pose, the high-fidelity image is generated by rendering using the 3D Gaussian sputtering differentiable rasterization algorithm, and the process further includes: The position and covariance of the second Gaussian point in the second Gaussian point set are updated using the pose.

7. The differentiable simulation method based on three-dimensional Gaussian sputtering according to claim 6, characterized in that, Training a policy network based on the high-fidelity image and the robot's internal motion state includes: during the training process of the policy network, inputting the high-fidelity image and the internal motion state into the policy network to output corresponding actions through the policy network, calculating a first loss based on the high-fidelity image, the internal motion state, and the corresponding actions output by the policy network, and then backpropagating the gradient of the first loss through the gradient backpropagation interface to update the parameters of the policy network. Fine-tuning the 3D Gaussian model based on the high-fidelity image and the robot's real visual observation image includes: during the fine-tuning process of the 3D Gaussian model, calculating a second loss based on the high-fidelity image and the real visual observation image, and backpropagating the gradient of the second loss through the gradient backpropagation interface to fine-tune the color and opacity of the 3D Gaussian model.

8. The differentiable simulation method based on three-dimensional Gaussian sputtering according to claim 7, characterized in that, The second loss is calculated based on the high-fidelity image and the real visual observation image, including: The high-fidelity image and the real visual observation image are extracted by a pre-trained feature extraction network to obtain the first feature corresponding to the high-fidelity image and the second feature corresponding to the real visual observation image. The Euclidean distance between the first feature and the second feature is calculated as the second loss.

9. A differentiable simulation device based on three-dimensional Gaussian sputtering, characterized in that, include: The building module is used to construct a 3D Gaussian model based on image and pose data of the robot's working area; The rendering module is used to perform three-dimensional Gaussian sputtering differentiable rendering to generate a high-fidelity image based on the three-dimensional Gaussian model, the robot's internal motion state and the actions output by the policy network. A training module is used to train a policy network based on the high-fidelity image and the robot's internal motion state; wherein, during the training of the policy network, the gradient of the first loss of the policy network is backpropagated. The fine-tuning module is used to fine-tune the three-dimensional Gaussian model based on the high-fidelity image and the robot's real visual observation image during the robot's movement based on the actions output by the policy network.

10. An electronic device, characterized in that, It includes a processor and a memory, the memory storing computer-executable instructions that can be executed by the processor, the processor executing the computer-executable instructions to implement the differentiable simulation method based on three-dimensional Gaussian sputtering as described in any one of claims 1 to 8.

Images