Robot attitude estimation method based on 3D Gaussian spatter and double-attention ray scoring network

Abstract

The invention discloses a robot attitude estimation method based on 3D Gaussian spatter and a double-attention ray scoring network, and aims to solve the problems that a single RGB image method is easily influenced by initial attitude dependence and rotation ambiguity, and a method depending on depth or multiple views is high in cost in storage and data acquisition. According to the method, a 3D Gaussian scene model of an indoor environment is obtained based on a 3D Gaussian splash method, a double-attention ray scoring network (DARS-Net) is established, ray scores are decomposed into position scores and direction scores through an improved geometric scoring mechanism, a high-position-score ray is used for predicting the position of a camera, and a high-direction-score ray is used for predicting the orientation of the camera. The DARS-Net effectively overcomes rotation ambiguity, and the translation and rotation precision is remarkably improved. And finally, further refining a coarse pose obtained based on 3DGS rays through efficient feature point matching. The invention belongs to the field of indoor service robots.

Description

Technical Field

[0001] This invention relates to the field of indoor service robots, and specifically to a posture estimation method for indoor service robots based on 3D Gaussian splashing and dual attention ray scoring networks. Background Technology

[0002] In the field of indoor service robots and augmented reality (AR) navigation, 6-DOF pose estimation is fundamental for achieving obstacle avoidance, grasping, and virtual-real fusion in indoor service robots. Currently, this task mainly relies on RGB-D sensors for environmental perception. Although these sensors have become standard equipment in many robotic vacuum cleaners or interactive robots, they often fail when facing highly reflective or transparent materials such as floor-to-ceiling windows and mirrors, causing indoor service robots to get lost or collide. At the same time, the high cost of sensors also limits the widespread adoption of consumer products. While monocular RGB solutions have wider applicability, they face a trade-off between accuracy and efficiency. Existing instance-level RGB pose estimation algorithms typically assume a known, precise 3D CAD model of the target. However, in real-world home scenarios, furniture styles vary greatly, and there are not only many areas with weak textures (such as solid-color walls and cabinets), but the placement and types of items also change dynamically. This requires indoor service robots to perform generalized perception without having to obtain 3D models of all objects in advance, which has become a major obstacle that current technology struggles to overcome.

[0003] To overcome this obstacle and move away from relying on object CAD models, researchers have begun to utilize the implicit representation of the scene itself to assist pose estimation, with scene reconstruction being the most representative example. Scene reconstruction mainly includes Neural Radiation Field (NeRF) and 3D Gaussian Scatter (3DGS). However, directly applying these reconstruction techniques to monocular pose estimation still faces challenges. NeRF-based methods require a large amount of computational ray stepping for implicit volume representation, making real-time applications difficult to implement, and training requires dense multi-view images, contradicting the single-image inference root requirement of many tasks. They still heavily rely on the initial pose and are prone to getting trapped in local optima. 3DGS methods such as SplatLoc and 3DGS-ReLoc rely on dense depth and multi-view images to complete scene reconstruction and initial pose acquisition, leading to a significant increase in storage and data acquisition costs. Conversely, methods based on single RGB images (such as 6DGS) directly utilize the differentiable rendering of 3DGS through rendering inversion, avoiding the use of depth or multi-frame images. However, 6DGS's Gaussian ellipsoidal ray sampling strategy can produce rotational ambiguity because it favors rays with the smallest perpendicular distance to the optical center while ignoring angular deviations. In summary, single RGB image methods are susceptible to initial pose dependence and rotation ambiguity, while depth- or multi-view methods are costly in terms of storage and data acquisition. Summary of the Invention

[0004] To address the issues that single RGB image methods are susceptible to initial pose dependence and rotation ambiguity, while depth- or multi-view methods are costly in terms of storage and data acquisition, this invention proposes a robot pose estimation method based on 3D Gaussian splashing and dual-attention ray scoring networks.

[0005] The technical solution adopted in this invention is:

[0006] It includes the following steps:

[0007] S1. Use a robot to collect multi-view RGB image sequences of the indoor environment, and use motion recovery structure to process the multi-view RGB image sequences to obtain the camera pose and sparse point cloud of the indoor environment.

[0008] S2. Based on the 3D Gaussian splashing method, each point in the sparse point cloud is initialized as a 3D Gaussian primitive. Each 3D Gaussian primitive is projected onto the camera's two-dimensional image plane for rendering according to the camera pose. The loss between the rendered image and the RGB image in S1 is calculated. The Gaussian properties are gradually updated using backpropagation to obtain a 3D Gaussian scene model.

[0009] S3. Establish a dual-attention ray scoring network, which includes an enhanced MLP network, a DINOv2 backbone network, and an attention module. and attention module The enhanced MLP network is set up in parallel with the DINOv2 backbone network, and the attention module... and attention module The enhanced MLP network is set up in parallel with the attention module. and attention module The DINOv2 backbone network is connected to the attention module. and attention module connect;

[0010] S4. Obtain the monocular RGB image of the indoor environment in S1. Input the monocular RGB image into the 3D Gaussian scene model to obtain the projected ray. Input the monocular RGB image and the projected ray into the dual attention ray scoring network and output the position score and direction score of the projected ray.

[0011] S5. Based on position and orientation scores, a Top-K filtering strategy is used to obtain high-confidence rays. Based on the high-confidence rays, the weighted sum algorithm of orientation vectors combined with the weighted least squares method is used to solve the approximate position and orientation of the camera. Based on the approximate position and orientation of the camera and the 3D Gaussian scene model, a rendered view of the indoor environment is generated.

[0012] S6. Use the feature matching algorithm to establish a correspondence between the monocular RGB image to be tested and the rendered view to obtain the robot's pose residual. Use the PnP algorithm to minimize the pose residual to obtain the 6-DOF robot pose.

[0013] The beneficial effects of this invention are as follows:

[0014] This invention proposes a novel framework integrating 3D Gaussian splashing and a bi-branch neural architecture, enabling geometric perception for 6-DOF pose estimation from a single RGB image via 3D Gaussian splashing. The DARS-Net attention mechanism effectively mitigates rotational ambiguity issues in 6-DoF pose estimation under single RGB image conditions.

[0015] This invention proposes a coarse-to-fine pose estimation process. Based on the 3D Gaussian splash method, a 3D Gaussian scene model of the indoor environment is obtained. A Dual Attention Ray Scoring Network (DARS-Net) is then established. Through an improved geometric scoring mechanism, ray scoring is decomposed into position and orientation scores. Rays with high position scores are used to predict camera position, and rays with high orientation scores are used to predict camera orientation. DARS-Net effectively overcomes rotation ambiguity and significantly improves translation and rotation accuracy. Finally, efficient feature point matching further refines the coarse pose obtained from 3DGS rays.

[0016] This invention is robust and scalable, pushing the boundaries of 3D model-based pose estimation technology. Attached Figure Description

[0017] Figure 1 This is a flowchart of the present invention; Detailed Implementation

[0018] Specific implementation method one: Combining Figure 1 This embodiment describes a robot pose estimation method based on 3D Gaussian spatter and dual-attention ray scoring network, which includes the following steps:

[0019] S1. An indoor service robot uses its monocular RGB camera to acquire multi-view RGB image sequences of the indoor environment. Structure for Motion Restoration (SfM) is used to preprocess these multi-view RGB image sequences to obtain the camera pose at the time of image acquisition and the sparse point cloud of the indoor environment. This serves as the initial input for the indoor scene. Precise camera pose provides a unified world coordinate system benchmark for subsequent scene optimization, ensuring the geometric consistency of multi-view images in three-dimensional space. This enables the invention to correctly establish the correspondence between pixels under different viewpoints, thereby achieving accurate calculation of the scene's geometric structure and appearance.

[0020] S2. Based on the 3D Gaussian splashing method, each point in the sparse point cloud is initialized as a 3D Gaussian element, and all 3D Gaussian elements are used to explicitly represent the indoor environment. In the world coordinate system, each 3D Gaussian element is represented by a mean vector. Covariance matrix Opacity and color vectors represented by spherical harmonic coefficients (SH) Composition. The mean vector is used to characterize the position of the Gaussian element, and the covariance matrix is ​​used to characterize the shape of the Gaussian element. The probability density function of each 3D Gaussian element is then... The representation is as follows:

[0021] (1)

[0022] in, is the base of the natural logarithm. Let be the three-dimensional coordinate (world coordinate system) vector of any point in the indoor environment. for The transpose of .

[0023] To ensure the covariance matrix To maintain physical validity during optimization, it is decomposed into scaling matrices. and rotation matrix Scaling matrix From three-dimensional scale vector To obtain, that is Rotation matrix Obtained through quaternion parameterization. The decomposed covariance matrix. Represented as:

[0024] (2)

[0025] in, for transpose, for The transpose of .

[0026] To convert a 3D scene into a 2D image, each 3D Gaussian primitive is projected onto the camera's 2D image plane for rendering based on the camera pose. The processing steps are as follows:

[0027] (1) Based on the camera pose, use the view transformation matrix And the Jacobian matrix obtained by the affine approximation of the projection transformation The covariance matrix of each 3D Gaussian element in the camera coordinate system is calculated. :

[0028] (3)

[0029] The origin of the camera coordinate system points to the optical center of the camera, the Z-axis points directly in front of the camera, the x-axis is horizontal to the right, and the y-axis is vertically downward.

[0030] (2) Based on the mean vector of the 3D Gaussian center (two-dimensional pixel coordinates) Covariance matrix of each 3D Gaussian element The three-dimensional Gaussian distribution is obtained using formula (1). .

[0031] (3) In order to achieve photorealistic rendering, a tile-based differential rasterizer is selected. Hybrid strategy. A tile-based differentiable raster is used to sort each overlapping 3D Gaussian primitive in the 3D Gaussian distribution according to its depth from shallow to deep. The weight of each 3D Gaussian primitive is determined based on the sorting result, and the weight is mapped to a value between 0 and 1, such as 0.1, 0.2, etc., according to the depth value. The color projected onto the pixel by each 3D Gaussian primitive is calculated based on the weight. The blending strategy accumulates the color attribute of each pixel, and the resulting color after accumulation is... It can be represented as:

[0032] (4)

[0033] in, The number of Gaussian pixels participating in the accumulation of each pixel. For the first The color of the high-sky element, , For the first The contribution of each Gaussian unit to the opacity Cumulative transmittance For the first The opacity contribution of each high-order pixel is calculated. This yields the rendering result for each pixel.

[0034] (4) Based on the rendering result of each pixel, obtain the rendered image, calculate the loss between the rendered image and the RGB image in S1, and use backpropagation to gradually update the Gaussian properties to obtain the 3D Gaussian scene model. The specific process is as follows:

[0035] By continuously accumulating each pixel, a rendered image corresponding to the RGB image acquired by S1 is obtained. The L2 loss between the rendered image and the RGB image is continuously calculated. Through backpropagation, the Gaussian attributes (position, shape, opacity, color) are gradually updated, and finally a 3D Gaussian scene model containing high-fidelity texture and geometric details is obtained. The 3D Gaussian scene model supports real-time, photorealistic new view synthesis and rendering from any new perspective.

[0036] S3. The rotational ambiguity problem in monocular 6-DOF pose estimation mainly stems from the fact that traditional ray scoring methods do not distinguish between spatial and angular information. To solve this problem, this invention establishes a dual-attention ray scoring network (DARS-Net). This network uses a dual-attention mechanism to model the position and orientation scores of the projected rays separately, thereby independently estimating the camera's position and pose.

[0037] The dual-attention ray scoring network includes an enhanced MLP network, a DINOv2 backbone network, and an attention module. and attention module The enhanced MLP network is set up in parallel with the DINOv2 backbone network, and the attention module... and attention module The enhanced MLP network is set up in parallel with the attention module. and attention module The DINOv2 backbone network is connected to the attention module. and attention module connect.

[0038] The enhanced MLP network sequentially comprises a positional encoding layer, a fully connected feature extraction layer, and a feature projection layer. The fully connected feature extraction layer consists of multiple linear layers interspersed with nonlinear activation functions, specifically including eight MLP+nonlinear activation function structures. The fully connected feature extraction layer also employs residual connections or skip connections. These residual or skip connection structures fuse the original input with features from intermediate layers, enhancing feature propagation and preventing gradient vanishing or information loss in deep networks.

[0039] Attention module and attention module Using the same structure, each layer sequentially includes a fully connected layer, a correlation calculation layer, a normalization layer, and a softmax layer. Attention module. and attention module They have the same input, but output position and orientation scores respectively.

[0040] S4. Obtain the monocular RGB image of the indoor environment in S1. Input the monocular RGB image into the 3D Gaussian scene model to obtain the projected ray. Input the monocular RGB image and the projected ray into the dual-attention ray scoring network to output the independent and decoupled projected ray position score and direction score. The specific process is as follows:

[0041] S41. Obtain the monocular RGB image of the indoor environment in S1. The monocular RGB image to be tested The input is a 3D Gaussian scene model, which generates multiple projection rays for each Gaussian ellipsoid and automatically selects those that match the monocular RGB image to be tested. The corresponding ray is used as the output.

[0042] S42. The monocular RGB image to be tested And in the corresponding ray input dual-attention ray scoring network, the position score and direction score of each projected ray are output. The specific process is as follows:

[0043] S421, The monocular RGB image to be tested It inputs into the DINOv2 backbone network and outputs a set of image features. Image features ,in, , Image width, Image height, For feature dimensions.

[0044] S422. The corresponding rays are input into the position encoding layer of the enhanced MLP network. Based on the original geometric parameters of each ray, the position encoding layer maps each ray to a high-dimensional space using a sine function, cosine function, or hash encoding, generating a high-frequency position embedding vector for each ray. This overcomes the spectral bias problem that makes it difficult for neural networks to capture high-frequency details. The original geometric parameters consist of the coordinate vectors of the ray's origin and direction.

[0045] The high-frequency position of each ray is then embedded into the vector input fully connected feature extraction layer, and the fused feature of each ray is output.

[0046] Finally, the fused features of each ray are input into the feature projection layer, which then uses a linear transformation to adjust the fused feature dimension of each ray to the preset feature dimension. Output the feature set for each ray. The features of each ray are: ,in, This represents the total number of rays, thereby improving the feature discrimination capability.

[0047] S423. Input the feature set of each ray and the image feature set into the attention module. In the fully connected layer, ray features are used as queries and image features are used as keys. The ray features and image features are mapped to a unified attention dimension space to generate a set of transformed query vectors Q and a set of key vectors K.

[0048] The query vector set and the key vector set are input into the relevance calculation layer. By performing matrix multiplication on the query vector set and the key vector set, the similarity between each ray and each image pixel feature is calculated, resulting in multiple original attention maps with a size of "number of rays × number of image pixels".

[0049] Multiple original attention maps are input into the normalization layer. The Softmax operation is used to normalize the correlation values ​​of each original attention map, generating probability distribution weights with values ​​ranging from 0 to 1, resulting in multiple normalized original attention maps.

[0050] Multiple normalized original attention maps are input into the Softmax layer. For each normalized original attention map, a row-wise summation operation is performed, which accumulates the attention weights of a single ray relative to all pixels to characterize the overall relevance strength of the ray with the entire image. Based on the functional definition of the dual attention ray scoring network, the direction score of each ray is output. ,in, For attention module The Middle 1 pixel.

[0051] S424. Input the feature set of each ray and the image feature set into the attention module. In the fully connected layer, ray features are used as queries and image features are used as keys. The ray features and image features are mapped to a unified attention dimension space to generate a set of query vectors and a set of key vectors.

[0052] The query vector set and the key vector set are input into the relevance calculation layer, and matrix multiplication is performed on the query vector set and the key vector set to obtain multiple original attention maps with a size of "number of rays × number of image pixels".

[0053] Multiple original attention maps are input into the normalization layer to obtain multiple normalized original attention maps;

[0054] Multiple normalized original attention maps are input into the Softmax layer. For each normalized original attention map, a row-wise summation operation is performed, which accumulates the attention weights of a single ray relative to all pixels, and outputs the position score of each ray. ,in, For attention module The Middle 1 pixel.

[0055] Before the formal application of the dual-attention ray scoring network, this invention trains the dual-attention ray scoring network by repeatedly executing S41 and S42 to obtain a stable parameter structure for the network, providing accurate position and orientation scores for subsequent processing. This invention aims to improve the predicted position scores... and direction fraction Fitting the true positional scores with direction fraction Use in the attention module The loss function is defined as follows:

[0056] (5)

[0057] (6)

[0058] (7)

[0059] in, For the first ray, For the loss of positional scores, For the loss of direction fraction, This represents the total loss of the dual-attention ray scoring network.

[0060] This invention obtains the true position score of each ray from all images of the indoor scene in S1. with direction fraction The specific process is as follows:

[0061] The distance from the camera origin to the ray projection point is defined as ,when When the ray passes through the optical center, it indicates that the ray has passed through the optical center. Projection amount Calculated as ,in, For camera position, As the origin of the ray, The direction of the ray.

[0062] The angle between the camera's orientation and the ray direction is defined as... Calculate, where, The camera is facing.

[0063] The aforementioned distances and angles are mapped to attention map supervision scores, i.e., the true score for each ray is obtained through formulas (5) and (6):

[0064] (8)

[0065] (9)

[0066] in, The smoothing coefficient (scaling parameter) controls the steepness of the tanh function, representing the degree to which rays are assigned to the camera. The larger the value, the higher the tolerance of the dual-attention ray scoring network to bias, and the slower the weights decrease.

[0067] In addition, since the attention module includes a softmax operation, the true scores need to be normalized before the relevance calculation layer.

[0068] S5, Coarse Pose Estimation

[0069] Based on the position and direction scores of each ray output by S4, a Top-K filtering strategy is used to select... A high-confidence projected ray, For custom values. Based on... The projected ray is used to calculate the camera's approximate position and orientation using a weighted sum of direction vectors algorithm (Formula 11) combined with weighted least squares. The predicted position score... and direction fraction They will be used independently to select the frontmost relevant to position and orientation. There are at most one ray per ellipsoid. The camera position is then estimated using the intersections of these rays, and the problem can be formalized as a weighted least squares optimization. Since the rays generated by DARS-Net contain discretization noise, they often do not strictly converge at a single point; therefore, the camera position is solved by minimizing the sum of squared normal distances from the predicted camera position to the rays. For each ray... , Its error is defined as the prediction of the coarse position of the camera. and it The square of the distance between the orthogonal projections on:

[0070] (10)

[0071] in, Indicates the first The starting point of the ray, Indicates the first The direction vector corresponding to each ray. for The transpose of .

[0072] To minimize the above expression, Taking the derivative, we get:

[0073] (11)

[0074] in, It is the identity matrix. for transpose, For the first The position fraction predicted by the ray.

[0075] The camera's orientation is calculated as a negative weighted sum of the selected ray direction vectors, with weights derived from their predicted direction fractions. Decision. The calculated coarse orientation vector of the camera. Expressed as:

[0076] (12)

[0077] The normalization operation ensures that the calculated direction vector has a unit length.

[0078] S53. Generate a rendered view of the indoor environment based on the approximate position and orientation of the camera and the 3D Gaussian scene model.

[0079] S6, Pose Optimization

[0080] S5's coarse pose estimation relies on ray sampling, but due to the presence of noisy rays, even high-resolution rays may not accurately pass through the camera's optical center, resulting in inaccurate position and orientation estimations. This situation imposes a natural upper limit on the accuracy of pose estimation, thus necessitating further refinement. This invention utilizes a feature matching algorithm to establish a 2D-3D correspondence between the test monocular RGB image and the rendered view, obtaining the pose residual of the indoor service robot. The PnP algorithm is then used to minimize this pose residual, ultimately yielding a refined, high-precision 6-DOF indoor service robot pose. The specific process is as follows:

[0081] First, LoFTR (a Transformer-based feature matching method) is used to extract and match 2D feature points between the test monocular RGB image and the rendered view, obtaining a set of feature point correspondences. LoFTR can be replaced by other feature matchers. Next, the 2D feature points on the rendered view are back-projected onto the 3D Gaussian scene model using the depth information of 3D Gaussian primitives and camera intrinsics, thus forming a 2D feature point-3D feature point correspondence, which yields the pose residual of the indoor service robot. Finally, the PNP algorithm is used to minimize the pose residual of the indoor service robot, estimating a high-precision 6-DOF indoor service robot pose.

[0082] This invention may have other embodiments. Without departing from the spirit and essence of this invention, those skilled in the art can make various corresponding changes and modifications according to this invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

Claims

1. A robot pose estimation method based on 3D Gaussian spatter and dual-attention ray scoring network, characterized in that: It includes the following steps: S1. Use a robot to collect multi-view RGB image sequences of the indoor environment, and use motion recovery structure to process the multi-view RGB image sequences to obtain the camera pose and sparse point cloud of the indoor environment. S2. Based on the 3D Gaussian splashing method, each point in the sparse point cloud is initialized as a 3D Gaussian primitive. Each 3D Gaussian primitive is projected onto the camera's two-dimensional image plane for rendering according to the camera pose. The loss between the rendered image and the RGB image in S1 is calculated. The Gaussian properties are gradually updated using backpropagation to obtain a 3D Gaussian scene model. S3. Establish a dual-attention ray scoring network, which includes an enhanced MLP network, a DINOv2 backbone network, and an attention module. and attention module The enhanced MLP network is set up in parallel with the DINOv2 backbone network, and the attention module... and attention module The enhanced MLP network is set up in parallel with the attention module. and attention module The DINOv2 backbone network is connected to the attention module. and attention module connect; S4. Obtain the monocular RGB image of the indoor environment in S1. Input the monocular RGB image into the 3D Gaussian scene model to obtain the projected ray. Input the monocular RGB image and the projected ray into the dual attention ray scoring network and output the position score and direction score of the projected ray. S5. Based on position and orientation scores, a Top-K filtering strategy is used to obtain high-confidence rays. Based on the high-confidence rays, the weighted sum algorithm of orientation vectors combined with the weighted least squares method is used to solve the approximate position and orientation of the camera. Based on the approximate position and orientation of the camera and the 3D Gaussian scene model, a rendered view of the indoor environment is generated. S6. Use the feature matching algorithm to establish a correspondence between the monocular RGB image to be tested and the rendered view to obtain the robot's pose residual. Use the PnP algorithm to minimize the pose residual to obtain the 6-DOF robot pose.

2. The robot pose estimation method based on 3D Gaussian spatter and dual-attention ray scoring network according to claim 1, characterized in that: The probability density function of the 3D Gaussian unit in S2 for: （1） in, Let be the mean vector of 3D Gaussian elements. The covariance matrix of the 3D Gaussian elements. is the base of the natural logarithm. Let be the three-dimensional coordinate vector of any point in the indoor environment. for The transpose of .

3. The robot pose estimation method based on 3D Gaussian spatter and dual-attention ray scoring network according to claim 2, characterized in that: The specific process of S2 is as follows: S21. Based on the 3D Gaussian splashing method, each point in the sparse point cloud is initialized as a 3D Gaussian primitive; S22. Based on the camera pose, project each 3D Gaussian primitive onto the camera's 2D image plane for rendering. The specific process is as follows: (1) Based on the camera pose, use the view transformation matrix And the Jacobian matrix obtained by the affine approximation of the projection transformation The covariance matrix of each 3D Gaussian element in the camera coordinate system was calculated. : （2） (2) Based on the mean vector of the 3D Gaussian center Covariance matrix of each 3D Gaussian element The three-dimensional Gaussian distribution is obtained using formula (1). ; (3) A tile-based differentiable rasterizer is used to sort each 3D Gaussian primitive overlapping in the three-dimensional Gaussian distribution from shallow to deep. The weight of each 3D Gaussian primitive is determined according to the sorting result. The color of each 3D Gaussian primitive projected onto the pixel is calculated according to the weight, and the rendering result of each pixel is obtained. S23. Based on the rendering result of each pixel, obtain the rendered image, calculate the loss between the rendered image and the RGB image in S1, and use backpropagation to gradually update the Gaussian properties to obtain a 3D Gaussian scene model.

4. The robot pose estimation method based on 3D Gaussian spatter and dual-attention ray scoring network according to claim 3, characterized in that: The specific process in S22 (3) is as follows: The patch-based differential grating employs Hybrid strategy; The tile-based differentiable rasterizer first sorts each overlapping 3D Gaussian primitive in the 3D Gaussian distribution according to its depth from shallow to deep. Then, it determines the weight of each 3D Gaussian primitive based on the sorting result. Finally, it calculates the color projected onto a pixel by each 3D Gaussian primitive based on its weight. The blending strategy accumulates the color attribute of each pixel, and the resulting color after accumulation is... for: （3） in, The number of Gaussian pixels participating in the accumulation of each pixel. For the first The color of the high-sky element, , For the first The contribution of each Gaussian unit to the opacity Cumulative transmittance For the first The contribution of each Gaussian unit to the opacity; Obtain the rendering result for each pixel.

5. The robot pose estimation method based on 3D Gaussian spatter and dual-attention ray scoring network according to claim 4, characterized in that: The specific process of S23 is as follows: By continuously accumulating each pixel, a rendered image corresponding to the RGB image acquired by S1 is obtained, and the loss between the rendered image and the RGB image is continuously calculated. The Gaussian attributes are gradually updated through backpropagation to obtain a 3D Gaussian scene model.

6. The robot pose estimation method based on 3D Gaussian spatter and dual-attention ray scoring network according to claim 5, characterized in that: The enhanced MLP network in S3 includes a position encoding layer, a fully connected feature extraction layer, and a feature projection layer. The fully connected feature extraction layer is composed of multiple linear layers interspersed with nonlinear activation functions. The fully connected feature extraction layer uses residual connections or skip connections.

7. The robot pose estimation method based on 3D Gaussian spatter and dual-attention ray scoring network according to claim 6, characterized in that: The attention module in S3 and attention module Attention modules that use the same structure and have the same input and attention module Each layer consists of a fully connected layer, a correlation calculation layer, a normalization layer, and a Softmax layer, in that order.

8. The robot pose estimation method based on 3D Gaussian spatter and dual-attention ray scoring network according to claim 7, characterized in that: The specific process of S4 is as follows: S41. Obtain the monocular RGB image of the indoor environment in S1. Input the monocular RGB image into the 3D Gaussian scene model. The 3D Gaussian scene model generates multiple projection rays for each Gaussian ellipsoid and automatically selects the rays corresponding to the monocular RGB image. S42. Input the monocular RGB image to be tested and the rays corresponding to the monocular RGB image to be tested into the dual-attention ray scoring network, and output the position score and direction score of each ray. The specific process is as follows: S421. Input the monocular RGB image to be tested into the DINOv2 backbone network and output the image feature set. S422. The ray corresponding to the monocular RGB image to be tested is input into the position coding layer of the enhanced MLP network. The position coding layer maps each ray to a high-dimensional space using a sine function, cosine function or hash coding according to the original geometric parameters of each ray, and generates a high-frequency position embedding vector for each ray. The original geometric parameters are composed of the coordinate vectors of the ray origin and the ray direction. The high-frequency position of each ray is embedded into the vector input fully connected feature extraction layer, and the fused feature of each ray is output. The fusion features of each ray are input into the feature projection layer. The feature projection layer adjusts the fusion feature dimension of each ray to the preset feature dimension through linear transformation and outputs the feature set of each ray. S423. Input the feature set of each ray and the image feature set into the attention module. In the fully connected layer, ray features are used as queries and image features are used as keys. The ray features and image features are mapped to a unified attention dimension space to generate a set of query vectors and a set of key vectors. The query vector set and the key vector set are input into the relevance calculation layer, and matrix multiplication is performed on the query vector set and the key vector set to obtain multiple original attention maps with a size of "number of rays × number of image pixels". Multiple original attention maps are input into the normalization layer to obtain multiple normalized original attention maps; Multiple normalized original attention maps are input into the Softmax layer. For each normalized original attention map, a row-wise summation operation is performed, which accumulates the attention weight of a single ray relative to all pixels and outputs the direction score of each ray. The expression for the direction fraction is: （4） in, For direction fractions, For attention module The Middle 1 pixel, , Image width, Image height; S424. Input the feature set of each ray and the image feature set into the attention module. In the fully connected layer, ray features are used as queries and image features are used as keys. The ray features and image features are mapped to a unified attention dimension space to generate a set of query vectors and a set of key vectors. The query vector set and the key vector set are input into the relevance calculation layer, and matrix multiplication is performed on the query vector set and the key vector set to obtain multiple original attention maps with a size of "number of rays × number of image pixels"; Multiple original attention maps are input into the normalization layer to obtain multiple normalized original attention maps; Multiple normalized original attention maps are input into the Softmax layer. For each normalized original attention map, a row-wise summation operation is performed, which accumulates the attention weights of a single ray relative to all pixels and outputs the position score of each ray. The expression for the position fraction is: （5） in, For positional fractions, For attention module The Middle 1 pixel.

9. The robot pose estimation method based on 3D Gaussian spatter and dual-attention ray scoring network according to claim 8, characterized in that: The specific process of S5 is as follows: S51. Based on the position and direction scores of each ray, select using the Top-K screening strategy. A high-confidence ray; S52, according to For a high-confidence ray, the approximate position and orientation of the camera are determined using a weighted sum of direction vectors combined with weighted least squares. The specific process is as follows: The weighted sum algorithm for the direction vectors is as follows: （6） in, This is a rough estimate of the camera's position. , For the first The position fraction of the rays, It is the identity matrix. For the first The direction vector corresponding to each ray. for transpose, For the first The starting point of a ray; The approximate position of the camera is obtained by using a weighted least squares algorithm to solve for the weighted sum of the direction vectors. （7） in, For the camera's approximate orientation, For the first The direction fraction of a ray; S53. Generate a rendered view of the indoor environment based on the approximate position and orientation of the camera and the 3D Gaussian scene model.

10. The robot pose estimation method based on 3D Gaussian spatter and dual-attention ray scoring network according to claim 9, characterized in that: The specific process of S6 is as follows: S61. Use the LoFTR method to extract and match 2D feature points between the monocular RGB image under test and the rendered view; S62. Using the depth information of 3D Gaussian primitives and camera intrinsics, project the 2D feature points on the rendered view obtained in S61 onto the 3D Gaussian scene model to obtain the correspondence between 2D feature points and 3D feature points. Based on the correspondence between 2D feature points and 3D feature points, obtain the robot's pose residual. S63. The pose residual of the robot is minimized using the PNP algorithm to obtain the pose of the 6-DOF robot.

Images