A vision-enhanced 3D Gaussian SLAM method, model, computer system, and storage medium for humanoid robots.

Abstract

This application relates to the field of artificial intelligence technology, and in particular to a vision-enhanced 3D Gaussian SLAM method, model, computer system, and storage medium for humanoid robots. An initial binary mask is generated by instance segmentation of acquired RGB-depth images, and then optimized using composite morphological operations. High-confidence static feature points obtained from a dynamic mask with precise boundaries are used to estimate the robot's pose. A keyframe selection strategy is designed based on the humanoid robot's motion model, dynamically adjusting candidate intervals according to changes in the robot's motion state, and selecting the optimal keyframe through weighted selection within each interval. A multi-scale optimization mechanism fusing Gaussian-Laplacian pyramids and screen space errors is proposed, preserving high-frequency details through Laplacian residual compensation and achieving adaptive splitting of the Gaussian ellipsoid by combining a hybrid error metric. The aim is to address the problem of improving the localization accuracy of humanoid robots in complex environments.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a vision-enhanced 3D Gaussian SLAM method, model, computer system and storage medium for humanoid robots. Background Technology

[0002] Visual SLAM (vSLAM), a core technology in intelligent robotics and autonomous driving, aims to achieve real-time localization and high-precision scene reconstruction of intelligent agents in unknown environments based on streaming data provided by visual sensors. Traditional vSLAM algorithms generally assume that the environment is static. While this premise facilitated the efficient development of early SLAM systems, it severely limited their applicability in real-world scenarios due to their inability to adapt to dynamic targets with random walks.

[0003] Among the relevant technical solutions, 3D Gaussian Splatting (3DGS) technology has been proposed. It achieves a balance between real-time photorealistic rendering and efficient parameter optimization through explicit modeling of anisotropic Gaussian ellipsoids in space, thereby promoting the development of 3DGS-SLAM systems.

[0004] However, current 3D GS-SLAM schemes cannot accurately identify complex, dynamic targets in dynamic scenes. Therefore, this application proposes a novel 3D Gaussian SLAM method for humanoid robots, aiming to improve the localization accuracy of humanoid robots in dynamic crowd environments while simultaneously enhancing the quality of keyframe selection. Summary of the Invention

[0005] The main objective of this application is to provide a vision-enhanced 3D Gaussian SLAM method for humanoid robots, aiming to solve the problem of how to improve the positioning accuracy of humanoid robots in complex environments.

[0006] To achieve the above objectives, this application provides a vision-enhanced 3D Gaussian SLAM method for humanoid robots, the method comprising: S10, acquire the RGB-depth image collected by the humanoid robot; S20, the RGB-depth image is segmented to obtain a binary mask and then subjected to composite morphological optimization to obtain an optimized binary mask; S30, Select a set of valid feature points from the optimized binary mask, and obtain the current pose of the humanoid robot based on the set of valid feature points; S40, determine the state interval based on the current pose, and filter out the keyframe set from the state interval; S50, the keyframe set is input into a pre-trained deep model to extract Gaussian ellipsoid features, wherein the pre-trained deep model samples the features of each keyframe through a Gaussian pyramid and restores the sampled features of the Gaussian pyramid through a Laplacian pyramid; S60, calculate the Laplacian residual and screen space error of the Gaussian ellipsoid feature, and based on the Laplacian residual and screen space error, determine the target Gaussian ellipsoid feature that needs to be split and the splitting direction, so as to iteratively split the target Gaussian ellipsoid feature according to the splitting direction.

[0007] Optionally, the S20 includes: S21, using YOLOv11 to process the input RGB-depth image frame Instance segmentation is performed, and a binary mask is generated through thresholding:

[0008] S22, adopts Elliptical structural elements ,right Perform a closing operation, then use Elliptical structural elements The mask is expanded, that is:

[0009] in, This represents the closing operation, used to fill small holes inside a mask; This represents the dilation operation, used to expand the boundaries of foreground pixels to compensate for undersegmentation of edges in instance segmentation; S23, using a two-dimensional Gaussian check Perform convolution, then use Gaussian smoothing to eliminate jagged edge artifacts, generating a smoothed grayscale mask:

[0010] in, For convolution operations, The standard deviation is the Gaussian kernel. S24, the smoothed grayscale mask Perform binarization again to generate an optimized binary mask:

[0011] in, Represents pixels This refers to an invalid area that is dynamic or occluded. Represents pixels This refers to a static, unobstructed effective area.

[0012] Optionally, S30 includes: S31, from optimizing binary masks Select the effective feature point set from the data:

[0013] S32, Construct a function that minimizes the sum of squares of the projection error:

[0014] in, A camera projection function that projects 3D points onto 2D pixels; Represents valid points in the image The corresponding 3D point in the world coordinate system; R is the rotation matrix; t is the translation vector; S33, Constraint Matrix Based on Humanoid Robot Pose Vector Solve for all valid points Corresponding projection results The current pose of the humanoid robot is obtained by finding R and t that minimize the sum of squared errors from the actual effective points. ,in: .

[0015] Optionally, S40 includes: S41, Determine the change in motion pose of the humanoid robot based on the current pose. :

[0016] set up The upper and lower bounds are respectively and , and the interval The input ordinary frames are divided into candidate intervals; S42, determine the state interval of the target frame that is in the candidate interval. :

[0017] In the formula, This represents the state interval of the previous state after scaling. For interval Kalman gain, The interval for measuring residuals; in:

[0018]

[0019]

[0020] In the formula, It is a measurement matrix. It is the interval covariance matrix of the measurement residuals. For observation models, For interval midpoint, This is the scaling factor. This is the state interval of the previous state; S43, define the state interval The set of all candidate frames F is:

[0021] Filter the set according to at least one of the following rules Keyframes in: Rule 1: If If the data contains only one frame, then that frame is directly selected as the keyframe. Rule 2: If Then calculate the weights of each candidate frame F:

[0022] in, Pose information of candidate frame F; State interval The intermediate reference pose; It is a factor that controls the scale of weight changes; among them, the number of visible static map points in the candidate frame is greater than a preset threshold. ; Based on the weights of each candidate frame F Select keyframe W:

[0023] In the formula, s is the stability index of the robot during movement. This represents the number of visible static map points within the frame. in,

[0024] In the formula, Represents the joint torque vector of the robot Maximum torque modulus Indicates the maximum difference in force between the left and right feet The model, This is the adjustment coefficient; Rule 3: When the robot is stationary or moves a distance less than a preset distance threshold, if the time interval between the current frame and the previous keyframe exceeds the preset threshold, the current frame is directly selected as the keyframe.

[0025] Optionally, in S50, features of each keyframe are sampled using a Gaussian pyramid, and the sampled features of the Gaussian pyramid are restored using a Laplacian pyramid, including: S51, for Gaussian pyramid images of each level are generated by downsampling a Gaussian pyramid with a four-layer structure:

[0026] in, This indicates downsampling of the k-th Gaussian pyramid of the image, where s is the sampling factor; S52, calculate the residuals between two adjacent Gaussian pyramid images, and use them as keyframes. The k-th level of the Laplace Pyramid :

[0027] In the formula, This is an upsampling operation; S53, in the image reconstruction stage, the Gaussian pyramid image is recovered layer by layer using the Laplacian pyramid to reconstruct keyframes containing Gaussian ellipsoid features. : .

[0028] Optionally, the training function expression of the pre-trained deep model during the training process is:

[0029] In the formula, and These are the training error losses for the Gaussian pyramid and the Laplacian pyramid, respectively. Dynamic weights; The pre-trained deep model is trained on the highest-level Gaussian pyramid image during the training process. The optimization begins in the middle, where the expression for the objective function is:

[0030] In the formula, Indicates the first The rendered image of the layer.

[0031] Optionally, the S60 includes: S61, Calculate the Laplacian residual of the Gaussian ellipsoid feature in the l-th training layer. Screen Space Error (SSE):

[0032]

[0033] In the formula, For pixels In the High-frequency energy intensity of the layer, For the maximum high-frequency energy intensity, For a constant term, This represents the projection transformation operator composed of both camera intrinsic and extrinsic parameters; Wherein, the high-frequency energy intensity in the current level l is greater than or equal to a preset threshold. Laplace energy, preset threshold The expression is:

[0034] In the formula, Let l be the mean of the Laplace energy of the current level. The coefficient is used to control the threshold range. The standard deviation of the Laplace energy at the current level; S62, based on Laplace residuals And screen space error SSE, determine the mixed error weights :

[0035] In the formula, This represents the maximum value of the screen space error; S63, based on the mixed error weights For the original gradient threshold and screen space error threshold Adjustments will be made:

[0036] In the formula, For adjustment Sensitivity coefficient to the effect of the threshold; S64 will be greater than the gradient threshold. and screen space error threshold The Gaussian ellipsoid features are identified as the target Gaussian ellipsoid features. S65, iteratively split the target Gaussian ellipsoid feature, wherein the splitting direction is... Satisfy the following expression:

[0037] in, The 3D coordinates of the original Gaussian ellipsoid. The gradient of the mixed error weights, For the screen space error gradient, For random disturbance terms, These are the gradient coefficients.

[0038] Furthermore, to achieve the above objectives, this application also provides a humanoid robot 3D Gaussian SLAM model, which includes: The image acquisition module is used to acquire RGB-depth images captured by the humanoid robot; The dynamic mask acquisition and optimization module is used to perform instance segmentation on the RGB-depth image to obtain a binary mask and perform composite morphological optimization to obtain an optimized binary mask. The pose estimation module is used to filter out the effective feature point set from the optimized binary mask and obtain the current pose of the humanoid robot based on the effective feature point set. The keyframe selection module is used to determine a state interval based on the current pose and filter a set of keyframes from the state interval. The image training and feature extraction module is used to input the keyframe set into a pre-trained deep model to extract Gaussian ellipsoid features. In the pre-trained deep model, features of each keyframe are sampled through a Gaussian pyramid, and the sampled features of the Gaussian pyramid are restored through a Laplacian pyramid. The 3D Gaussian rendering and reconstruction module is used to calculate the Laplacian residual and screen space error of the Gaussian ellipsoid feature. Based on the Laplacian residual and screen space error, it determines the target Gaussian ellipsoid feature that needs to be split and the splitting direction, so as to iteratively split the target Gaussian ellipsoid feature according to the splitting direction.

[0039] In addition, to achieve the above objectives, this application also provides a computer system comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the vision-enhanced humanoid robot 3D Gaussian SLAM method as described in any of the preceding claims.

[0040] In addition, to achieve the above objectives, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the vision-enhanced humanoid robot 3D Gaussian SLAM method as described in any of the preceding claims.

[0041] This application has at least the following beneficial effects: 1. An initial binary mask is generated by instance segmentation of the acquired RGB-depth image. Then, the mask is optimized by combining compound morphological operations. The robot's pose is estimated by using high-confidence static feature points obtained by the dynamic mask with accurate boundaries. 2. To ensure stable pose tracking and efficient mapping of the robot, a keyframe selection strategy was designed based on the motion model of the humanoid robot. The candidate interval was dynamically adjusted according to the changes in the robot's motion state, and the optimal keyframe was selected by weighted selection within the interval. 3. Considering the difficulty in balancing efficiency and geometric detail preservation in 3D Gaussian modeling, a multi-scale optimization mechanism integrating Gaussian-Laplace pyramid and screen space error is proposed. High-frequency details are preserved through Laplace residual compensation, and adaptive splitting of the Gaussian ellipsoid is achieved by combining mixed error metrics. Attached Figure Description

[0042] Figure 1 This is a flowchart illustrating the vision-enhanced humanoid robot 3D Gaussian SLAM method described in the embodiments of this application. Figure 2 This is a comparison chart of the trajectory errors of ORB-SLAM3 on the TUM dataset sequence according to the embodiments of this application; Figure 3 This is a comparison chart of the trajectory error of the method (i.e., the algorithm in this paper) in the TUM dataset sequence. Figure 4 This is a comparison chart of the trajectory error of ORB-SLAM3 on the BONN dataset sequence in the embodiments of this application; Figure 5 A comparison chart of trajectory errors of the method (i.e., the algorithm in this paper) in the embodiments of this application on the BONN dataset sequence; Figure 6 These are trajectory maps generated by Photo-SLAM in real-world scenes using different algorithms involved in the embodiments of this application. Figure 7 These are ATE line graphs generated by Photo-SLAM in real-world scenes using different algorithms involved in the embodiments of this application. Figure 8 This is a comparison of the rendering effects of the conventional algorithm involved in the embodiments of this application using Photo-SLAM at the center coordinates of all Gaussian ellipsoids in a real scene; Figure 9 Comparison of the rendering effects of the proposed method in Photo-SLAM at the center coordinates of all Gaussian ellipsoids in a real scene; Figure 10 This is a schematic diagram of the architecture of the humanoid robot 3D Gaussian SLAM model involved in the embodiments of this application; Figure 11 This is a schematic diagram of the hardware operating environment of the computer system involved in the embodiments of this application.

[0043] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0044] To better understand the above technical solutions, exemplary embodiments of this disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of this disclosure to those skilled in the art.

[0045] First Embodiment In this embodiment, kinematic modeling of the humanoid robot is first performed. Firstly, the pose and velocity of the floating reference point relate to the robot's overall motion and control, while generalized coordinates and generalized velocity describe the robot's internal degrees of freedom and joint motions. The three key motion vectors are represented as follows: (1) (2) (3) in, This represents the robot's pose vector, including the pose of the floating base. and generalized coordinates ; This represents the robot's velocity vector, including the velocity of the floating reference. and generalized speed ; This represents the robot's acceleration vector, including the acceleration of the floating reference. and generalized acceleration .

[0046] Furthermore, the dynamic equations of the humanoid robot can be specified as follows: (4) in, These represent the external forces acting on the left and right feet, respectively. Each external force is six-dimensional, representing force and torque in 3D space. These are the Jacobian matrices for the left and right feet, respectively, which transform these external forces into the system's generalized coordinate space; This represents the inertial effect of the system, reflecting how the robot's acceleration is affected by the mass distribution; The speed of the robot The resulting Coriolis force and centrifugal force; This indicates the effect of gravity on different parts of the robot. The torque generated by the joint actuator is controlled by selecting a matrix. These torques are mapped onto the system's generalized coordinates.

[0047] Furthermore, the sensors configured on humanoid robots often cannot fully cover all state dimensions, so motion constraints are needed to compensate for the shortcomings of hardware measurements. Based on equation (4), the robot dynamics equations containing constraints can be further derived as follows: (5) In the formula, Indicates the constraint term. It is the Lagrange multiplier, representing the strength of the constraint force.

[0048] By solving the constraint forces From expression (5), we obtain the constraint force matrix that depends on the robot pose vector. Its relationship with its speed is as follows: (6) After completing the above kinematic modeling, this embodiment implements a vision-enhanced 3D Gaussian SLAM method for humanoid robots, including the following steps: S10, acquire the RGB-depth image collected by the humanoid robot; First, use an RGB-D camera to acquire RGB-D images of the surrounding environment; S20, the RGB-depth image is segmented to obtain a binary mask and then subjected to composite morphological optimization to obtain an optimized binary mask; Furthermore, in this embodiment, the pedestrians in the image are segmented using the lightweight object detection network YOLOv11 to generate dynamic masks, and the dynamic masks segmented by YOLOv11 are optimized using compound morphological operations. Specifically, the following steps are included: S21, using YOLOv11 to process the input RGB-depth image frame Instance segmentation is performed, and a binary mask is generated through thresholding: (7) S22, adopts Elliptical structural elements ,right Perform a closing operation, then use Elliptical structural elements The mask is expanded, that is: (8) in, This represents the closing operation, used to fill small holes inside a mask; This represents the dilation operation, used to expand the boundaries of foreground pixels and compensate for undersegmentation of edges in instance segmentation. S23, using a two-dimensional Gaussian check Perform convolution, then use Gaussian smoothing to eliminate jagged edge artifacts, generating a smoothed grayscale mask: (9) in, For convolution operations, The standard deviation is the Gaussian kernel. It should be noted that the purpose of S23 is to eliminate jagged artifacts at the edges through Gaussian smoothing, thereby achieving a natural transition at the mask edges.

[0049] S24, the smoothed grayscale mask Perform binarization again to generate an optimized binary mask: (10) in, Represents pixels This refers to an invalid area that is dynamic or occluded. Represents pixels This refers to a static, unobstructed effective area.

[0050] S30, Select a set of valid feature points from the optimized binary mask, and obtain the current pose of the humanoid robot based on the set of valid feature points; After obtaining the optimized binary mask, in step S30, the robot pose is estimated from the masked image to obtain high-confidence static features and achieve accurate pose estimation.

[0051] Specifically, S30 includes: S31, from optimizing binary masks Select the effective feature point set from the data: (11) S32, Construct a function that minimizes the sum of squares of the projection error: (12) in, A camera projection function that projects 3D points onto 2D pixels; Represents valid points in the image The corresponding 3D point in the world coordinate system; R is the rotation matrix; t is the translation vector; S33, Constraint Matrix Based on Humanoid Robot Pose Vector Solve for all valid points Corresponding projection results The current pose of the humanoid robot is obtained by finding R and t that minimize the sum of squared errors from the actual effective points. .

[0052] S40, determine the state interval based on the current pose, and filter out the keyframe set from the state interval; After obtaining the current pose of the humanoid robot, this embodiment creatively designs a keyframe selection strategy based on a scaling interval Kalman filter of a floating reference robot motion model. By dynamically dividing the robot's motion interval, the uncertainty of the pose is quantified, and a scaling factor is introduced to adaptively adjust the interval width, thereby filtering out high-quality keyframes and providing a guarantee for stable pose tracking and efficient mapping.

[0053] First, before proceeding with this step, this embodiment will explain the process of constructing the humanoid robot state transition equations required: Based on the motion vectors of the humanoid robot mentioned earlier, its state vector is defined as follows: Furthermore, combining with expression (4), its continuous-time state equation can be obtained as follows: (14) Furthermore, assuming the robot is stationary at time t=0, its state and input are as follows: and And the torque generated by its joint actuator =0, based on the weight of the humanoid robot, the external forces acting on the robot's left and right feet are: (15) Furthermore, at time k, the first-order Taylor expansion of expression (14) is as follows: (16) The state Jacobian matrix, control input matrix, and contact force Jacobian matrix are represented as follows: (17) (18) (19) (20) (twenty one) (twenty two) Furthermore, the constrained observations are constructed based on expression (6) as follows: (twenty three) Furthermore, the constraint measurement Jacobian matrix is: (twenty four) Furthermore, the constraint covariance matrix is ​​defined as follows: (25) in, As a unit array, It is a very small positive number in this invention. Take 1e-4.

[0054] Furthermore, according to expression (16), the discrete-time state transition equation for robot motion can be written as: (26) In the formula, It is Gaussian process noise.

[0055] Furthermore, performing forward Euler discretization on equation (26) yields: (27) Furthermore, construct the discrete input matrix: (28) (29) (30) After completing the above construction, the state transition equation of the humanoid robot is completed.

[0056] On the other hand, the construction of the scaling interval Kalman keyframe selection strategy is explained: First, define the robot's motion pose change. as follows: (31) set up The upper and lower bounds are respectively and , and the interval The input ordinary frames are divided into the same interval. In this way, in this embodiment, ordinary frames that meet the motion feature constraints are initially classified into the same candidate interval, thus defining the interval range for keyframe screening.

[0057] Furthermore, at time t=0, the joint state interval vector of the humanoid robot combined with the input is represented as: (32) Furthermore, let the initial state covariance matrix be: (33) in, It is the covariance matrix of the initial pose; It is the initial velocity covariance matrix; It is the initial joint torque covariance matrix; and These represent the initial force covariance matrices for the left and right feet, respectively.

[0058] The initial state covariance matrix is: (34) The observation model is: (35) in, It measures noise. It is a measurement matrix, and its form is as follows: (36) in, It is the IMU measurement submatrix. It is the joint encoder measurement sub-matrix. It is a visual measurement submatrix. It is a foot force sensor measurement sub-matrix. It is a constraint measurement submatrix.

[0059] Measuring noise interval covariance The format is as follows: (37) in, It is the IMU noise range; This is the noise range of the joint encoder; This is the visual measurement noise range; This is the noise range of the foot force sensor; It is a constraint measurement noise range.

[0060] Furthermore, during the prediction phase, the interval state transition equations for the humanoid robot are as follows: (38) (39) Process noise Represented as: (40) in, Noise during the floating reference pose process; Noise is generated during the joint angle process. This refers to the noise of the generalized velocity process.

[0061] Furthermore, during the scaling-down phase: The scaling factor is constructed as follows: (41) in, Interval estimation of current measurement uncertainty Interval estimation of baseline uncertainty The interval of the measurement time, The range of the reference time interval Uncertainty weighting coefficients. Furthermore, the following scaling is applied to the state and covariance: (42) (43) in, For interval midpoint.

[0062] Furthermore, during the update phase: Calculate the interval for measurement residuals: (44) Calculate the interval covariance matrix of the measurement residuals: (45) Calculate the interval Kalman gain: (46) Update interval state estimation: (47) Update interval covariance estimates: (48) After the above strategy is constructed, when executing step S40, the necessary functions can be called to complete the determination of the state range and the filtering of the key frame set.

[0063] Specifically, step S40 includes: S41, Determine the change in motion pose of the humanoid robot based on the current pose. :

[0064] set up The upper and lower bounds are respectively and , and the interval The input ordinary frames are divided into candidate intervals; S42, determine the state interval of the target frame that is in the candidate interval. :

[0065] In the formula, This represents the state interval of the previous state after scaling. For interval Kalman gain, The interval for measuring residuals; in:

[0066]

[0067]

[0068] In the formula, It is a measurement matrix. It is the interval covariance matrix of the measurement residuals. For observation models, For interval midpoint, This is the scaling factor. This is the state interval of the previous state; S43, define the state interval The set of all candidate frames F is: (49) Filter the set according to at least one of the following rules Keyframes in: Rule 1: If If the data contains only one frame, then that frame is directly selected as the keyframe. Rule 2: If Then calculate the weights of each candidate frame F: (50) in, Pose information of candidate frame F; State interval The intermediate reference pose; It is a factor that controls the scale of weight changes; among them, the number of visible static map points in the candidate frame is greater than a preset threshold. ; In some alternative implementations, The value is 300.

[0069] Based on the weights of each candidate frame F Select keyframe W: (51) In the formula, s is the stability index of the robot during movement. This represents the number of visible static map points within the frame. in, (52) In the formula, Represents the joint torque vector of the robot Maximum torque modulus Indicates the maximum difference in force between the left and right feet The model, This is the adjustment coefficient; Rule 3: When the robot is stationary or moves a distance less than a preset distance threshold, if the time interval between the current frame and the previous keyframe exceeds the preset threshold, the current frame is directly selected as the keyframe.

[0070] S50, the keyframe set is input into a pre-trained deep model to extract Gaussian ellipsoid features, wherein the pre-trained deep model samples the features of each keyframe through a Gaussian pyramid and restores the sampled features of the Gaussian pyramid through a Laplacian pyramid; S60, calculate the Laplacian residual and screen space error of the Gaussian ellipsoid feature, and based on the Laplacian residual and screen space error, determine the target Gaussian ellipsoid feature that needs to be split and the splitting direction, so as to iteratively split the target Gaussian ellipsoid feature according to the splitting direction.

[0071] After completing the keyframe selection and obtaining the keyframe set, steps S50 and S60 further extract Gaussian ellipsoid features and perform iterative splitting on the keyframe set. By combining 3D Gaussian splashing technology, this embodiment creatively proposes a mapping strategy that integrates Gaussian-Laplace pyramid and Gaussian ellipsoid adaptive density control.

[0072] It should be noted that in some implementation schemes, steps S50 and S60 are executed by a single module to implement the above-mentioned mapping strategy. However, for the sake of ease of description and understanding, this embodiment will describe them as S50 and S60.

[0073] In step S50, the keyframe set is input into a pre-trained deep model to extract Gaussian ellipsoid features. Compared to traditional general-purpose deep models, the pre-trained deep model proposed in this embodiment introduces a Laplacian pyramid to construct a multi-scale training structure for keyframes, building upon the Gaussian pyramid, thus achieving accurate preservation of high-frequency details. This provides crucial guidance for subsequent optimization of the 3D Gaussian ellipsoid density.

[0074] Step S50 specifically includes: S51, for Gaussian pyramid images at each level are generated by downsampling using a four-layer Gaussian pyramid structure to achieve image noise filtering and initial feature extraction. (53) in, This indicates downsampling of the k-th Gaussian pyramid of the image, where s is the sampling factor; In some alternative implementations, considering that traditional downsampling can easily lead to blurred edges of dynamic targets in complex dynamic scenes, which in turn can cause 3D reconstruction errors, in order to provide higher quality multi-scale input and adapt to complex dynamic scenes, .

[0075] S52, calculate the residuals between two adjacent Gaussian pyramid images, and use them as keyframes. The k-th level of the Laplace Pyramid : (54) In the formula, This is an upsampling operation; In some alternative implementations, upsampling is achieved using bilinear interpolation.

[0076] S53, in the image reconstruction stage, the Gaussian pyramid image is recovered layer by layer using the Laplacian pyramid to reconstruct keyframes containing Gaussian ellipsoid features. : (55) Furthermore, and optionally, in order to effectively preserve high-frequency details in keyframes while taking into account the stability of map optimization, this embodiment constructs the following loss function by combining the Laplacian pyramid and the Gaussian pyramid, and achieves progressive training through dynamic weight adjustment.

[0077] (57) in, and These are the training error losses for the Gaussian pyramid and the Laplacian pyramid, respectively. For dynamic weights.

[0078] In some alternative implementations, The initial value is 0.9, and it increases as training progresses. The weights are gradually decreased to progressively increase the contribution of the Laplacian pyramid. After calculating the loss, the gradient is calculated via backpropagation to update the weights of the trained model. The optimizer then adjusts the network parameters based on the gradient of the loss function to reduce the errors in the Gaussian and Laplacian pyramids.

[0079] exist At any given moment, the training process relies on the highest-level Gaussian pyramid image. Initiate the optimization process, with the objective function defined as follows: (58) in, Indicates the first The rendered image of the layer.

[0080] After extracting the Gaussian ellipsoid features in step S50, in step S60, the mixed error weights of the high-frequency regions in the keyframe are calculated based on the Laplacian residual and SSE, and high-frequency and high-error candidate regions, i.e., the target Gaussian ellipsoid features, are determined. The target Gaussian ellipsoid features are then subjected to directionally controllable iterative splitting. The advantage of this approach is that it not only identifies the Gaussian ellipsoids to be split by preserving the high-frequency details of the keyframes, but also controls the splitting direction of the Gaussian ellipsoids through gradient control, thereby reducing Gaussian ellipsoid redundancy and improving the mapping quality of the humanoid robot.

[0081] Step S60 specifically includes: S61, Calculate the Laplacian residual of the Gaussian ellipsoid feature in the l-th training layer. Screen Space Error (SSE): (59) (60) In the formula, For pixels In the High-frequency energy intensity of the layer, For the maximum high-frequency energy intensity, For a constant term, This represents the projection transformation operator composed of both camera intrinsic and extrinsic parameters; Wherein, the high-frequency energy intensity in the current level l is greater than or equal to a preset threshold. Laplace energy, preset threshold The expression is: (61) In the formula, Let l be the mean of the Laplace energy of the current level. The coefficient is used to control the threshold range. The standard deviation of the Laplace energy at the current level; It should be noted that the Laplacian residual calculated in S61 directly reflects the lost static high-frequency information in dynamic scenes. By performing multi-channel energy statistics on keyframes, the high-frequency feature intensity of each region can be quantified, marking "static detail regions worth optimizing" for 3DGS SLAM. Therefore, in order to quantify the Laplacian energy, the current training layer... Multi-channel energy statistics were performed on the Laplace residuals to reflect the high-frequency energy distribution of the layer.

[0082] In addition, to avoid the fixed threshold being insufficiently adaptable to different scenarios, this embodiment incorporates the Laplace energy of the current level. mean with standard deviation Determine the dynamic threshold in the high-frequency region That is, formula (61).

[0083] In some alternative implementations, .

[0084] S62, based on Laplace residuals And screen space error SSE, determine the mixed error weights : (62) In the formula, This represents the maximum value of the screen space error; It should be noted that in S62, the visual distortion information characterized by SSE is further integrated to calculate the combined error weight, in order to balance the need for high-frequency detail feature preservation and visual quality optimization. Additionally, in equation (62), It can highlight high-frequency regions (areas rich in static details) while also taking into account high-error regions (areas with significant visual distortion), forming a Gaussian ellipsoid splitting priority based on high-frequency features and visual quality, providing guidance for subsequent 3D Gaussian ellipsoid optimization.

[0085] S63, based on the mixed error weights For the original gradient threshold and screen space error threshold Adjustments will be made: (63) In the formula, For adjustment Sensitivity coefficient to the effect of the threshold; In some alternative implementations, .

[0086] S64 will be greater than the gradient threshold. and screen space error threshold The Gaussian ellipsoid features are identified as the target Gaussian ellipsoid features. S65, iteratively split the target Gaussian ellipsoid feature, wherein the splitting direction is... Satisfy the following expression: (64) in, The 3D coordinates of the original Gaussian ellipsoid. The gradient of the mixed error weights, For the screen space error gradient, For random disturbance terms, These are the gradient coefficients.

[0087] In some alternative implementations, .

[0088] In the technical solution provided in this embodiment, on the one hand, an initial binary mask is generated by instance segmentation of the acquired RGB-depth image, and then the mask is optimized by combining compound morphological operations. The high-confidence static feature points obtained by the dynamic mask with precise boundaries are used to realize the robot's pose estimation. On the other hand, in order to ensure stable pose tracking and efficient mapping of the robot, a keyframe selection strategy is designed based on the motion model of the humanoid robot. The candidate interval is dynamically adjusted according to the changes in the robot's motion state, and the optimal keyframe is selected by weighted filtering within the interval. Furthermore, considering the difficulty in balancing efficiency and geometric detail preservation in 3D Gaussian modeling, a multi-scale optimization mechanism that integrates Gaussian-Laplacian pyramid and screen space error is proposed. High-frequency details are preserved through Laplacian residual compensation, and adaptive splitting of the Gaussian ellipsoid is achieved by combining a hybrid error metric.

[0089] Verification Implementation Examples To verify the effectiveness of the method involved in the embodiments of this application, in this embodiment, simulation experiments were conducted on the TUM and BONN datasets and compared with ORB-SLAM3.

[0090] Simulation experiments were conducted using dynamic sequences fr3 / walking_static, fr3 / walking_xyz, fr3 / walking_halfsphere, and fr3 / walking_rpy from the TUM dataset, and rgbd_bonn_balloon, rgbd_bonn_balloon2, rgbd_bonn_balloon_tracking, rgbd_bonn_person_tracking, and rgbd_bonn_person_tracking2 from the BONN dataset. Different sequences correspond to different camera motion modes and scene structures. Absolute Trajectory Error (ATE) is typically used as an evaluation metric for system accuracy. ATE directly calculates the error between the system's true pose and estimated pose, reflecting the system's global positioning accuracy and used to evaluate the performance of the SLAM system.

[0091] Experimental platform: Operating system is Ubuntu 22.04, processor is i9-12900H.

[0092] Table 1. Comparison of RMSE of absolute trajectory error in TUM dataset

[0093] Table 2. Comparison of RMSE of absolute trajectory error in the BONN dataset

[0094] Table 1 compares the RMSE of the absolute trajectory error between the method of the present invention and ORB-SLAM3 on the TUM dataset, and Table 2 compares the RMSE of the absolute trajectory error between the method of the present invention and ORB-SLAM3 on the BONN dataset. As can be seen from the tables, the method of the present invention reduces the trajectory drift of the system by removing feature points of dynamic objects, effectively improving the positioning accuracy of the system in dynamic scenes.

[0095] On the other hand, refer to Figures 2 to 5 The absolute trajectory error diagrams of the proposed method and ORB-SLAM3 on the TUM and BONN datasets are shown respectively. Figure 2 and Figure 4 The data clearly shows that in sequences with high dynamic disturbances such as pedestrian movement and object movement, ORB-SLAM3 trajectory estimation is prone to significant drift, with a wide error distribution range and multiple anomalous peaks; in contrast, Figure 3 and Figure 5 The trajectory error obtained by the algorithm in this paper is smaller overall and more concentrated, as shown in the figure, and it can still remain stable in scenarios with large movements and complex occlusion.

[0096] On the other hand, although GARAD-SLAM extends its applicability to dynamic environments and introduces a dynamic point filtering mechanism based on Gaussian distributed conditional random fields (CRF) to recover Gaussian labels using sparse optical flow feature points, its background reconstruction effect in areas occluded by dynamic objects is still unsatisfactory, with a large amount of noise in the image details. This is mainly because the smoothness assumption on which CRF relies is difficult to adapt to the common non-rigid motion and deformation of dynamic objects, causing the system to be unable to effectively correct the local label errors recovered by sparse optical flow while removing dynamic objects, ultimately introducing a large number of "isolated noise points". In contrast, the method in this application generates and optimizes masks frame by frame in the front-end thread, and combined with a multi-threaded architecture, performs dynamic object removal synchronously in the tracking and mapping threads, thereby minimizing the consistency deviation between pose estimation and map construction. In addition, by introducing the Laplacian pyramid to guide the adaptive splitting process of the Gaussian ellipsoid, the method in this paper further improves the rendering quality and image sharpness, generating more complete, accurate and structurally clear object details. In scenes containing only human movement and no passively moving objects (such as bonn_crowd and bonn_synchronous2), the method in this application performs similarly to GARAD-SLAM in terms of human culling. However, in terms of background reconstruction, the method in this application provides lighting and color restoration that is closer to the original input image, showing better reconstruction fidelity.

[0097] On the other hand, refer to Figure 6 The illustrated trajectory diagrams generated by Photo-SLAM in a real-world scene using different algorithms show that Photo-SLAM exhibits significant offsets due to interference from dynamic objects, while the trajectory generated by the method in this embodiment fits the reference trajectory very well, demonstrating global consistency far exceeding that of Photo-SLAM. Furthermore, the rendering effect of a randomly selected path point reveals that even with pedestrian interference, the method in this application maintains high realism in rendering quality and demonstrates excellent repair performance for occluded background areas.

[0098] On the other hand, refer to Figure 7 The illustrated ATE line graphs of different algorithms generated by Photo-SLAM in a real-world scene show that the trajectory error curve of Photo-SLAM exhibits drastic fluctuations, especially in the 80-100 second period, where the maximum error exceeds 3m. This is because the humanoid robot is rotating and is strongly disturbed by pedestrians in the scene. In contrast, the error curve of this method remains at a relatively low level almost throughout, with a maximum error of only 0.8m. This demonstrates that the algorithm in this embodiment possesses excellent anti-interference capabilities and high-precision error control.

[0099] On the other hand, refer to Figure 8 and Figure 9 The comparison of the rendering effects of different algorithms using Photo-SLAM at the center coordinates of all Gaussian ellipsoids in a real scene shows that, in realistic dynamic crowd scenes, Photo-SLAM, due to its lack of dynamic feature culling, suffers from severe error accumulation in pose estimation. This further leads to distortion of the overall geometric structure of the 3D Gaussian model and the formation of numerous dense, redundant Gaussian ellipsoids in the pedestrian area, severely impacting the accuracy of the mapping. However, the algorithm in this embodiment, through accurate dynamic feature detection and pose robust optimization, effectively suppresses the impact of dynamic interference on pose estimation. The constructed 3D Gaussian model not only closely matches the geometric shape of the real scene but also achieves a more visually realistic global rendering effect.

[0100] In addition, refer to Figure 10 The embodiments of this application also relate to a humanoid robot 3D Gaussian SLAM model, the humanoid robot 3D Gaussian SLAM model comprising: The image acquisition module is used to acquire RGB-depth images captured by the humanoid robot; The dynamic mask acquisition and optimization module is used to perform instance segmentation on the RGB-depth image to obtain a binary mask and perform composite morphological optimization to obtain an optimized binary mask. The pose estimation module is used to filter out the effective feature point set from the optimized binary mask and obtain the current pose of the humanoid robot based on the effective feature point set. The keyframe selection module is used to determine a state interval based on the current pose and filter a set of keyframes from the state interval. The image training and feature extraction module is used to input the keyframe set into a pre-trained deep model to extract Gaussian ellipsoid features. In the pre-trained deep model, features of each keyframe are sampled through a Gaussian pyramid, and the sampled features of the Gaussian pyramid are restored through a Laplacian pyramid. The 3D Gaussian rendering and reconstruction module is used to calculate the Laplacian residual and screen space error of the Gaussian ellipsoid feature. Based on the Laplacian residual and screen space error, it determines the target Gaussian ellipsoid feature that needs to be split and the splitting direction, so as to iteratively split the target Gaussian ellipsoid feature according to the splitting direction.

[0101] As one implementation scheme, Figure 11 This is a schematic diagram of the hardware operating environment of the computer system involved in the embodiments of this application.

[0102] like Figure 11 As shown, the computer system may include: a processor 1001, such as a CPU; a memory 1005; a user interface 1003; a network interface 1004; and a communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0103] Those skilled in the art will understand that Figure 11 The computer system architecture shown does not constitute a limitation on the computer system and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0104] like Figure 11As shown, the memory 1005, as a storage medium, may include an operating system, a network communication module, a user interface module, and computer programs. The operating system is a program that manages and controls the hardware and software resources of the computer system, as well as the operation of the computer programs and other software or programs.

[0105] exist Figure 11 In the computer system shown, the user interface 1003 is mainly used to connect to the terminal and communicate with the terminal; the network interface 1004 is mainly used to communicate with the backend server; and the processor 1001 can be used to call the computer program stored in the memory 1005.

[0106] In this embodiment, the computer system includes: a memory 1005, a processor 1001, and a computer program stored in the memory and executable on the processor, wherein: When processor 1001 calls a computer program stored in memory 1005, it performs the following operations: S10, acquire the RGB-depth image collected by the humanoid robot; S20, the RGB-depth image is segmented to obtain a binary mask and then subjected to composite morphological optimization to obtain an optimized binary mask; S30, Select a set of valid feature points from the optimized binary mask, and obtain the current pose of the humanoid robot based on the set of valid feature points; S40, determine the state interval based on the current pose, and filter out the keyframe set from the state interval; S50, the keyframe set is input into a pre-trained deep model to extract Gaussian ellipsoid features, wherein the pre-trained deep model samples the features of each keyframe through a Gaussian pyramid and restores the sampled features of the Gaussian pyramid through a Laplacian pyramid; S60, calculate the Laplacian residual and screen space error of the Gaussian ellipsoid feature, and based on the Laplacian residual and screen space error, determine the target Gaussian ellipsoid feature that needs to be split and the splitting direction, so as to iteratively split the target Gaussian ellipsoid feature according to the splitting direction.

[0107] Furthermore, those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in a computer system to implement the process steps of the embodiments of the above methods.

[0108] Therefore, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the various steps of the vision-enhanced humanoid robot 3D Gaussian SLAM method as described in the above embodiments.

[0109] The computer-readable storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0110] It should be noted that, since the storage medium provided in the embodiments of this application is the storage medium used to implement the methods of the embodiments of this application, those skilled in the art can understand the specific structure and variations of the storage medium based on the methods described in the embodiments of this application, and therefore will not be repeated here. All storage media used in the methods of the embodiments of this application fall within the scope of protection of this application.

[0111] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0112] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0113] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0114] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0115] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0116] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A vision-enhanced 3D Gaussian SLAM method for humanoid robots, characterized in that, The method includes the following steps: S10, acquire the RGB-depth image collected by the humanoid robot; S20, the RGB-depth image is segmented to obtain a binary mask and then subjected to composite morphological optimization to obtain an optimized binary mask; S30, Select a set of valid feature points from the optimized binary mask, and obtain the current pose of the humanoid robot based on the set of valid feature points; S40, determine the state interval based on the current pose, and filter out the keyframe set from the state interval; S50, the keyframe set is input into a pre-trained deep model to extract Gaussian ellipsoid features, wherein the pre-trained deep model samples the features of each keyframe through a Gaussian pyramid and restores the sampled features of the Gaussian pyramid through a Laplacian pyramid; S60, calculate the Laplacian residual and screen space error of the Gaussian ellipsoid feature, and based on the Laplacian residual and screen space error, determine the target Gaussian ellipsoid feature that needs to be split and the splitting direction, so as to iteratively split the target Gaussian ellipsoid feature according to the splitting direction.

2. The vision-enhanced humanoid robot 3D Gaussian SLAM method as described in claim 1, characterized in that, S20 includes: S21, using YOLOv11 to process the input RGB-depth image frame Instance segmentation is performed, and a binary mask is generated through thresholding: ； S22, adopts Elliptical structural elements ,right Perform a closing operation, then use Elliptical structural elements The mask is expanded, that is: ； in, This represents the closing operation, used to fill small holes inside a mask; This represents the dilation operation, used to expand the boundaries of foreground pixels and compensate for undersegmentation of edges in instance segmentation. S23, using a two-dimensional Gaussian check Perform convolution, then use Gaussian smoothing to eliminate jagged edge artifacts, generating a smoothed grayscale mask: ； in, For convolution operations, The standard deviation is the Gaussian kernel. S24, the smoothed grayscale mask Perform binarization again to generate an optimized binary mask: ； in, Represents pixels This refers to an invalid area that is dynamic or occluded. Represents pixels This refers to a static, unobstructed effective area.

3. The vision-enhanced humanoid robot 3D Gaussian SLAM method as described in claim 1, characterized in that, S30 includes: S31, from optimizing binary masks Select the effective feature point set from the data: ； S32, Construct a function that minimizes the sum of squares of the projection error: ； in, A camera projection function that projects 3D points onto 2D pixels; Represents valid points in the image The corresponding 3D point in the world coordinate system; R is the rotation matrix; t is the translation vector; S33, Constraint Matrix Based on Humanoid Robot Pose Vector Solve for all valid points Corresponding projection results The current pose of the humanoid robot is obtained by finding R and t that minimize the sum of squared errors from the actual effective points. ,in: 。 4. The vision-enhanced humanoid robot 3D Gaussian SLAM method as described in claim 1, characterized in that, S40 includes: S41, Determine the change in motion pose of the humanoid robot based on the current pose. : ； set up The upper and lower bounds are respectively and , and the interval The input ordinary frames are divided into candidate intervals; S42, determine the state interval of the target frame that is in the candidate interval. : ； In the formula, This represents the state interval of the previous state after scaling. For interval Kalman gain, The interval for measuring residuals; in: ； ； ； In the formula, It is a measurement matrix. It is the interval covariance matrix of the measurement residuals. For observation models, For interval midpoint, This is the scaling factor. This is the state interval of the previous state; S43, define the state interval The set of all candidate frames F is: ； Filter the set according to at least one of the following rules Keyframes in: Rule 1: If If the data contains only one frame, then that frame is directly selected as the keyframe. Rule 2: If Then calculate the weights of each candidate frame F: ； in, Pose information of candidate frame F; State interval The intermediate reference pose; It is a factor that controls the scale of weight changes; among them, the number of visible static map points in the candidate frame is greater than a preset threshold. ; Based on the weights of each candidate frame F Select keyframe W: ； In the formula, s is the stability index of the robot during movement. This represents the number of visible static map points within the frame. in, ; In the formula, Represents the joint torque vector of the robot Maximum torque modulus Indicates the maximum difference in force between the left and right feet The model, This is the adjustment coefficient; Rule 3: When the robot is stationary or moves a distance less than a preset distance threshold, if the time interval between the current frame and the previous keyframe exceeds the preset threshold, the current frame is directly selected as the keyframe.

5. The vision-enhanced humanoid robot 3D Gaussian SLAM method as described in claim 1, characterized in that, In S50, features of each keyframe are sampled using a Gaussian pyramid, and the sampled features from the Gaussian pyramid are then reconstructed using a Laplacian pyramid, including: S51, for Gaussian pyramid images of each level are generated by downsampling a Gaussian pyramid with a four-layer structure: ； in, This indicates downsampling of the k-th Gaussian pyramid of the image, where s is the sampling factor; S52, calculate the residuals between two adjacent Gaussian pyramid images, and use them as keyframes. The k-th level of the Laplace Pyramid : ； In the formula, This is an upsampling operation; S53, in the image reconstruction stage, the Gaussian pyramid image is recovered layer by layer using the Laplacian pyramid to reconstruct keyframes containing Gaussian ellipsoid features. : 。 6. The vision-enhanced humanoid robot 3D Gaussian SLAM method as described in claim 5, characterized in that, The training function expression for the pre-trained deep model during the training process is as follows: ； In the formula, and These are the training error losses for the Gaussian pyramid and the Laplacian pyramid, respectively. Dynamic weights; The pre-trained deep model is trained on the highest-level Gaussian pyramid image during the training process. The optimization begins here, where the expression for the objective function is: ； In the formula, Indicates the first The rendered image of the layer.

7. The vision-enhanced humanoid robot 3D Gaussian SLAM method as described in claim 1, characterized in that, S60 includes: S61, Calculate the Laplacian residual of the Gaussian ellipsoid feature in the l-th training layer. Screen Space Error (SSE): ； ； In the formula, For pixels In the High-frequency energy intensity of the layer, For the maximum high-frequency energy intensity, For a constant term, This represents the projection transformation operator composed of both camera intrinsic and extrinsic parameters; Wherein, the high-frequency energy intensity in the current level l is greater than or equal to a preset threshold. Laplace energy, preset threshold The expression is: ； In the formula, Let l be the mean of the Laplace energy at the current level. The coefficient is used to control the threshold range. The standard deviation of the Laplace energy at the current level; S62, based on Laplace residuals And screen space error SSE, determine the mixed error weights : ； In the formula, This represents the maximum value of the screen space error; S63, based on the mixed error weights For the original gradient threshold and screen space error threshold Adjustments will be made: ； In the formula, For adjustment Sensitivity coefficient to the effect of the threshold; S64 will be greater than the gradient threshold. and screen space error threshold The Gaussian ellipsoid features are identified as the target Gaussian ellipsoid features. S65, iteratively split the target Gaussian ellipsoid feature, wherein the splitting direction is... Satisfy the following expression: ； in, The 3D coordinates of the original Gaussian ellipsoid. The gradient of the mixed error weights, For the screen space error gradient, For random disturbance terms, These are the gradient coefficients.

8. A 3D Gaussian SLAM model of a humanoid robot, characterized in that, The humanoid robot 3D Gaussian SLAM model includes: The image acquisition module is used to acquire RGB-depth images captured by the humanoid robot; The dynamic mask acquisition and optimization module is used to perform instance segmentation on the RGB-depth image to obtain a binary mask and perform composite morphological optimization to obtain an optimized binary mask. The pose estimation module is used to filter out the effective feature point set from the optimized binary mask and obtain the current pose of the humanoid robot based on the effective feature point set. The keyframe selection module is used to determine a state interval based on the current pose and filter a set of keyframes from the state interval. The image training and feature extraction module is used to input the keyframe set into a pre-trained deep model to extract Gaussian ellipsoid features. In the pre-trained deep model, features of each keyframe are sampled through a Gaussian pyramid, and the sampled features of the Gaussian pyramid are restored through a Laplacian pyramid. The 3D Gaussian rendering and reconstruction module is used to calculate the Laplacian residual and screen space error of the Gaussian ellipsoid feature. Based on the Laplacian residual and screen space error, it determines the target Gaussian ellipsoid feature that needs to be split and the splitting direction, so as to iteratively split the target Gaussian ellipsoid feature according to the splitting direction.

9. A computer system, characterized in that, The computer system includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the vision-enhanced humanoid robot 3D Gaussian SLAM method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the vision-enhanced humanoid robot 3D Gaussian SLAM method as described in any one of claims 1 to 7.

Images