Anti-false positive noise occupied grid prediction method and system for tracked robot in unstructured scene

Abstract

The invention provides an occupancy grid prediction method and system for preventing false positive noise of a tracked robot in an unstructured scene, and aims to overcome the defects that an existing visual three-dimensional occupancy prediction model is insufficient in depth estimation precision and poor in false positive suppression capability. The consistency of BEV features is improved by using depth reconstruction and difference weighting, and negative samples are screened based on an uncertainty graph to suppress false occupancy. According to the invention, the accuracy and robustness of depth prediction are effectively improved, and a more reliable and more accurate three-dimensional environment perception capability is provided for a perception system of an unstructured environment.

Description

Technical Field

[0001] This invention relates to the field of robot environmental perception technology, and in particular to an occupancy grid prediction method and system for tracked robots in unstructured scenarios to resist false positive noise. Background Technology

[0002] With the development of robot perception technology, constructing continuous and complete 3D unstructured environment representations is crucial. Vision-based 3D occupancy prediction has become a research hotspot due to its low cost and rich semantic information. However, this task faces two major challenges: first, recovering 3D depth from 2D images is an ill-posed problem, leading to insufficient depth estimation accuracy, especially in distant and occluded regions; second, depth errors can cause repeated predictions or even false positive occupancy along the camera ray direction, seriously affecting the safety of downstream planning and control modules. Existing methods have significant shortcomings in handling depth features of targets at different distances and in suppressing false positive noise in high-uncertainty regions. Summary of the Invention

[0003] (a) Technical problems to be solved

[0004] Based on this, the present invention provides an occupancy grid prediction method and system for tracked robots in unstructured scenarios to resist false positive noise, which significantly improves the accuracy of depth estimation and the ability to suppress false positive noise.

[0005] (II) Technical Solution

[0006] To achieve the above objectives, this invention provides a method for predicting the occupied grid of a tracked robot in unstructured scenarios to resist false positive noise, comprising:

[0007] Step 1: Acquire synchronized multi-view images of the tracked robot's surroundings;

[0008] Step 2: Construct a hierarchical deep denoising occupancy prediction model HDD-Occ; train the HDD-Occ model based on the Occ3D-nuScenes dataset;

[0009] Step 3: Input the multi-view images into the trained HDD-Occ model and output the 3D occupancy prediction results;

[0010] Specifically, constructing the hierarchical deep denoising occupancy prediction model HDD-Occ includes:

[0011] Step S1: Two-dimensional image feature extraction: Two-dimensional image features are extracted from the input multi-view images using a lightweight ResNet-50 backbone network;

[0012] Step S2: Multi-channel layered depth feature extraction: Construct a multi-channel layered depth processing network, input the two-dimensional image features into the multi-channel layered depth processing network, and output multi-channel layered depth features and predicted depth map;

[0013] Step S3: View transformation processing: Deeply fuse multi-channel layered depth features with two-dimensional image features, and project the result onto a unified BEV space to generate preliminary BEV features;

[0014] Step S4: Depth difference weighting: First, map the preliminary BEV features back to the depth image plane to obtain the reconstructed depth map; then, calculate the pixel-wise difference between the predicted depth map and the reconstructed depth map, and transform this difference map to the BEV space to generate auxiliary BEV features; finally, perform weighted fusion of the preliminary BEV features and the auxiliary BEV features to generate the difference-weighted BEV features.

[0015] Step S5: False Positive Denoising Processing: First, calculate the uncertainty map based on the true depth map and the predicted depth map; then, in regions with high uncertainty, select real 3D points that occupy empty labels as negative samples, and perform depth perturbation sampling on these negative samples; next, encode these negative sample information into denoised BEV features through a multilayer perceptron and a Gaussian scattering projection layer; finally, concatenate and fuse the difference-weighted BEV features obtained in Step S4 with the denoised BEV features to generate the final BEV features;

[0016] Step S6: Feed the final BEV features into the occupancy prediction head module and output a dense 3D occupancy grid and its semantic labels.

[0017] Furthermore, step S2 specifically includes:

[0018] Step S21: Extract depth features at different scales through a multi-scale pyramid structure to obtain multi-scale depth features;

[0019] Step S22: Extract depth features with different numbers of channels by passing the multi-scale depth features through three parallel branches: small channel branch, medium channel branch and large channel branch;

[0020] Step S23: Weighted fusion of the depth features output from the three parallel branches to output multi-channel hierarchical depth features and predicted depth map.

[0021] Furthermore, a forward-backward paradigm based on LSS and BEVFormer is adopted for forward and backward projection.

[0022] Furthermore, step S4 specifically includes:

[0023] Step S41: Map the initial BEV features back to the depth image plane to obtain the reconstructed depth map;

[0024] Step S42: Calculate the pixel-by-pixel difference between the predicted depth map and the reconstructed depth map, and transform this difference map into the BEV space to generate auxiliary BEV features;

[0025] Calculate the predicted depth With reconstruction depth The differences between them are used to obtain a depth difference map; for regions with large differences, i.e., regions with inconsistent depth, their weights are reduced when fused into the BEV representation; for regions with small differences, their influence weights are increased.

[0026] Step S43: Weighted fusion of the initial BEV features and auxiliary BEV features to generate differentially weighted BEV features.

[0027] Furthermore, step S5 specifically includes:

[0028] Step S51: Convert the actual depth value Compared with predicted depth value Transformation, respectively yielding and ;in, , For deep feature extraction transformation function;

[0029] Step S52: According to and Construct an uncertainty graph;

[0030] The depth map is normalized, and an uncertainty map is obtained by mapping using the sigmoid function.

[0031]

[0032] in, This setting is to prevent division by zero; a higher value in the uncertainty graph indicates less accurate depth prediction.

[0033] Step S53: Based on the uncertainty map and the true labels of the occupied grid, select points with high uncertainty values ​​and empty true labels of the occupied grid as negative sample points;

[0034] First, candidate points with high uncertainty values ​​are selected using an uncertainty map. Then, the world 3D point coordinates corresponding to the candidate points are calculated using the pixel coordinates of the candidate points and their depth values ​​in the predicted depth map. Finally, it is determined whether the actual grid occupied by the world 3D point coordinates is empty. If so, the candidate point is a negative sample point.

[0035] The world 3D point coordinates corresponding to the candidate points are calculated using the following formula:

[0036]

[0037]

[0038]

[0039]

[0040] in, Indicates the image input width; Indicates the image input height; Indicates the horizontal field of view angle; Indicates the vertical field of view angle; Indicates camera intrinsic parameters; Indicates external reference. Represents the rotation matrix. Represents the translation vector; Represents the coordinates of a 3D point in the world (with the center of the tracked robot as the origin of the world coordinate system); Represents the product of an image pixel and the predicted depth value;

[0041] Step S54: Offset sampling of negative sample points along the depth;

[0042] Let the sampled value It follows a Beta distribution, i.e. The offset is:

[0043]

[0044] in, This is a hyperparameter representing the sampling radius;

[0045] By scrambling the depth of negative sample points, a series of neighborhood point samples are generated to remind the model that there is no target in this range;

[0046] Step S55: The negative sample information is mapped into a denoised feature vector through a multilayer perceptron (MLP), and then projected onto the BEV space through a Gaussian scattering projection layer to obtain the denoised BEV features; wherein, the negative sample information includes negative sample points and their offset sampling points along the depth direction.

[0047] The Gaussian convolution kernel formula used in the Gaussian scattering projection layer is as follows:

[0048]

[0049] in, Represents the horizontal coordinate of BEV; Represents the vertical coordinates of the BEV; Indicates the smoothness parameter;

[0050] Step S56: The difference-weighted BEV features obtained in step S4 are concatenated and fused with the denoised BEV features to generate the final BEV features.

[0051] Furthermore, in step S6, the occupancy prediction head module includes a 3D convolutional upsampling layer, which ultimately outputs a dense 3D occupancy grid with a resolution of 100×100×8 and its 18 semantic labels.

[0052] Further, in step S23, the cross-entropy loss is calculated for the depth output of each branch, and the entropy value of its predicted depth distribution is used as a regularization term; the hierarchical depth loss is defined as:

[0053]

[0054] in, Represents cross-entropy loss; express The entropy of the branch prediction probability distribution; Indicates the true probability label; Indicates the channel-level weighting coefficient; This represents the entropy weighting coefficient;

[0055] In step S43, a loss term is introduced to minimize the deviation between the reconstructed depth map and the predicted depth map, ensuring consistency; the depth difference loss function is given by the following formula:

[0056]

[0057] in, Indicates the number of depth maps in the training batch; This represents the weighting coefficient of the loss;

[0058] In step S52, a distribution difference loss function was designed based on the true depth map. With predicted depth map The maximum mean difference loss (MMD) is calculated in the feature space to reduce depth distribution differences; the distribution loss is defined as:

[0059]

[0060] in, The calculation formula is:

[0061]

[0062] in, Represents the characteristics of real data samples; Indicates the characteristics of the predicted data samples; Represents the Gaussian projection mapping function; Indicates the number of real data samples; Indicates the number of samples in the prediction data; Denotes the norm in the reproducing kernel Hilbert space;

[0063] In step S54, for the selected negative sample points, the binary cross-entropy is calculated as the denoising loss; assuming there exists A selected false positive point is set. For the first If the predicted occupancy probability of a point is given, then:

[0064]

[0065] in, Represents the binary cross-entropy loss function; Indicates whether the prediction point is occupied; Batch indexed 3D coordinates The prediction point at the location; the loss function penalizes cases where a non-occupied location is predicted as an occupied voxel;

[0066] In step S6, supervised learning was performed using the total loss function, which includes: voxel loss, layer depth loss, depth difference loss, distribution difference loss, and denoising loss, defined as follows:

[0067]

[0068] Among them, voxel loss is a computational function used to measure the degree of difference between the dense 3D occupancy mesh output by the occupancy prediction head module and the actual 3D occupancy mesh.

[0069] On the other hand, the present invention also provides an occupancy grid prediction system for tracked robots in unstructured scenarios to resist false positive noise, the method comprising:

[0070] Multi-view image input module: used to receive synchronized image data from multiple cameras and acquire multi-view images;

[0071] Image feature extraction module: used to extract two-dimensional image features from multi-view images;

[0072] Multi-channel hierarchical depth feature extraction module: It is used to extract depth features at different scales by taking two-dimensional image features as input; then, through a parallel network composed of small channel branch, medium channel branch and large channel branch, it processes the depth information of near-range large targets, medium-range regular targets and far-range small targets respectively, and outputs multi-channel hierarchical depth features and predicted depth map;

[0073] View Transformation Module: Used to fuse multi-channel hierarchical depth features with two-dimensional image features, and to generate preliminary BEV features by mapping to the BEV space through a forward-backward paradigm based on LSS and BEVFormer.

[0074] The depth difference weighting module is used to perform depth reconstruction on the initial BEV features to obtain the reconstructed depth map, and calculate the difference between the reconstructed depth map and the predicted depth map to generate a depth difference map; then the depth difference map is transformed to the BEV space to generate auxiliary BEV features; finally, the initial BEV features and auxiliary BEV features are weighted and fused to generate the difference-weighted BEV features.

[0075] False positive denoising module: It is used to construct an uncertainty map based on the real depth map and the predicted depth map, select voxels with high uncertainty and empty labels as negative samples, and perform depth perturbation sampling on these negative samples; then the negative sample information is mapped to the BEV space through a multilayer perceptron and Gaussian scattering projection to generate denoised BEV features.

[0076] Feature fusion module: used to fuse the difference-weighted BEV features with the denoised BEV features to generate the final BEV features;

[0077] Occupation Prediction Head Module: Used to upsample the final BEV features and predict a dense 3D occupancy grid and its semantic category.

[0078] (III) Beneficial Effects

[0079] As can be seen from the above technical solution, the occupancy grid prediction method and system for tracked robots in unstructured scenarios proposed in this invention have the following beneficial effects:

[0080] 1. The multi-channel hierarchical depth feature extraction proposed in this invention effectively alleviates the problem that traditional single-branch networks cannot take into account both near and far distance targets, making depth prediction more balanced and stable.

[0081] 2. The view transformation and depth difference weighting processing proposed in this invention supplements the predicted depth by reconstructing the depth and enhances the weight of high-confidence regions to improve the BEV representation quality generated by the backward module, thus significantly improving the geometric accuracy of BEV features.

[0082] 3. For high-discrepancy regions, a depth perturbation sampling denoising mechanism is proposed. By actively screening negative samples for supervised learning, the false positive phenomenon caused by inaccurate depth is fundamentally suppressed, and the robustness and practicality of the 3D occupancy grid prediction model are improved. Attached Figure Description

[0083] The features and advantages of the invention will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the invention in any way. In the drawings:

[0084] Figure 1 This is a flowchart illustrating the occupancy grid prediction method for tracked robots in unstructured scenarios according to the present invention.

[0085] Figure 2 This is a schematic diagram of the multi-channel layered depth feature extraction of the present invention;

[0086] Figure 3 This is a schematic diagram of the view transformation and depth difference weighting processing of the present invention;

[0087] Figure 4 This is a schematic diagram of the false positive noise reduction process of the present invention;

[0088] Figure 5 This is a comparison chart of the prediction accuracy of HDD-Occ and FB-Occ for each type of occupied grid in this invention;

[0089] Figure 6 This is a convergence trend graph of the ablation experiment of this invention;

[0090] Figure 7 This is a visual comparison analysis diagram of the present invention with real occupancy and FB-Occ occupancy in different scenarios;

[0091] Figure 8 This is a visualization of the depth prediction map results of the present invention in different scenarios;

[0092] Figure 9 This is a schematic diagram of the framework of the occupancy grid prediction system for tracked robots in unstructured scenarios according to the present invention. Detailed Implementation

[0093] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0094] In the era of autonomous driving and advanced robotics, perception systems not only need to detect specific objects but also construct continuous and complete 3D representations of the environment. Vision-based 3D spatial occupancy prediction, as an emerging paradigm, predicts the occupancy status and semantic categories of surrounding spatial voxels through monocular or multi-camera input, providing richer spatial information for decision-making and planning. Recent research shows that compared to sparse bird's-eye view (BEV) object detection or 3D object detection, dense occupancy representation can capture vertical structures, occluded areas, and complex geometries, significantly improving the system's ability to understand the environment.

[0095] like Figure 1 As shown, this invention provides a method for predicting the occupied grid of a tracked robot in unstructured scenarios to resist false positive noise, including:

[0096] Step 1: Multi-view image input: Acquire synchronous multi-view images of the tracked robot's surroundings;

[0097] In this embodiment, the multi-view images are synchronous RGB images from six cameras around the tracked robot, located at the front left, front right, front right, rear left, rear right, and rear right respectively.

[0098] Step 2: Construct the hierarchical deep denoising occupancy prediction network (HDD-Occ); train the HDD-Occ model based on the Occ3D-nuScenes dataset.

[0099] Step 3: Input the multi-view images into the trained HDD-Occ model and output the 3D occupancy prediction results.

[0100] Furthermore, the construction of the hierarchical deep denoising occupancy prediction model HDD-Occ specifically includes:

[0101] Step S1: Two-dimensional image feature extraction: Two-dimensional image features are extracted from the input multi-view images using a lightweight ResNet-50 backbone network;

[0102] Step S2: Multi-channel layered depth feature extraction: Construct a multi-channel layered depth processing network, input the two-dimensional image features into the multi-channel layered depth processing network, and output multi-channel layered depth features and predicted depth map;

[0103] Generally, near-field regions have relatively sparse targets and larger imaging scales, while far-field regions contain densely packed small targets and are more susceptible to perspective distortion. This difference means that a network may perform well at medium distances but has "blind spots" at extremely close or far distances. Inspired by the hierarchical structure and parallel processing mechanisms of the brain's visual cortex, this invention designs a multi-channel hierarchical depth processing network capable of adaptively capturing depth features at different target densities. Figure 2 As shown, in this network, the layered depth module corresponds to a deep layered architecture, while the multi-channel depth module is associated with a flattening processing scheme; specifically including:

[0104] Step S21: Extract depth features at different scales through a multi-scale pyramid structure to obtain multi-scale depth features;

[0105] In this embodiment, ASPP (Atrous Spatial Pyramid Pooling) is used to obtain multi-scale depth features.

[0106] Step S22: Extract depth features with different numbers of channels from the multi-scale depth features through three parallel branches (small channel branch, medium channel branch, and large channel branch);

[0107] Multi-channel depth processing comprises three parallel branches, responsible for extracting features from the "small channel," "medium channel," and "large channel," respectively. Specifically:

[0108] (1) Small channel branch: Using a smaller number of convolutional channels, it focuses on medium and near-range targets in the image that have strong feature responses and are easy to identify. Since near and medium-range targets usually occupy a large area in the image and contain richer color or texture information, the small channel branch can efficiently extract these features with a lightweight structure, ensuring that the network can make fast responses and accurate predictions for large-scale objects at close range.

[0109] (2) Medium-channel branch: A medium number of channels and parameters are used to achieve a balance between network width and detail capture capability. This branch retains a moderate network capacity to extract features of various targets at medium distances, while avoiding excessive computational costs caused by an overly wide network, thus achieving a good balance between accuracy and efficiency.

[0110] (3) Large channel branch: Using more convolution channels to capture subtle depth information of densely distributed small targets in distant regions. The large channel branch can more fully exploit the low-resolution features at the far end of the image, enhance the network's sensitivity to weak signals, and thus reduce the generation of "hole" blind spots at the far end.

[0111] Step S23: Weightedly fuse the depth features output from the three parallel branches to output multi-channel hierarchical depth features and a predicted depth map;

[0112] During the training phase, the depth features output from the three parallel branches are weighted and fused. Through a weighted loss function, the multi-channel hierarchical depth processing network can adaptively learn the optimal combination of depth features in different scenarios and output multi-channel hierarchical depth features and predicted depth maps.

[0113] Given the greater difficulty in depth prediction for dense target regions, a hierarchical weighted loss strategy is employed to handle the three depth branches. Specifically, for the depth output of each branch, a cross-entropy loss is calculated, and the entropy value of its predicted depth distribution is used as a regularization term (penalizing high-discrepancy predictions). The hierarchical depth loss is defined as follows:

[0114]

[0115] in, Represents cross-entropy loss; express The entropy of the branch prediction probability distribution; Indicates the true probability label; Indicates the channel-level weighting coefficient; This represents the entropy weighting coefficient.

[0116] This invention assigns higher weights to the middle and large channel branches to force the model to focus on optimizing the depth prediction capability of dense target regions.

[0117] Through this hierarchical network design and weighted loss calculation, the network can learn depth features at near, medium, and far distances more evenly, effectively reducing blind spots at extreme distances.

[0118] Step S3: View transformation processing: Deeply fuse multi-channel layered depth features with two-dimensional image features, and project the result onto a unified BEV space to generate preliminary BEV features;

[0119] The predicted depth map output by the multi-channel hierarchical depth processing network is denoted as... Through the back projection module (i.e., B-VTM), Elevate to BEV space to generate initial BEV characteristics .

[0120] In this embodiment, forward and backward projection are performed using a forward-backward paradigm based on LSS (Lift-Splat-Shoot) and BEVFormer. This is derived from J. Philion and S. Fidler, “Lift, splat, shoot: Encoding images from arbitrary camera rigs by implicitly unprojecting to 3d,” in European conference on computer vision, Springer, 2020, pp. 194–210, and Z. Li, W. Wang, H. Li, E. Xie, C. Sima, T. Lu, Q. Yu, and J. Dai, “Bevformer: learning bird's-eye-view representation from lidar-camera via spatiotemporal transformers,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. This is prior art and will not be elaborated upon further in this paper.

[0121] Step S4: Depth difference weighting: First, map the preliminary BEV features back to the depth image plane to obtain the reconstructed depth map; then, calculate the pixel-wise difference between the predicted depth map and the reconstructed depth map, and transform this difference map to the BEV space to generate auxiliary BEV features; finally, perform weighted fusion of the preliminary BEV features and the auxiliary BEV features to generate the difference-weighted BEV features.

[0122] To enhance the consistency between depth estimation and bird's-eye view (BEV) features, this invention introduces "projection depth reconstruction" and "depth difference feedback" on top of the standard forward and backward projection view transformation process. These elements together impose strict dual constraints on the subsequent depth difference weighting process. Figure 3 As shown, it specifically includes:

[0123] Step S41: Map the initial BEV features back to the depth image plane to obtain the reconstructed depth map;

[0124] Although semantic information from multi-view images has been integrated However, discrepancies may still exist between the actual depth map and the real depth map. To verify and compensate for these discrepancies, a projection reconstruction operation is performed: [The following text appears to be a separate, unrelated section:] ... Reprojecting back onto the depth image plane yields the reconstructed depth map. This process can be viewed as a depth autoencoder-decoder structure: first, the image and depth features are "encoded" into the BEV space, and then "decoded" back to pixel-level depth. This closed loop not only verifies the degree to which depth information is preserved between the two view transformations, but also provides a reliable basis for subsequent depth consistency analysis.

[0125] Step S42: Calculate the pixel-by-pixel difference between the predicted depth map and the reconstructed depth map, and transform this difference map into the BEV space to generate auxiliary BEV features;

[0126] Calculate the predicted depth With reconstruction depth The differences between the regions are used to obtain a depth difference map. For regions with large differences (i.e., regions with inconsistent depth), their weights are reduced when fused into the BEV representation; for regions with small differences, their influence weights are increased. Specifically, the depth difference calculated at each pixel location is transformed into the BEV space to generate an auxiliary BEV feature map that encodes depth consistency. This auxiliary BEV feature was then used to reweight the initial BEV feature.

[0127] Step S43: Weight and fuse the initial BEV features and the auxiliary BEV features to generate differentially weighted BEV features;

[0128] By employing a depth difference weighting mechanism, this invention imposes dual constraints on the projection process, effectively correcting spatial distortions caused by depth estimation errors.

[0129] During the training phase, this embodiment designs a depth difference loss function. By introducing a loss term, it minimizes the deviation between the reconstructed depth map and the predicted depth map, ensuring consistency. The depth difference loss function is given by the following equation:

[0130]

[0131] in, Indicates the number of depth maps in the training batch; This represents the weighting coefficient of the loss.

[0132] Step S5: False Positive Denoising Processing: First, calculate the uncertainty map based on the true depth map and the predicted depth map; then, in regions with high uncertainty, select real 3D points that occupy empty labels as negative samples, and perform depth perturbation sampling on these negative samples; next, encode these negative sample information into denoised BEV features through a multilayer perceptron and a Gaussian scattering projection layer; finally, concatenate and fuse the difference-weighted BEV features obtained in Step S4 with the denoised BEV features to generate the final BEV features;

[0133] Depth localization errors often lead to 3D points in images without actual objects being incorrectly identified as occupied voxels (i.e., the true label is empty, but the predicted label is a certain category), resulting in false positives. To suppress this type of noise, this invention proposes a false positive denoising process, such as... Figure 4 As shown, it specifically includes:

[0134] Step S51: Convert the actual depth value Compared with predicted depth value Transformation, respectively yielding and ;in, , For deep feature extraction transformation function;

[0135] Step S52: According to and Construct an uncertainty graph;

[0136] This invention normalizes the depth map and obtains an uncertainty map by mapping using the sigmoid function:

[0137]

[0138] in, This setting is used to prevent division by zero. Higher values ​​in the uncertainty graph indicate less accurate depth predictions.

[0139] During the training phase, this embodiment designs a distribution difference loss function based on the real depth map. With predicted depth map The maximum mean difference (MMD) loss is calculated in the feature space to reduce depth distribution variance. The distribution loss is defined as:

[0140]

[0141] in, The calculation formula is:

[0142]

[0143] in, Represents the characteristics of real data samples; Indicates the characteristics of the predicted data samples; Represents the Gaussian projection mapping function; Indicates the number of real data samples; Indicates the number of samples in the prediction data; This represents the norm in the Reproducing Kernel Hilbert Space (RKHS).

[0144] Step S53: Based on the uncertainty map and the true labels of the occupied grid, select points with high uncertainty values ​​and empty true labels of the occupied grid as negative sample points;

[0145] First, candidate points with high uncertainty values ​​are selected using an uncertainty map. Then, the world 3D point coordinates corresponding to the candidate points are calculated using the pixel coordinates of the candidate points and their depth values ​​in the predicted depth map. Finally, it is determined whether the actual occupied grid of the world 3D point coordinates is empty. If so, the candidate point is a negative sample point.

[0146] The world 3D point coordinates corresponding to the candidate points are calculated using the following formula:

[0147]

[0148]

[0149]

[0150]

[0151] in, Indicates the image input width; Indicates the image input height; Indicates the horizontal field of view angle; Indicates the vertical field of view angle; Indicates camera intrinsic parameters; Indicates external reference. Represents the rotation matrix. Represents the translation vector; Represents the coordinates of a 3D point in the world (with the center of the tracked robot as the origin of the world coordinate system); This represents the product of the image pixel value and the predicted depth value.

[0152] Negative sample points are potential false positive areas, and the model needs to learn to avoid predicting occupied states in these areas.

[0153] Step S54: Offset sampling of negative sample points along the depth;

[0154] To further enhance robustness, depth offset sampling is applied to selected negative sample points. Let the sampled values ​​be... It follows a Beta distribution, i.e. The offset is:

[0155]

[0156] in, This is a hyperparameter representing the sampling radius.

[0157] By scrambling the negative sample points at depth, a series of neighborhood point samples are generated to remind the model that there is no target in this range.

[0158] During the training phase, this embodiment calculates the binary cross-entropy as the denoising loss for selected negative sample points (including perturbed samples). It is assumed that... A selected false positive point is set. For the first If the predicted occupancy probability of a point is given, then:

[0159]

[0160] in, Represents the binary cross-entropy loss function; This indicates whether the predicted point is occupied (the label here is 0, meaning it is not occupied). Batch indexed 3D coordinates The prediction point is located at a given location. This loss function penalizes cases where a non-occupied location is predicted as an occupied voxel (i.e., a false positive).

[0161] Step S55: The negative sample information (including negative sample points and their offset sampling points along the depth direction) is mapped into a denoising feature vector through a multilayer perceptron (MLP), and then projected onto the BEV space through a Gaussian scattering projection layer to obtain the denoising BEV features.

[0162] The denoised feature vectors are projected onto the BEV space / grid via a Gaussian scattering mapping layer, distributing the influence of each vector to neighboring BEV cells and creating "hot spots" at the corresponding sample locations on the BEV grid. The Gaussian convolution kernel formula used in the Gaussian scattering projection layer is as follows:

[0163]

[0164] in, Represents the horizontal coordinate of BEV; Represents the vertical coordinates of the BEV; This represents the smoothness parameter.

[0165] By using Gaussian convolution kernels, image smoothing is achieved effectively while maintaining the sharpness of image edges. The generated denoised BEV feature map highlights areas that may generate false positives.

[0166] Step S56: The difference-weighted BEV features obtained in step S4 are concatenated and fused with the denoised BEV features to generate the final BEV features;

[0167] Fusing the denoised BEV features with the difference-weighted BEV features can effectively correct errors caused by false alarms and reduce the weight of features in regions that may be incorrectly predicted.

[0168] Through the above steps, the present invention can effectively suppress the phenomenon of false grid occupancy caused by inaccurate depth prediction and achieve noise reduction processing for high-probability misjudgment areas.

[0169] Step S6: Feed the final BEV features into the occupancy prediction head module and output a dense 3D occupancy grid and its semantic labels;

[0170] In this embodiment, the occupancy prediction head module includes a 3D convolutional upsampling layer, which ultimately outputs a dense 3D occupancy grid with a resolution of 100×100×8 and its 18 semantic labels (17 object categories and an empty label).

[0171] During the training phase, this embodiment performs supervised learning using a total loss function. The total loss includes: voxel loss, layer depth loss, depth difference loss, distribution difference loss, and denoising loss, which are defined as follows:

[0172]

[0173] Among them, voxel loss is a computational function used to measure the degree of difference between the dense 3D occupancy mesh output by the occupancy prediction head module and the actual 3D occupancy mesh.

[0174] In this embodiment, the HDD-Occ model is trained using the publicly available Occ3D-nuScenes dataset, employing only six RGB panoramic cameras as input and without using LiDAR signals. Occ3D-nuScenes contains 1,000 video sequences, divided into a training set (700 videos), a validation set (150 videos), and a test set (150 videos). Each video is approximately 20 seconds long, with annotations added at 0.5-second intervals. Each frame provides an occupancy label covering the range [-40m, -40m, -1m, 40m, 40m, 5.4m], and each cubic voxel has a side length of 0.4m. The dataset contains 17 object categories and provides camera masks to indicate whether voxels can be observed by the camera.

[0175] The mean intersection-union ratio (mIoU) is used as the metric to evaluate model performance, and its calculation formula is as follows:

[0176]

[0177] in, , and Representing the corresponding categories The number of true positives, false positives, and false negatives; This indicates the total number of categories.

[0178] This embodiment uses an input image size of 256×704 and generates a 3D voxel mesh with a resolution of 100×100×8. To reduce computational cost, a lightweight ResNet-50 backbone network is employed. It uses 16 frames of temporal input and only camera data. Training is performed on four NVIDIA RTX 4090 GPUs with a batch size of 4 and a learning rate of 2×10⁻⁶. -4 AdamW weights decay to 1×10 -2 It ran for 20 cycles, with a total training time of 55 hours.

[0179] On the Occ3D-nuScenes dataset, the proposed HDD-Occ model was compared with other state-of-the-art methods such as FB-Occ, Proto-Occ, and Fast-Occ.

[0180] Table 1. Comparison of prediction models on the Occ3D-nuScenes dataset

[0181]

[0182] As shown in Table 1, the proposed HDD-Occ significantly outperforms the Proto-Occ method by 1.05% in models using the same backbone network; even compared with methods using larger backbone networks or higher resolution inputs, HDD-Occ has an advantage, improving upon Fast-Occ by 1.40%. The FB-Occ method is derived from Z. Li, Z. Yu, D. Austin, M. Fang, S. Lan, J. Kautz, and JM Alvarez, “Fb-occ: 3d occupancy prediction based on forward-backward view transformation,” arXiv preprint arXiv:2307.01492, 2023; the Proto-Occ method is derived from J. Kim, C. Kang, D. Lee, S. Choi, and JW Choi, “Protoocc: Accurate, efficient 3d occupancy prediction using dualbranch encoder-prototype query decoder,” in Proceedings of the AAAIConference on Artificial Intelligence, vol. 39, no. 4, 2025, pp. 4284–4292; and the Fast-Occ method is derived from J. Hou, X. Li, W. Guan, G. Zhang, D. Feng, Y. Du, X. Xue, and J. Pu, “Fastocc: Accelerating 3d occupancy prediction by fusing the 2dbird’seye view and perspective view,” in 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 16 425–16 431.

[0183] Figure 5This is a comparison chart of the prediction accuracy of HDD-Occ and FB-Occ in each grid category. The yellow bars represent the IoU value of HDD-Occ; the blue bars represent the IoU value of FB-Occ; and the orange bars represent the improvement of this application in each category compared to FB-Occ. Figure 2 As shown, compared to FB-Occ, this application shows significant improvements in all categories except for a slight decrease in the bus category; the most significant improvements are in the terrain (+3.4%), sidewalks (+3.19%), other flat surfaces (+2.69%), traversable surfaces (+2.3%), and automobiles (+2.09%) categories. These substantial improvements make the perception of the static environment more reliable, which is beneficial for downstream decision-making and reduces errors caused by missed detections or false alarms.

[0184] In addition, an ablation experiment was also conducted in this embodiment. Figure 6 The curves showing the mean Intersection over Union (mIoU) variation over training epochs for different module configurations are presented. FB-Occ converges more slowly than all model variants in this application. The blue curve represents the baseline model of this application (excluding the multi-channel hierarchical depth feature extraction module, view transformation and depth difference weighting module, and false positive denoising module); the orange curve represents version V1 (adding the multi-channel hierarchical depth feature extraction module to the baseline model); the yellow curve represents version V2-1 (adding the multi-channel hierarchical depth feature extraction module and view transformation module to the baseline model); the green curve represents version V2-2 (adding the multi-channel hierarchical depth feature extraction module, view transformation and depth difference weighting module to the baseline model); and the cyan curve represents version V3 (adding the multi-channel hierarchical depth feature extraction module, view transformation and depth difference weighting module, and false positive denoising module to the baseline model). Our method exhibits a significantly faster learning speed in the early stages of training and converges substantially around the 16th epoch, demonstrating superior training efficiency.

[0185] Table 2 shows the ablation experiment results under different module settings. FB-Occ achieved an mIoU of 39.11%. When the multi-channel hierarchical depth feature extraction module was added alone, the model showed a stable improvement in each training cycle, with mIoU increasing to 39.41%. Next, the introduction of view transformation and depth difference weighting modules significantly improved the depth consistency mismatch problem, further increasing mIoU to 40.10%. Finally, based on the integration of the first two improvements, a false positive denoising module was added, achieving an mIoU of 40.61%, realizing the greatest overall performance improvement.

[0186] Table 2 Comparison of ablation results for each module after 20 rounds of training

[0187]

[0188] Furthermore, this embodiment performs qualitative analysis in two typical scenarios using six-channel camera view images, actual occupancy maps, HDD-Occ occupancy maps, and FB-Occ occupancy maps, such as... Figure 4 As shown.

[0189] Figure 7 (a) Visual contrast analysis in a closed environment: the left image is the ground truth (GT) occupancy map, the middle image is the occupancy map of our proposed method (HDD-Occ), and the right image is the occupancy map of FB-Occ. In closed environments, occupancy prediction tasks often face challenges in target recognition and classification, mainly due to the dense spatial distribution of multiple entities. In a cluttered scene, the first yellow box in the camera image contains four cars and one construction vehicle; the second box contains several pedestrians. The results show that the FB-Occ method misses targets at close range, while our proposed method successfully detects these targets.

[0190] Figure 7 (b) Visual contrast analysis in open environments: the left image is the true occupancy map (GT), the middle image is the occupancy map of our proposed method (HDD-Occ), and the right image is the occupancy map of FB-Occ. In open areas, accurate depth estimation and long-range object recognition pose significant challenges to occupancy prediction, primarily due to the decrease in sensor resolution with increasing distance and the reduction in feature discriminability. The first yellow box in the camera image shows wall-like obstacles; the second box contains multiple cars, a pedestrian, and a raised road barrier. The FB-Occ method has low prediction accuracy for these objects, while our proposed method correctly identifies the missed road barrier and sharpens the boundaries of the remaining objects.

[0191] Figure 8 The visualization results of the depth prediction maps generated by this method in two typical scenarios are shown. Figure 8 (a) and (b) represent enclosed and open environments, respectively. Higher color temperatures indicate closer distances, while cooler colors correspond to greater predicted depth. The results show that the proposed multi-channel hierarchical depth processing network has significant advantages in depth estimation performance, especially in capturing small-scale objects such as poles, pedestrians, and tree branches.

[0192] Overall, the HDD-Occ model proposed in this invention includes a multi-channel hierarchical depth feature extraction module (step S2), a view transformation and depth difference weighting module (steps S3-S4), and a false positive noise reduction module (step S5). When these modules are combined, the mIoU is improved by 1.5 percentage points compared to FB-Occ, verifying the effectiveness of this invention in dense 3D occupancy grid prediction tasks.

[0193] like Figure 9As shown, the present invention also provides an occupancy grid prediction system for tracked robots in unstructured scenarios to resist false positive noise, comprising:

[0194] Multi-view image input module: used to receive synchronized image data from multiple cameras and acquire multi-view images;

[0195] Image feature extraction module: used to extract two-dimensional image features from multi-view images;

[0196] Multi-channel hierarchical depth feature extraction module: It is used to extract depth features at different scales by taking two-dimensional image features as input; then, through a parallel network composed of small channel branch, medium channel branch and large channel branch, it processes the depth information of near-range large targets, medium-range regular targets and far-range small targets respectively, and outputs multi-channel hierarchical depth features and predicted depth map;

[0197] View Transformation Module: Used to fuse multi-channel hierarchical depth features with two-dimensional image features, and to generate preliminary BEV features by mapping to the bird's-eye view (BEV) space through a forward-backward paradigm based on LSS and BEVFormer.

[0198] The depth difference weighting module is used to perform depth reconstruction on the initial BEV features to obtain the reconstructed depth map, and calculate the difference between the reconstructed depth map and the predicted depth map to generate a depth difference map; then the depth difference map is transformed to the BEV space to generate auxiliary BEV features; finally, the initial BEV features and auxiliary BEV features are weighted and fused to generate the difference-weighted BEV features.

[0199] False positive denoising module: It is used to construct an uncertainty map based on the real depth map and the predicted depth map, select voxels with high uncertainty and empty labels as negative samples, and perform depth perturbation sampling on these negative samples; then the negative sample information is mapped to the BEV space through a multilayer perceptron and Gaussian scattering projection to generate denoised BEV features.

[0200] Feature fusion module: used to fuse the difference-weighted BEV features with the denoised BEV features to generate the final BEV features;

[0201] Occupation Prediction Head Module: Used to upsample the final BEV features and predict a dense 3D occupancy grid and its semantic category.

[0202] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A false positive noise resistant occupancy grid prediction method for tracked robots in unstructured environments, characterized in that, The application relates to a method for predicting three-dimensional occupancy based on a tracked crawler robot, and belongs to the field of computer vision. Step 1: acquiring synchronous multi-view images of the tracked crawler robot; Step 2: constructing a hierarchical deep denoising occupancy prediction model HDD-Occ; training the HDD-Occ model based on an Occ3D-nuScenes data set; Step 3: inputting the multi-view images into the trained HDD-Occ model to output three-dimensional occupancy prediction results; The hierarchical deep denoising occupancy prediction model HDD-Occ specifically comprises the following steps: Step S1: two-dimensional image feature extraction: extracting two-dimensional image features in the input multi-view images through a lightweight ResNet-50 backbone network; Step S2: multi-channel hierarchical depth feature extraction: constructing a multi-channel hierarchical depth processing network, inputting the two-dimensional image features into the multi-channel hierarchical depth processing network, and outputting multi-channel hierarchical depth features and a predicted depth map; Step S3: view transformation processing: performing deep fusion on the multi-channel hierarchical depth features and the two-dimensional image features, projecting the results to a unified BEV space, and generating preliminary BEV features; Step S4: depth difference weighting processing: first, mapping the preliminary BEV features back to a depth image plane to obtain a reconstructed depth map; then, calculating the pixel-by-pixel difference between the predicted depth map and the reconstructed depth map, and transforming the difference map to the BEV space to generate auxiliary BEV features; finally, performing weighted fusion on the preliminary BEV features and the auxiliary BEV features to generate difference-weighted BEV features; Step S5: false positive denoising processing: first, calculating an uncertainty map according to the real depth map and the predicted depth map; then, in the area with high uncertainty, filtering out 3D points that are real and have empty occupancy labels as negative samples, and performing depth disturbance sampling on the negative samples; then, encoding the negative sample information into denoising BEV features through a multi-layer perception and a Gaussian scattering projection layer; finally, splicing and fusing the difference-weighted BEV features obtained in step S4 and the denoising BEV features to generate final BEV features; Step S6: inputting the final BEV features into an occupancy prediction head module to output a dense three-dimensional occupancy grid and semantic labels thereof.

2. The method of claim 1, wherein, Step S2 specifically comprises the following steps: Step S21: extracting depth features at different scales through a multi-scale pyramid structure to obtain multi-scale depth features; Step S22: extracting depth features at different channel numbers through three parallel branches, namely a small-channel branch, a medium-channel branch and a large-channel branch; Step S23: performing weighted fusion on the depth features output by the three parallel branches to output multi-channel hierarchical depth features and a predicted depth map.

3. The method of claim 2, wherein, The forward-backward paradigm based on the LSS and the BEVFormer is adopted to perform forward projection and reverse projection.

4. The method of claim 3, wherein, Step S4 specifically comprises the following steps: Step S41: mapping the preliminary BEV features back to the depth image plane to obtain a reconstructed depth map; Step S42: calculating the pixel-by-pixel difference between the predicted depth map and the reconstructed depth map, and transforming the difference map to the BEV space to generate auxiliary BEV features; by calculating the difference between the predicted depth and the reconstructed depth , a depth difference map is obtained; for the regions with larger difference, i.e. the depth inconsistent regions, their weights are reduced when fused into the BEV representation; for the regions with smaller difference, their influence weights are enhanced; Step S43: the preliminary BEV feature and the auxiliary BEV feature are fused by weighting to generate a difference-weighted BEV feature.

5. The method of claim 4, wherein, Step S5 specifically includes: Step S51: converting the real depth value with the predicted depth value , respectively, to obtain and ; wherein , is a depth feature extraction conversion function; Step S52: According to With constructing an uncertainty graph; The depth map is normalized and mapped to an uncertainty map UncertaintyMap by a sigmoid function: wherein, Set to prevent division by zero; higher values in the uncertainty map indicate less accurate depth predictions; Step S53: according to the uncertainty map and the real label of the occupancy grid, points with high uncertainty values and empty real labels of the occupancy grid are selected as negative sample points; First, the candidate points with high uncertainty values are selected by using the uncertainty map; then, the world 3D point coordinates corresponding to the candidate points are calculated through the pixel coordinates of the candidate points and the depth values of the candidate points in the predicted depth map; finally, it is judged whether the real occupancy grid of the world 3D point coordinates is empty, if yes, the candidate point is a negative sample point; Wherein, the world 3D point coordinates corresponding to the candidate points are calculated according to the following formula: wherein, denotes the image input width; denotes the image input height; denotes the horizontal field of view angle; denotes the vertical field of view angle; denotes the camera intrinsic parameters; denotes the extrinsic parameters, denotes the rotation matrix, denotes the translation vector; denotes the world 3D point coordinate (with the center of the tracked robot as the world coordinate system origin); denotes the product of the image pixel point and the predicted depth value; Step S54: offset sampling is performed on the negative sample points along the depth; Let the sampled value Subject to a Beta distribution, i.e. The offset is then wherein, is a hyperparameter representing the sampling radius; By adding noise to the depth of the negative sample points, a series of neighborhood point samples are generated to remind the model that there is no target in this range; Step S55: the negative sample information is mapped into a noise reduction feature vector through a multi-layer perception network MLP, and then projected to the BEV space through a Gaussian scattering projection layer to obtain a noise reduction BEV feature; wherein, the negative sample information includes the negative sample points and the offset sampling points along the depth direction; The Gaussian convolution kernel formula used by the Gaussian scattering projection layer is as follows: wherein, denotes the BEV horizontal coordinate; denotes the BEV vertical coordinate; denotes the smoothness parameter; Step S56: the difference-weighted BEV feature obtained in step S4 is spliced and fused with the noise reduction BEV feature to generate a final BEV feature.

6. The method of claim 5, wherein, In step S6, the occupancy prediction head module includes a 3D convolution up-sampling layer, and finally outputs a dense three-dimensional occupancy grid with a resolution of 100x100x8 and its 18-class semantic labels.

7. The method of claim 6, wherein, In step S23, the cross-entropy loss is calculated for the depth output of each branch, and the entropy value of the predicted depth distribution is taken as a regularization term; the hierarchical depth loss is defined as: wherein, denotes a cross-entropy loss; denotes an entropy of the branch prediction probability distribution; denotes a true probability label; denotes a channel-level weight coefficient; denotes an entropy weight coefficient; In step S43, a loss term is introduced to minimize the deviation between the reconstructed depth map and the predicted depth map to ensure consistency; the depth difference loss function is given by: wherein, denotes the number of depth maps in the training batch; denotes a weight coefficient for the loss. In step S52, a distribution discrepancy loss function is designed, based on the real depth map and the predicted depth map The maximum mean discrepancy loss MMD is calculated in the feature space to reduce the depth distribution discrepancy; the distribution loss is defined as: wherein, The calculation formula is: wherein, represents a real data sample feature; represents a predicted data sample feature; represents a Gaussian projection mapping function; represents a number of real data samples; represents a number of predicted data samples; represents a norm in a reproducing kernel Hilbert space; In step S54, the binary cross-entropy is calculated as the denoising loss for the selected negative sample points; assuming there are selected false positive occupancy points, let be the predicted occupancy probability of the th point, then: where, denotes the binary cross-entropy loss function; denotes whether a predicted point is occupied or not; indexed by batch , three-dimensional coordinates of predicted points; this loss function penalizes predicting occupied voxels at non-occupied locations; In step S6, supervised learning is performed through the total loss function, and the total loss includes: voxel loss, hierarchical depth loss, depth difference loss, distribution difference loss and denoising loss, which are defined as: wherein the voxel loss is an operation function used to measure the degree of difference between the dense three-dimensional occupancy grid occupying the output of the prediction head module and the real three-dimensional occupancy grid.

8. An anti-false positive noise occupancy grid prediction system for tracked robots in unstructured environments, employing the method of any one of claims 1-6. Including: Multi-view image input module: used for receiving synchronous image data from multiple cameras to obtain multi-view images; Image feature extraction module: used for extracting two-dimensional image features from multi-view images; Multi-channel hierarchical depth feature extraction module: used for extracting depth features at different scales by taking two-dimensional image features as input; Then, the depth information of close-range large targets, medium-distance regular targets and long-distance small targets is processed respectively in a parallel network composed of a small-channel branch, a medium-channel branch and a large-channel branch, and multi-channel hierarchical depth features and predicted depth maps are output; View transformation module: used for fusing multi-channel hierarchical depth features and two-dimensional image features, and mapping to BEV space through a forward-backward paradigm based on LSS and BEVFormer to generate a preliminary BEV feature; a depth difference weighting module, configured to reconstruct the preliminary BEV feature in depth to obtain a reconstructed depth map, calculate a difference between the reconstructed depth map and a predicted depth map, and generate a depth difference map; transform the depth difference map to a BEV space to generate an auxiliary BEV feature; and finally, weight and fuse the preliminary BEV feature and the auxiliary BEV feature to generate a BEV feature weighted by the difference; a false positive noise reduction module, configured to construct an uncertainty map according to a real depth map and a predicted depth map, screen out voxels with high uncertainty and empty real occupancy labels as negative samples, and perform depth perturbation sampling on the negative samples; then map the negative sample information to a BEV space through a multi-layer perception and a Gaussian scattering projection to generate a noise reduction BEV feature; a feature fusion module, configured to fuse the BEV feature weighted by the difference and the noise reduction BEV feature to generate a final BEV feature; an occupancy prediction head module, configured to upsample the final BEV feature and predict a dense three-dimensional occupancy grid and a semantic category thereof.

Images