Point cloud noise identification method and device, and unmanned vehicle

Abstract

The present disclosure provides a point cloud noise identification method, device and unmanned vehicle, comprising: performing feature identification on the point cloud data of the current frame to obtain semantic point cloud; performing rasterization processing on the semantic point cloud, and removing the grid unit representing the dynamic target category according to the category to which each point in the semantic point cloud belongs, to determine the target semantic grid data of the current frame; fusing the target semantic grid data of the current frame with the target semantic grid data of the historical frame to generate terrain data; and checking the point cloud data of the current frame according to the terrain data to determine the noise of the point cloud data of the current frame.

Description

Technical Field

[0001] This disclosure relates to the fields of smart mining, autonomous driving, and vehicle control technology, and in particular to a method, device, and unmanned vehicle for identifying point cloud noise. Background Technology

[0002] In the construction of smart mines, radar is often used for environmental perception and the creation of high-precision maps. For example, radar is applied in key scenarios such as mine terrain data modeling, retaining wall structure inspection, and unmanned vehicle target recognition. However, the complex and harsh working environment of mines can easily cause serious interference to radar detection signals, carrying a large amount of noise in the generated point cloud data, significantly reducing the effectiveness and reliability of the point cloud data. For example, dust generated by excavation, transportation, or strong winds can scatter or reflect radar detection signals, creating dust noise. In winter operations, due to the low surface temperature and the high heat carried by the underground soil generated by excavation, the temperature difference causes water molecules in the air to liquefy, causing refraction interference to radar detection signals and forming thermal noise. In severe weather conditions such as fog, rain, and snow, a large amount of water vapor or solid particulate matter suspended in the air can obstruct the propagation path of radar detection signals, causing signal attenuation and distortion, forming severe weather noise.

[0003] Because point cloud data carries the aforementioned noise, it can negatively impact multiple aspects of intelligent mining operations. For example, during terrain data modeling, dust or heat floating above retaining walls can cause abnormal height values ​​in the point cloud data of the retaining wall area, resulting in distortion and contamination of the terrain data. During retaining wall structure inspection, noise can interfere with the judgment of parameters such as the actual height and flatness of the retaining wall, affecting the accuracy of the inspection results. In addition, noise can easily be misidentified as an obstacle by unmanned vehicles, triggering accidental braking operations, which not only reduces transportation efficiency but may also pose safety risks. Summary of the Invention

[0004] This disclosure provides a method, apparatus, and unmanned vehicle for identifying point cloud noise, in order to solve the problem that noise in existing point cloud data cannot be accurately identified.

[0005] In view of the above problems, firstly, the present disclosure provides a method for identifying point cloud noise, comprising: Perform feature recognition on the point cloud data of the current frame to obtain a semantic point cloud; The semantic point cloud is rasterized, and the raster units representing the dynamic target category are removed according to the category to which each point in the semantic point cloud belongs, so as to determine the target semantic raster data of the current frame. The target semantic raster data of the current frame is fused with the target semantic raster data of the historical frames to generate terrain data; The point cloud data of the current frame is verified based on the terrain data to determine the noise of the point cloud data of the current frame.

[0006] In conjunction with the first aspect, in one possible implementation, fusing the target semantic raster data of the current frame with the target semantic raster data of the historical frames to generate terrain data includes: For each raster cell in the target semantic raster data of the current frame, determine the probability coefficient corresponding to the semantic information of each raster cell; The target probability coefficient is determined based on the probability coefficient of the current frame raster cell and the probability coefficient of the corresponding raster cell in the target semantic raster data of the historical frame. Based on the target probability coefficient and the preset probability coefficient threshold, the grid cell is determined from the grid cell; The grid cells are scaled to generate terrain data.

[0007] In conjunction with the first aspect, in one possible implementation, the semantic information of the grid cell includes: ground and static obstacles; For each raster cell in the target semantic raster data of the current frame, determine the probability coefficient corresponding to the semantic information of each raster cell, including: For each grid cell in the target semantic grid data of the current frame, the following steps are performed: if the semantic information of the grid cell is ground, use the first preset coefficient as the probability coefficient of the grid cell; if the semantic information of the grid cell is a static obstacle, use the second preset coefficient as the probability coefficient of the grid cell. Preferably, determining the grid cell from the grid cells based on the target probability coefficient and the preset probability coefficient threshold includes: when the target probability coefficient satisfies the first preset probability coefficient threshold, using the corresponding grid cell as the grid cell with semantic information of a static obstacle; and when the target probability coefficient satisfies the second preset probability coefficient threshold, using the corresponding grid cell as the grid cell with semantic information of the ground. Preferably, determining the target probability coefficient based on the probability coefficient of the current frame raster unit and the probability coefficient of the corresponding position raster unit in the target semantic raster data of the historical frame includes: summing the probability coefficient of the current frame raster unit and the probability coefficient of the corresponding position raster unit in the target semantic raster data of the historical frame according to a preset operation to determine the target probability coefficient. Preferably, the scaling process of the grid cells to generate terrain data includes: The grid cells whose semantic information is static obstacles and the grid cells whose semantic information is ground are scaled to generate terrain data.

[0008] In conjunction with the first aspect, in one possible implementation, fusing the target semantic raster data of the current frame with the target semantic raster data of the historical frames to generate terrain data includes: For each raster cell in the target semantic raster data of the current frame, determine the height value of the raster cell; Multiply the height value of the current frame raster cell by the preset height coefficient of the current frame raster cell, and add the product of the height value of the raster cell at the corresponding position in the target semantic raster data of the historical frame and the preset height coefficient of the historical frame raster cell to determine the target height value of the raster cell. Terrain data is generated based on the target height value of the grid cell and the grid cell itself.

[0009] In conjunction with the first aspect, in one possible implementation, feature recognition is performed on the point cloud data of the current frame to obtain a semantic point cloud, including: The point cloud data of the current frame is processed into regularized features to generate regularized feature maps. The regularized feature map is input into a preset segmentation model to determine the probability distribution of the category to which each point in the point cloud data of the current frame belongs; The semantic point cloud is determined based on the probability distribution of the category to which each point belongs in the point cloud data of the current frame; Preferably, the preset segmentation model is obtained in the following manner: The sample point cloud data is processed by rules to obtain training samples; and the sample point cloud data is labeled to obtain semantic point cloud ground truth. The training samples are input into the initial segmentation model for feature recognition, resulting in a semantic point cloud generated during training. Based on the loss between the semantic point cloud generated during training and the ground truth value of the semantic point cloud, the segmentation model parameters are updated to obtain the preset segmentation model.

[0010] In conjunction with the first aspect, in one possible implementation, the point cloud data of the current frame is subjected to regularization processing to generate a regularized feature map, including: The point cloud data of the current frame is transformed into a two-dimensional distance map through spherical projection; the two-dimensional distance map is then subjected to data augmentation and normalization processing to output a first regularized feature map; and / or, The point cloud data of the current frame is converted into a voxel grid by voxelization, and the voxel grid is subjected to data augmentation and normalization processing to output a second regularized feature map. The first regularized feature map and the second regularized feature map include depth information and / or reflection intensity information.

[0011] Preferably, the preset segmentation model includes: an encoder module and a decoder module; the encoder module includes: a convolutional neural network and a Transformer encoder, and the convolutional neural network includes: a 2D convolutional neural network and / or a 3D convolutional neural network; The step of inputting the regularized feature map into a preset segmentation model to determine the probability distribution of the category to which each point in the point cloud data of the current frame belongs includes: If the generated regularized feature map is a first regularized feature map, the first regularized feature map is input into a 2D convolutional neural network for convolution operation; and / or, if the generated regularized feature map is a second regularized feature map, the second regularized feature map is input into a 3D convolutional neural network for convolution operation; The output features obtained from the convolution operation are input into the Transformer encoder to obtain a high-dimensional feature map; The high-dimensional feature map is input, deconvolved or interpolated, and then fused with the intermediate layer features obtained from the convolution operation. The fused features are then input to the decoder module for decoding, and the output layer outputs the probability distribution of the category to which each point in the current frame's point cloud data belongs.

[0012] In conjunction with the first aspect, in one possible implementation, the step of rasterizing the semantic point cloud, and removing raster units representing dynamic target categories based on the category to which each point in the semantic point cloud belongs, to determine the target semantic raster data of the current frame, includes: Based on multiple preset grid scales, the semantic point cloud is projected onto a preset grid space of corresponding resolution to obtain multiple sets of grid unit data; For each raster cell in each group of raster cell data, determine the semantic attributes of the raster cell based on the probability distribution of the category to which each point in the raster cell data belongs; Based on the raster unit semantic attributes determined from multiple sets of raster unit data, raster units representing dynamic target categories are removed to determine the target semantic raster data for the current frame.

[0013] In conjunction with the first aspect, in one possible implementation, the method further includes: If the current frame is the first frame, generate the first frame terrain data based on the target semantic raster data of the current frame; The point cloud data of the current frame is verified based on the terrain data of the first frame to identify and delete noise in the point cloud data of the current frame.

[0014] In conjunction with the first aspect, in one possible implementation, the step of verifying the point cloud data of the current frame based on the terrain data to determine the noise of the point cloud data of the current frame includes: Points in the current frame's point cloud data whose locations are outside the terrain data are marked as noise in the current frame's point cloud data; and / or, Points in the current frame's point cloud data with height values ​​higher than those in the terrain data are marked as noise in the current frame's point cloud data.

[0015] Secondly, a point cloud noise identification device is provided, comprising: The semantic segmentation module is used to perform feature recognition on the point cloud data of the current frame to obtain semantic point cloud; The dynamic target removal module is used to perform rasterization processing on the semantic point cloud, and remove the raster units that represent the dynamic target category according to the category to which each point in the semantic point cloud belongs, so as to determine the target semantic raster data of the current frame. The semantic raster data fusion module is used to fuse the target semantic raster data of the current frame with the target semantic raster data of the historical frames to generate terrain data; The noise recognition module is used to verify the point cloud data of the current frame based on the terrain data and determine the noise of the point cloud data of the current frame.

[0016] Thirdly, an unmanned vehicle is provided, including: a point cloud noise identification device as described in the second aspect.

[0017] The beneficial effects of the embodiments disclosed herein include: This disclosure provides a method, apparatus, and unmanned vehicle for identifying point cloud noise, comprising: performing feature recognition on point cloud data of the current frame to obtain a semantic point cloud; performing rasterization processing on the semantic point cloud, and removing raster units representing dynamic target categories according to the category to which each point in the semantic point cloud belongs, to determine the target semantic raster data of the current frame; fusing the target semantic raster data of the current frame with the target semantic raster data of historical frames to generate terrain data; and verifying the point cloud data of the current frame based on the terrain data to determine the noise of the point cloud data of the current frame. The point cloud noise identification method provided in this disclosure removes dynamic target interference through point cloud feature recognition and rasterization processing, constructs an accurate terrain data benchmark by combining multi-frame fusion, and then reverse-verifies the point cloud of the current frame to accurately identify and remove noise, solving the problems of noise pollution, terrain modeling distortion, and perception interference of unmanned vehicles in complex mining environments. Removing raster units representing dynamic target categories reduces the impact of dynamic interference on terrain data modeling from the source. Multi-frame fusion compensates for single-frame data deviations, suppresses single-frame noise and recognition errors, and generates terrain data that can accurately reconstruct key parameters such as retaining wall height and ground slope, solving problems of terrain distortion and height anomalies. Reverse verification using the fused terrain data as a benchmark accurately distinguishes between real terrain and noise, reducing the rate of missed and false detections, providing a precise basis for noise removal, and ensuring the perception safety and inspection accuracy of autonomous vehicles. Attached Figure Description

[0018] Figure 1 A flowchart illustrating a point cloud noise identification method provided in this embodiment of the disclosure; Figure 2 This is a structural diagram of the point cloud noise identification device provided in an embodiment of this disclosure. Detailed Implementation

[0019] This disclosure provides a method, apparatus, and unmanned vehicle for identifying point cloud noise. Preferred embodiments of this disclosure are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustrative and explanatory purposes only and are not intended to limit the scope of this disclosure. Furthermore, the embodiments and features described herein can be combined with each other unless otherwise specified.

[0020] This disclosure provides a method for identifying point cloud noise, such as... Figure 1 As shown, it includes: S101. Perform feature recognition on the point cloud data of the current frame to obtain a semantic point cloud; S102. Perform rasterization processing on the semantic point cloud. Based on the category to which each point in the semantic point cloud belongs, remove the raster units that represent the category of the dynamic target to determine the target semantic raster data of the current frame. S103. Merge the target semantic raster data of the current frame with the target semantic raster data of the historical frames to generate terrain data; S104. Verify the point cloud data of the current frame based on the terrain data to determine the noise of the point cloud data of the current frame.

[0021] In this embodiment of the disclosure, radar (such as lidar) is an important device for environmental perception and high-precision mapping in the construction of smart mines, and is widely used in key scenarios such as mine terrain data modeling, retaining wall structure inspection, and unmanned vehicle target recognition. For example, a mechanical lidar uses a motor to drive the transceiver module to rotate and scan the surrounding environment to obtain point cloud data; another example is a solid-state lidar, which scans the target area in the forward or local field of view to obtain point cloud data. However, the complex and harsh working environment of mines can easily cause strong interference to radar detection signals, resulting in a large amount of noise in the generated point cloud data, which significantly reduces the effectiveness and reliability of the data. This type of noise mainly includes: dust noise caused by excavation, transportation operations or strong winds; hot air noise caused by water vapor liquefaction due to the temperature difference between the surface and underground soil in winter; and severe weather noise caused by water vapor or suspended particulate matter in the air during foggy, rainy, or snowy weather. These noises can adversely affect multiple aspects of intelligent mining operations: when modeling terrain data, dust and heat noise above the retaining wall can cause abnormal point cloud heights in the area, leading to terrain data distortion and contamination; during retaining wall structure inspections, noise can interfere with the judgment of parameters such as the actual height and flatness of the retaining wall, reducing the accuracy of the inspection; and unmanned vehicles are prone to misidentifying noise as obstacles and triggering false braking, which not only affects transportation efficiency but also easily causes operational safety risks.

[0022] In this embodiment, feature recognition is performed on the point cloud data of the current frame to obtain a semantic point cloud. For example, radar collects point cloud data of the current frame, and through feature extraction or semantic segmentation techniques, a unique semantic category label is assigned to each point, outputting a semantic point cloud carrying semantic information. The semantic category label is used to characterize the category to which each point belongs. Based on the mobility attributes of these categories, they can be divided into: static target categories and dynamic target categories. For example, static target categories may include: static obstacles such as ground and retaining walls, while dynamic target categories may include: workers, mining vehicles, dust noise, heat noise, and severe weather noise. The semantic point cloud of the current frame is rasterized to obtain raster cells. Rasterization can be based on a preset raster scale, projecting the semantic point cloud onto a preset raster space to obtain raster cells. By statistically analyzing the frequency of the category to which each point belongs within the raster cell, for example, using the category with the highest frequency, the category to which the semantic information of the raster cell belongs is determined. By filtering raster cells of static target categories and removing raster cells of dynamic target categories, target semantic raster data that retains static semantic information is obtained, thus eliminating the interference of dynamic targets on terrain data modeling.

[0023] Furthermore, target semantic raster data from all historical frames or target semantic raster data from historical frames within a preset time window are retrieved. The target semantic raster data from historical frames retains raster units of static target categories while removing raster units of dynamic target categories. The target semantic raster data from the current frame is fused with that from historical frames, for example, by performing consistency checks on the target semantic raster data to generate high-precision, high-stability terrain data. For example, the categories to which the semantic information of the same raster unit belongs are integrated to correct deviations in the semantic information of the raster unit. The fused target semantic raster data is then subjected to 3D terrain transformation to generate structured terrain data including terrain elevation and semantic boundaries. Using the terrain data as a benchmark, the point cloud data of the current frame is back-checked, for example, by comparing the coordinates and elevation information of each point in the point cloud data with the differences in the terrain data, filtering out points exceeding a threshold range as noise, thereby accurately identifying and marking noise in the current frame's point cloud data and providing a basis for subsequent noise removal and point cloud data correction.

[0024] This application embodiment addresses issues such as noise pollution, terrain data modeling distortion, and unmanned vehicle perception interference in complex mining environments by performing feature recognition and rasterization on point cloud data to eliminate interference from dynamic target categories. A reliable terrain data benchmark is then constructed based on multi-frame fusion to reverse-verify the point cloud data of the current frame, identifying and removing noise. By eliminating raster units representing dynamic target categories, the impact of dynamic interference on terrain data modeling is reduced at the source. Multi-frame target semantic raster data fusion compensates for local biases and insufficient information in single-frame point cloud data, effectively suppressing the impact of single-frame noise and inaccurate semantic recognition. The generated terrain data possesses both real-time performance and reliability, accurately reconstructing key parameters such as retaining wall height and ground slope, resolving issues of abnormal point cloud height and terrain data distortion in mining retaining wall areas. Using the multi-frame fused terrain data as a benchmark for reverse-verification of point cloud data accurately distinguishes between real terrain and noise, reducing false positives and false negatives, providing a precise basis for subsequent noise removal, and ensuring the perception safety and inspection accuracy of unmanned vehicles.

[0025] In another embodiment of this disclosure, step S103 above, which involves fusing the target semantic raster data of the current frame with the target semantic raster data of historical frames to generate terrain data, includes the following steps: Step 1: For each raster cell in the target semantic raster data of the current frame, determine the probability coefficient corresponding to the semantic information of each raster cell; Step 2: Determine the target probability coefficient based on the probability coefficient of the current frame raster cell and the probability coefficient of the corresponding raster cell in the target semantic raster data of the historical frame; Step 3: Determine the grid cells from the grid cells based on the target probability coefficient and the preset probability coefficient threshold; Step 4: Scale the grid cells to generate terrain data.

[0026] In this embodiment, based on the probability coefficients of the semantic information of each grid cell, accurate fusion of target semantic grid data of the current frame and historical frames is achieved. The credibility of the semantic information of the grid cells is quantified by the probability coefficients, and high-credibility grid cells are selected. Then, the 3D terrain is restored through scaling, ensuring that the generated terrain data possesses semantic accuracy, spatial stability, and precision adaptability, providing a reliable benchmark for subsequent point cloud data verification and noise reduction. For step 1 above, for the target semantic grid data of the current frame, the probability coefficients corresponding to the semantic information of each grid cell are determined. The semantic information of the grid cells in the target semantic grid data can include ground and static obstacles. Ground can be unobstructed, large-area, continuously distributed static terrain such as natural ground or hardened road surfaces in a mining operation area. Static obstacles can include retaining walls and fixed facilities, where retaining walls correspond to wall structures for mine slope protection and area isolation, and fixed facilities correspond to immovable static structures such as drilling platforms, equipment bases, and fixed markers. Preset probability coefficients for ground and static obstacles are provided respectively. For example, a grid cell with semantic information representing the ground has a probability coefficient of -10, while a grid cell with semantic information representing a static obstacle has a probability coefficient of 10. Regarding step 2 above, the probability coefficients of grid cells at corresponding positions in the current frame and historical frames are collaboratively calculated to generate target probability coefficients for each fused grid cell, achieving information complementarity across multiple frames. The target semantic grid data of the current frame and the target semantic grid data of historical frames are spatially aligned to ensure accurate matching of grid cells at the same physical location. The probability coefficients of grid cells at corresponding positions in each historical frame are extracted. A weighted fusion algorithm can be used to calculate the target probability coefficient by combining the probability coefficient of the current frame with the weighted probability coefficient of the historical frames. For example, the target probability coefficient = α × current frame probability coefficient + (1-α) × historical frame probability coefficient. Alternatively, the probability coefficients of the current frame and the historical frames can be summed to obtain the target probability coefficient. Regarding step 3 above, the target probability coefficients are filtered based on a preset threshold, eliminating low-confidence non-fitting grid cells and retaining high-confidence fitting grid cells to ensure the accuracy of subsequent terrain modeling. For example, for grid cells whose semantic information is the ground, their target probability coefficient should be less than -20; for grid cells whose semantic information is static obstacles, their target probability coefficient should be greater than 20; and grid cells with target probability coefficients between -20 and 20 are considered non-grid cells. In step 3 above, the grid cells are scaled and transformed using grid scale and grid space mapping rules to restore them to three-dimensional structured terrain data, adapting to the needs of mine terrain modeling.Based on a preset grid scale (physical size of grid cells and overall size), the scaling ratio between the grid cells and the 3D physical space is determined, as well as the 3D spatial range corresponding to each grid cell (e.g., the horizontal scaling ratio corresponds to the grid cell width, and the vertical scaling ratio corresponds to the terrain elevation accuracy). 3D coordinate transformation is performed based on the UV coordinates, semantic information, and target probability coefficients of the grid cells, combined with the scaling ratio. For example, the horizontal and vertical pixel coordinates of the grid cells are scaled to X and Y coordinates in 3D space. The Z coordinate is determined based on the elevation statistics corresponding to semantic information (e.g., static obstacles, ground), and the elevation value is corrected by combining the target probability coefficient; for example, the higher the target probability coefficient, the greater the weight of the elevation value. The 3D coordinates of all grid cells are integrated, redundant coordinate information is removed, and the terrain contour is smoothed (e.g., the elevation difference between adjacent grid cells is corrected to avoid abrupt changes), generating structured terrain data containing terrain elevation, semantic boundaries of static facilities, and spatial contours. The data format is adapted to subsequent point cloud verification requirements. By quantizing probability coefficients, the semantic information of raster units is transformed into calculable quantitative indicators, replacing the qualitative judgment of traditional fusion. The collaborative calculation of probability coefficients across multiple frames enables differentiated fusion of multi-frame data, effectively compensating for the semantic bias of single-frame raster units. This significantly improves the credibility of the semantic information of the merged raster units and avoids low-credibility data interfering with terrain data modeling.

[0027] In another embodiment of this disclosure, the semantic information of the grid cell includes: ground and static obstacles; In step 1 above, for each raster cell in the target semantic raster data of the current frame, the probability coefficient corresponding to the semantic information of each raster cell is determined, including: For each grid cell in the target semantic grid data of the current frame, the following steps are performed: if the semantic information of the grid cell is ground, use the first preset coefficient as the probability coefficient of the grid cell; if the semantic information of the grid cell is a static obstacle, use the second preset coefficient as the probability coefficient of the grid cell. Preferably, in step 3 above, determining the grid cell from the grid cells based on the target probability coefficient and the preset probability coefficient threshold includes: when the target probability coefficient meets the first preset probability coefficient threshold, the corresponding grid cell is used as the grid cell with the semantic information of a static obstacle; when the target probability coefficient meets the second preset probability coefficient threshold, the corresponding grid cell is used as the grid cell with the semantic information of the ground. Preferably, in step 2 above, determining the target probability coefficient based on the probability coefficient of the current frame raster unit and the probability coefficient of the corresponding position raster unit in the target semantic raster data of the historical frame includes: summing the probability coefficient of the current frame raster unit and the probability coefficient of the corresponding position raster unit in the target semantic raster data of the historical frame according to a preset operation to determine the target probability coefficient. Preferably, in step 4 above, the grid cells are scaled to generate terrain data, including: The grid cells with semantic information of static obstacles and the grid cells with semantic information of ground are scaled to generate terrain data.

[0028] In this embodiment, by defining the probability coefficient configuration rules for the ground and static obstacles, and combining them with optimized summation and fusion, preset probability coefficient threshold filtering, and grid cell scaling, accurate fusion and 3D terrain reconstruction of multi-frame target semantic raster data are achieved. This ensures that the generated terrain data is semantically accurate and spatially stable, providing a reliable benchmark for subsequent point cloud verification and noise reduction. The semantic information of the raster cells includes the ground and static obstacles. For each raster cell in the target semantic raster data of the current frame, the following probability coefficient assignment operation is performed: If the semantic information of the grid cell is ground, the first preset coefficient is used as its probability coefficient; If the semantic information of the grid cell is a static obstacle, the second preset coefficient is used as its probability coefficient; The first and second preset coefficients satisfy the following condition: when the first and second preset coefficients are summed, the trends of the summed results are opposite. For example, the first and second preset coefficients must have opposite signs (e.g., the first preset coefficient is set to -10, and the second preset coefficient is set to 10). By differentiating the signs and values, the semantics of ground and static obstacles can be quickly distinguished. This assignment algorithm is simple and efficient, requiring no complex feature calculations, and can adapt to the data processing needs of real-time mining operations.

[0029] By employing a pre-defined summation method, the probability coefficients of corresponding raster cells in the current frame and historical frames are fused and calculated to generate target probability coefficients. This enables complementary superposition of semantic information from multiple frames, enhancing data stability. The target semantic raster data of the current frame is spatially aligned with the target semantic raster data of historical frames. Relying on the UV coordinate system of the pre-defined raster space, precise matching of raster cells at the same physical location is ensured, avoiding fusion errors caused by spatial misalignment.

[0030] The probability coefficients of the current frame's grid cells and the corresponding grid cells in historical frames are summed according to preset calculation rules to obtain the target probability coefficient of that grid cell. For example: if any static obstacle grid cell in the current frame (second preset coefficient 10) corresponds to two historical frames where the same position is also a static obstacle grid cell (total probability coefficient 10 + 10 = 20), then the target probability coefficient = 10 + 20 = 30; if any ground grid cell in the current frame (first preset coefficient -10) corresponds to two historical frames where the same position is also a ground grid cell (total probability coefficient (-10) + (-10) = (-20)), then the target probability coefficient = -10 + (-20) = -30. However, if any static obstacle grid cell in the current frame (second preset coefficient 10) corresponds to two historical frames where the same position is also a ground grid cell (total probability coefficient (-10) + (-10) = (-20)), then the target probability coefficient = 10 + (-20) = -10. It has low computational load, strong real-time performance, and can enhance the cumulative effect of semantic credibility and suppress single-frame noise interference by superimposing coefficients from multiple frames.

[0031] Threshold rules are set separately for ground and static obstacle semantics to accurately select high-confidence grid cells and eliminate low-confidence data, ensuring the accuracy of terrain modeling. A first preset probability coefficient threshold corresponds to static obstacle semantics, and a second preset probability coefficient threshold corresponds to ground semantics. If the target probability coefficient of a grid cell meets the first preset probability coefficient threshold (e.g., target probability coefficient > 20, adapting to the summation logic of the second preset coefficient 10), then the grid cell is determined to be a grid cell with semantic information of a static obstacle. If the target probability coefficient of a grid cell meets the second preset probability coefficient threshold (e.g., target probability coefficient < -20, adapting to the summation logic of the first preset coefficient -10), then the grid cell is determined to be a grid cell with semantic information of ground. If the target probability coefficient does not meet the corresponding category threshold (e.g., within the range of -20 to 20), it is determined to be a non-grid cell and is eliminated. By accurately matching the classification threshold with semantic categories and probability coefficients, low-confidence raster units caused by semantic misjudgment in a single frame or noise interference can be precisely filtered out, while avoiding confusion between different semantic categories and further improving the purity of the fused data.

[0032] For grid cells representing static obstacles and the ground, scaling is performed separately. Semantic information is then used to optimize 3D coordinate transformation, generating structured terrain data suitable for mine terrain modeling and inspection needs. Based on preset grid scales (grid cell physical dimensions and overall dimensions), the scaling ratio between the grid cells and the 3D physical space is determined. For example, the horizontal scaling ratio corresponds to the grid cell width, and the vertical scaling ratio corresponds to the terrain elevation accuracy, ensuring that the scaled data is consistent with the actual physical space of the mine. For grid cells representing static obstacles: their UV coordinates are converted to 3D X and Y coordinates based on the horizontal scaling ratio; the initial Z coordinate is determined based on typical elevation statistics of static obstacles (retaining walls, fixed facilities), and then corrected using target probability coefficients (the higher the absolute value of the target probability coefficient, the greater the elevation weight; for example, a target probability coefficient of 30 for a retaining wall grid cell compared to 25 increases the elevation weight, resulting in a more accurate match to the actual height). For grid cells whose semantic information is the ground: similarly transform the X and Y coordinates, determine the Z coordinate based on the statistical value of the ground reference elevation, and combine the target probability coefficient for correction (e.g., the ground elevation stability weight is increased when the target probability coefficient is -30 compared to -25, to avoid local fluctuation errors). The probability coefficient assignment is concise and efficient, with high semantic discriminative power. Using fixed preset coefficients (first and second preset coefficients) eliminates the need for complex feature extraction and calculation, significantly reducing data processing computational overhead and adapting to the real-time operation requirements of mines. Simultaneously, the design of opposite positive and negative coefficients enables rapid quantitative differentiation of the semantics of ground and static obstacles, providing a clear basis for subsequent classification processing. The summation operation has low computational load and simple logic, enabling rapid multi-frame data fusion. The accumulation and superposition of preset coefficients across multiple frames strengthens the temporal consistency of semantic information, effectively suppressing semantic deviations caused by single-frame noise. Compared to weighted fusion, it is more suitable for the high real-time requirements of mine scenarios. Preset probability coefficient thresholds for ground and static obstacles specifically filter low-confidence raster units of different semantic information categories, avoiding cross-category mis-screening and interference from low-confidence data. Compared to single-threshold screening, this significantly improves the semantic accuracy of grid merging, laying the foundation for high-precision terrain modeling. By scaling and correcting the semantics of ground and static obstacles separately, and optimizing the elevation values ​​by combining the target probability coefficient, the system can accurately restore the ground benchmark elevation and the true height of static obstacles (retaining walls, fixed facilities), solve the problem of elevation anomalies caused by noise in the retaining wall area of ​​the mine, and optimize the terrain smoothness at the junction of the ground and obstacles, thereby improving the practicality of the terrain data.

[0033] In another embodiment of this disclosure, step S103 above, which involves fusing the target semantic raster data of the current frame with the target semantic raster data of historical frames to generate terrain data, includes the following steps: Step 1: For each raster cell in the target semantic raster data of the current frame, determine the height value of the raster cell; Step 2: Multiply the height value of the current frame raster cell by the preset height coefficient of the current frame raster cell, and add the product of the height value of the raster cell at the corresponding position in the target semantic raster data of the historical frame and the preset height coefficient of the historical frame raster cell to determine the target height value of the raster cell. Step 3: Generate terrain data based on the target height value of the raster cells and the raster cells themselves.

[0034] In this embodiment, the height value of the current frame's raster unit is accurately extracted, and a weighted summation algorithm is used to calculate the target height value. This is then combined with the raster unit itself to generate terrain data. This achieves refined fusion of the height dimension, taking into account both real-time terrain features and the stability of historical data. It ensures that the generated terrain data has accurate elevation and spatial continuity, providing a reliable height value benchmark for subsequent point cloud verification and noise reduction, and adapting to needs such as mine retaining wall height detection and terrain modeling. The three-dimensional elevation information corresponding to each raster unit in the target semantic raster data of the current frame (after removing dynamic targets) is extracted, providing basic quantitative data for inter-frame height fusion and ensuring accurate correspondence between height values ​​and semantic categories and spatial locations. Specifically, in step one above, the height value corresponding to each raster unit in the target semantic raster data of the current frame (after removing dynamic targets) is extracted, providing standardized basic quantitative data for subsequent inter-frame height fusion and ensuring accurate correspondence between height values ​​and the spatial location of raster units. The algorithm iterates through all raster cells in the current frame's target semantic raster data, locking the 3D spatial region corresponding to each raster cell based on the UV coordinate system of the preset raster space. For all raster cells, the same height value statistical rules are applied, extracting the Z-coordinate feature values ​​of all associated point cloud data within that raster cell as the height value. Preferably, the average Z-coordinate is used. If the point cloud data within a raster cell is sparse, height values ​​from adjacent raster cells can be interpolated to fill in the missing height values. The calculated height values ​​are then bound to the corresponding raster cells, laying the data foundation for subsequent inter-frame weighted calculations. In step two above, the current frame's target semantic raster data is spatially aligned with historical frame target semantic raster data, using the UV coordinate system of the preset raster space to ensure accurate matching of raster cells at the same physical spatial location, avoiding height fusion errors caused by spatial misalignment. The height coefficients of the current frame raster units and the historical frame raster units are preset separately. These can be flexibly set according to the real-time and stability requirements of the mining operation scenario (e.g., the height coefficient of the current frame raster unit is 0.6, and the height coefficient of the historical frame is 0.4). For each pair of precisely matched current and historical frame raster units, a unified weighted summation operation is performed. The calculation formula is: Target height value = (Current frame raster unit height value × Preset height coefficient of the current frame raster unit) + (Height value of the raster unit at the corresponding position in the historical frame × Preset height coefficient of the historical frame raster unit). The calculated target height value is bound to the corresponding raster unit, and a target height value is generated for each raster unit, completing the fusion of height information from multiple frames. In step three above, the fused target height value and the raster units are deeply integrated. Through the mapping and transformation between the spatial coordinates of the raster units and the target height value, structured three-dimensional terrain data is generated, ensuring the spatial continuity and elevation accuracy of the terrain data.Based on a preset grid scale (including the physical dimensions of grid cells and the overall grid size), each grid cell is matched with corresponding 3D physical space X and Y coordinates. The target height value is used as the 3D physical space Z coordinate of the grid cell, realizing the transformation of the grid cell from 2D UV coordinates to 3D X, Y, Z coordinates. The 3D coordinates (X, Y, Z) of all grid cells are summarized, and redundant coordinate information is removed to ensure that each 3D spatial location corresponds to clear elevation information. The 3D coordinates of all integrated grid cells are smoothed, and the difference is corrected for abrupt changes in target height values ​​between adjacent grid cells to ensure that the terrain contour is continuous and natural. Finally, 3D structured terrain data with accurate elevation information based on grid cells is generated, and the data format is adapted to the needs of subsequent point cloud verification and noise reduction. Semantic information such as ground and static obstacles are not distinguished. A unified height value extraction, inter-frame fusion, and modeling rules are adopted, eliminating the cumbersome steps of semantic category judgment and classification processing, simplifying the algorithm logic, reducing computing power consumption, and improving the overall processing efficiency of grid fusion and terrain generation, adapting to the timeliness requirements of real-time perception and dynamic terrain mapping for unmanned vehicles in mines. By weighting and summing the height coefficients of the current frame and historical frames, the system retains the real-time characteristics of the terrain height in the current frame, accurately capturing subtle changes in the mine terrain. Simultaneously, it uses historical frame height data to suppress height anomalies caused by noise interference from dust, heat, fog, rain, and snow in single-frame point clouds, effectively improving the reliability of target height values ​​and preventing single-frame noise from affecting terrain modeling accuracy. The generated terrain data accurately restores the true elevation characteristics of the mine terrain and can be directly used as a unified benchmark for subsequent point cloud data elevation verification. It can accurately identify elevation anomalies caused by noise in the point cloud data, improving the accuracy of subsequent point cloud noise reduction and providing precise data support for mine retaining wall height detection, terrain mapping, and other operations.

[0035] In another embodiment of this disclosure, step S101 above, which involves performing feature recognition on the point cloud data of the current frame to obtain a semantic point cloud, includes: Step (1): Perform regularization processing on the point cloud data of the current frame to generate a regularized feature map; Step (2): Input the regularized feature map into the preset segmentation model to determine the probability distribution of the category of each point in the point cloud data of the current frame; Step (3): Determine the semantic point cloud based on the probability distribution of the category to which each point belongs in the point cloud data of the current frame; Preferably, the preset segmentation model is obtained in the following manner: Step (1): Perform regularization processing on the sample point cloud data to obtain training samples; and label the sample point cloud data to obtain semantic point cloud ground truth. Step (2): Input the training samples into the initial segmentation model for feature recognition to obtain the semantic point cloud generated by the training. Step (3): Update the segmentation model parameters based on the loss between the semantic point cloud generated during training and the ground truth of the semantic point cloud to obtain the preset segmentation model.

[0036] In this embodiment, unstructured point cloud data is transformed into a regularized feature map adapted to the input of a preset segmentation model through regularization processing. The preset segmentation model, after training and optimization, accurately outputs the category probability distribution, and finally determines the semantic point cloud, providing high-quality semantic data support for subsequent steps such as rasterization de-dynamics and multi-frame fusion. For the above step (1), the current frame point cloud data is pre-processed for denoising, removing isolated noise points, correcting abnormal coordinate points, and retaining valid point cloud data; and coordinate normalization processing is performed to map the point cloud coordinates to a unified numerical range, avoiding inconsistencies in feature scales caused by differences in the mining operation area. Using a rasterization mapping method, based on a preset raster scale, the three-dimensional unstructured point cloud is projected onto a two-dimensional plane or a three-dimensional voxel grid to generate a regularized feature map. By statistically analyzing the geometric and physical features of the point cloud within each raster unit or voxel, such as the mean Z-coordinate, point cloud density, spatial dispersion, and reflection intensity, the features are integrated into the pixel value or voxel value of the feature map, ensuring that the regularized feature map has both spatial location information and point cloud feature information. Output a fixed-size, structured, regularized feature map to fit the input requirements of the preset segmentation model. For step (2) above, train the optimized preset segmentation model, perform deep feature extraction and category prediction on the regularized feature map, and output the probability distribution of each point cloud corresponding to each category to achieve quantitative judgment of semantic category. Input the regularized feature map into the preset segmentation model, extract the local and global features of the point cloud, and output the category probability distribution of each grid cell or voxel corresponding to the point cloud based on the feature mapping result. That is, the probability value of each point belonging to each category, the category includes at least static target category and dynamic target category, to meet the subsequent semantic filtering requirements. For step 3 above, map the probability distribution of the category predicted by the preset segmentation model to the point cloud data, and each point carries the probability distribution information of its category. Set the judgment rules based on the probability distribution of the category, filter the final semantic category of each point, generate a semantic point cloud with a clear semantic category label, and realize the transformation from raw point cloud data to semantic point cloud.

[0037] For example, the semantic category of the point cloud is determined using the principle of maximum probability priority. That is, the category with the highest probability value in the probability distribution of each point is selected as the candidate category. If the probability value of the candidate category is greater than or equal to a preset threshold, then that category is determined as the final semantic category of the point; if the probability value is less than the preset threshold, it is determined to be a semantically ambiguous point and is either removed or marked as a point to be processed. Each point cloud is assigned a semantic category label (such as "ground," "retaining wall," "work vehicle," "dust," etc.), forming a correlation between 3D coordinates and semantic category labels. All point cloud data carrying semantic labels are integrated, semantically ambiguous points and invalid points are removed, and finally a structured semantic point cloud is generated.

[0038] By training, calculating loss, and iterating parameters using samples from mining scenarios, the initial segmentation model is optimized to obtain a preset segmentation model adapted to the complex point cloud features of mining, ensuring that the preset segmentation model has high-precision category probability prediction capabilities. For step (I) above, a large amount of multi-condition sample point cloud data from mining scenarios is collected (covering noise scenarios such as dust, heat, fog, rain, snow, etc., as well as point cloud data from different working periods and terrains). Regularization processing is performed on each batch of sample point cloud data to generate training samples (regularized feature maps), ensuring consistency between the training and inference processes. Furthermore, a combination of manual and automatic annotation can be used to accurately label the sample point cloud data with semantic category labels, forming semantic point cloud ground truths, which serve as the supervision basis for training the preset segmentation model. For step (II) above, the prepared training samples are divided into training and validation sets according to a preset ratio. The training set is input into the initial segmentation model, and the model performs feature recognition and semantic point cloud generation based on the training samples, outputting the trained semantic point cloud, completing one round of forward propagation. For step (iii) above, the loss between the semantic point cloud generated during training and the ground truth semantic point cloud is calculated. The cross-entropy loss function is preferred to quantify the deviation between the prediction result and the true label. Based on the loss value, the backpropagation algorithm (such as gradient descent) is used to iteratively update the convolutional kernel weights, biases, and other parameters of the initial segmentation model to reduce the prediction loss. The training, loss calculation, and parameter update process is repeated until the model's prediction accuracy on the validation set reaches a preset threshold (e.g., accuracy ≥ 90%), at which point the iteration stops, resulting in a preset segmentation model adapted to the mining scenario. Regularization processing addresses the disordered and sparse nature of the original point cloud data, generating a regularized feature map adapted to the input of the preset segmentation model. Training the optimized preset segmentation model can accurately extract complex features from the mining point cloud data, significantly improving the accuracy of category probability prediction. Training with sample point cloud data allows the preset segmentation model to adapt to noisy scenarios such as dust and heat, further enhancing the scenario adaptability of semantic classification.

[0039] In another embodiment of this disclosure, step (1) above, which involves performing regularization processing on the point cloud data of the current frame to generate a regularized feature map, includes: Step 1) Convert the point cloud data of the current frame into a 2D distance map using spherical projection; perform data augmentation and normalization on the 2D distance map to output the first regularized feature map; and / or, Step 2) Convert the point cloud data of the current frame into voxel grids through voxelization, perform data augmentation and normalization on the voxel grids, and output the second regularized feature map. The first regularized feature map and the second regularized feature map include depth information and / or reflection intensity information.

[0040] Preferably, the preset segmentation model includes: an encoder module and a decoder module; the encoder module includes: a convolutional neural network and a Transformer encoder, and the convolutional neural network includes: a 2D convolutional neural network and / or a 3D convolutional neural network; In step (2) above, the regularized feature map is input into the preset segmentation model to determine the probability distribution of the category to which each point in the point cloud data of the current frame belongs, including: Step 1) If the generated regularized feature map is a first regularized feature map, input the first regularized feature map into a 2D convolutional neural network for convolution operation; and / or, if the generated regularized feature map is a second regularized feature map, input the second regularized feature map into a 3D convolutional neural network for convolution operation; Step 2) Input the output features obtained from the convolution operation into the Transformer encoder to obtain a high-dimensional feature map; Step 3) After deconvolution or interpolation of the high-dimensional feature map input, it is fused with the intermediate layer features obtained by the convolution operation and input into the decoder module for decoding. The output layer outputs the probability distribution of the category to which each point in the current frame's point cloud data belongs.

[0041] In this embodiment, regularization is performed through spherical projection and / or voxelization, combining the local feature capture capability of convolution with the global correlation modeling advantage of Transformer. Through feature fusion and decoding, the output category probability distribution is preserved, retaining core information such as point cloud depth and reflection intensity, providing high-quality feature support for subsequent semantic judgment, and adapting to the point cloud semantic segmentation requirements in complex noisy mining environments. For step 1), spherical projection and voxelization (optional paths, which can be used individually or in combination) are employed to transform the 3D point cloud into a regularized feature map. Combined with data augmentation and normalization, key information such as point cloud depth and reflection intensity is preserved, eliminating data heterogeneity interference and adapting to the input requirements of subsequent segmentation models. The spherical projection transformation can be based on a spherical coordinate system (azimuth, elevation, and distance) of a radar sensor, projecting the current frame's 3D point cloud data onto a 2D plane to generate a 2D distance map. During projection, the azimuth angle is mapped to the width dimension of the two-dimensional image, and the pitch angle is mapped to the height dimension. The grayscale value of each pixel corresponds to the distance from the point cloud to the sensor (depth information). Furthermore, the reflection intensity information of the point cloud can be superimposed as an additional channel to form a multi-channel two-dimensional distance map, taking into account both depth and physical features. Data augmentation operations are performed on the two-dimensional distance map, including random flipping, rotation, and local cropping, to improve the model's generalization ability and adapt to point cloud data from different working angles and scenarios in the mine. Subsequently, normalization processing is performed to map the distance value and reflection intensity value to the [0,1] interval, eliminating numerical scale differences and avoiding feature weight imbalance caused by distance and reflection intensity fluctuations. Finally, a first regularized feature map carrying depth information and / or reflection intensity information is output. For step 2) above, voxelization can be performed as follows: Based on a preset voxel size, the 3D point cloud space is divided into uniform cubic voxel grids. Each point is traversed and assigned to the corresponding voxel unit. For each voxel unit, the depth information (mean or maximum value of Z coordinate), reflection intensity information (mean or variance), and point cloud density of the internal point cloud are statistically analyzed. The statistical values ​​are used as feature values ​​of the voxel grid to form a 3D voxel grid, which fully preserves the 3D spatial structure features of the point cloud. Data augmentation operations are performed on the voxel grid, including random translation, scaling, and voxel shuffling, to enhance the model's adaptability to subtle changes in mine terrain and sparse point cloud areas. Then, normalization is performed to unify the numerical range of voxel feature values ​​and correct the problem of excessive differences in features between voxels. Finally, a second regularized feature map carrying depth information and / or reflection intensity information is output.

[0042] Regarding step one above, if the input is a first regularized feature map (two-dimensional), it is fed into a 2D convolutional neural network. Through multiple rounds of 2D convolution, BatchNorm, and activation function (such as ReLU) operations, two-dimensional local features are extracted, and the corresponding dimensional convolutional features are output. If the input is a second regularized feature map (three-dimensional), it is fed into a 3D convolutional neural network. Through multiple rounds of 3D convolution operations, three-dimensional spatial correlation features between voxels are extracted, and three-dimensional convolutional features are output. If both the first and second regularized feature maps are input simultaneously, features can be extracted by 2D and 3D convolutional neural networks respectively, and then concatenated to form cross-dimensional fusion features, balancing two-dimensional efficiency and three-dimensional structural information. Regarding step two above, the features output from the convolution operation are input into a Transformer encoder. Through a self-attention mechanism, global correlation weights between features are calculated, effective features are strengthened, and noisy features are suppressed. Local features and global correlations are fused to generate a high-dimensional feature map that combines detail and global perspective, improving semantic category discrimination ability. For step three above, the high-dimensional feature map is improved in resolution through deconvolution (adapting to 2D features) or 3D deconvolution (adapting to 3D features) and interpolation. This is then element-wise added to or fused with the intermediate layer features obtained during the convolution operation in step one (preserving more detailed information), supplementing the detailed features for semantic segmentation and correcting the resolution loss of the high-dimensional feature map. The fused features are input to the decoder module, and the feature map size is gradually restored through multiple rounds of decoding. Finally, the output layer (such as a Softmax layer) calculates the probability value for each category of each point cloud, outputting the probability distribution. The output category probability distribution is mapped to the original point cloud data, with each point cloud carrying complete probability distribution information, providing a quantitative basis for subsequent semantic point cloud judgment. The convolutional network accurately captures local detailed features, while the Transformer encoder models global correlations. The fusion of these two approaches solves the problem of traditional models being accurate locally but lacking global perspective, effectively distinguishing mine noise (dust, heat) from real targets and improving semantic classification accuracy. The fusion of high-dimensional feature maps with intermediate layer features corrects resolution loss, supplements semantic boundary details (such as retaining wall edges and ground undulations), avoids ambiguity in semantic segmentation, ensures the accuracy of class probability prediction, and provides a reliable basis for subsequent semantic point cloud determination. High-dimensional feature maps are condensed features processed layer by layer by convolution and Transformer encoders; their size is much smaller than the initial regularized feature map, and they cannot directly correspond one-to-one with the spatial locations of the original point cloud. Deconvolution (transposed convolution) expands the feature map size through upsampling, and interpolation (such as bilinear interpolation and trilinear interpolation) completes pixel / voxel information through feature value fitting. Both can accurately restore the resolution of the high-dimensional feature map to a scale that matches the intermediate layer features of the convolution and the initial regularized feature map, ensuring that the subsequent output class probability distribution can achieve accurate spatial binding with each point of the current frame's point cloud data, avoiding semantic classification misalignment caused by scale mismatch.

[0043] In another embodiment of this disclosure, in step S102 above, the semantic point cloud is rasterized, and raster units representing the dynamic target category are removed according to the category to which each point in the semantic point cloud belongs, so as to determine the target semantic raster data of the current frame, including: Step 01: Based on multiple preset grid scales, project the semantic point cloud onto the preset grid space of the corresponding resolution to obtain multiple sets of grid cell data; Step 02: For each raster cell in each group of raster cell data, determine the semantic attributes of the raster cell based on the probability distribution of the category to which each point belongs within the raster cell data. Step 03: Based on the raster unit semantic attributes determined by multiple sets of raster unit data, remove raster units that represent dynamic target categories to determine the target semantic raster data of the current frame.

[0044] In this embodiment, semantic point clouds are projected at multiple preset raster scales at multiple resolutions. The semantic attributes of the raster are determined by combining the probability distribution of point cloud categories. Finally, dynamic target rasteres are filtered and eliminated across scales, retaining static semantic raster units as target semantic raster data. For step 01 above, multiple raster scales are pre-defined to form a scale system of low resolution, medium resolution, and high resolution. Each scale corresponds to an independent preset raster space, and a balance between computational power and accuracy is maintained between scales. Low-resolution raster (large unit size, such as 0.5m × 0.5m) has low computational load and fast processing speed, used to quickly capture global semantic distribution; high-resolution raster (small unit size, such as 0.1m × 0.1m) has high accuracy but relatively high computational load, focusing only on local key areas (such as the area around retaining walls or densely populated work areas); medium-resolution raster serves as a benchmark, balancing accuracy and efficiency. The coordinate system and mapping rules of each raster space are unified to ensure projection consistency and data compatibility between scales. The semantic point cloud is projected onto each preset raster space, with computation optimized according to scale characteristics during the projection process: low-resolution raster uses a fast mapping algorithm to directly allocate point clouds according to coordinate range, reducing redundant calculations; high-resolution raster only performs fine-grained projection on local key areas, avoiding high computational consumption across the entire scene. Furthermore, it supports parallel processing of multi-scale projection tasks, further shortening the overall computation time, ultimately forming multiple sets of raster unit data, each corresponding to a scale, and each raster unit is associated with several semantic points and their category probability distribution information. Multiple sets of raster unit data with differentiated resolutions are output, preserving both the global semantic association and high-speed processing advantages at low scales, and retaining the local semantic details at high scales. Simultaneously, through scale layering and parallel computation, compared to single high-resolution rasterization processing, computational efficiency is improved, achieving the dual goals of computational acceleration and accuracy preservation. For step 02 above, each group of raster cell data is iterated one by one. If the highest probability value of a point in a raster cell belonging to the category of a dynamic target is greater than or equal to a preset threshold, and the proportion of the corresponding points is greater than or equal to a quantity threshold, it is determined to be a dynamic target. If both the highest probability value and the proportion of the static target category meet the criteria, it is determined to be a static target. If neither condition is met, it is marked as a semantically ambiguous raster. The semantic attribute is labeled for each group of raster cells, and the semantic attribute is the category to which the semantic information of the raster cell belongs. Multiple groups of raster cell data carrying semantic attributes are formed to ensure the accuracy of raster attribute determination at each scale. For step 03 above, all raster cell data is quickly traversed, raster cells determined to be dynamic target categories are removed, raster cells with semantic ambiguity that cannot be supplemented for determination are filtered, and raster cells of the static target type are retained. The generated target semantic raster data has eliminated dynamic interference, and through multi-scale accelerated computation, the overall processing time is significantly lower than that of the single-scale process, while the data purity and accuracy are better.

[0045] In another embodiment of this disclosure, the method further includes: Step S1: If the current frame is the first frame, generate the first frame terrain data based on the target semantic raster data of the current frame; Step S2: Verify the point cloud data of the current frame based on the terrain data of the first frame, identify the noise in the point cloud data of the current frame, and delete it.

[0046] In this embodiment, for the first frame scene without historical frame reference, the first frame terrain data is generated directly based on the target semantic raster data with dynamic targets removed, through rapid structured modeling. This balances modeling efficiency and basic accuracy, providing a benchmark for subsequent first frame point cloud verification. Regarding step S1, firstly, it is determined whether the current frame is the first frame (if there is no historical frame target semantic raster data cache, it is the first frame) by using the frame sequence number or data cache status. If it is determined to be the first frame, the target semantic raster data of the current frame (with dynamic targets removed, containing only static target category raster units) is rapidly scaled and coordinate transformed based on a preset raster scale to generate the first frame terrain data. Regarding step S2, the original point cloud data of the first frame and the generated first frame terrain data are precisely aligned in spatial coordinates. Based on the radar calibration parameters and the preset raster spatial coordinate system, it is ensured that each original point accurately matches the corresponding spatial region in the terrain data, avoiding misjudgment noise caused by alignment deviations. The process iterates through the first frame of raw point cloud data, comparing the 3D coordinates of each point with the corresponding 3D coordinates of the terrain data. If the point cloud height exceeds the elevation of the terrain data, or if its spatial location is outside the terrain data, it is identified as noise and deleted. By validating the raw point cloud data with the first frame of terrain data, noise such as dust and heat is precisely removed, preventing noise from being carried over into subsequent multi-frame fusions. This reduces noise accumulation at the source and ensures the accuracy of subsequent fused terrain data.

[0047] In another embodiment of this disclosure, step S104 above, which verifies the point cloud data of the current frame based on terrain data to determine the noise of the point cloud data of the current frame, includes: Mark points in the current frame's point cloud data that are located outside the terrain data as noise in the current frame's point cloud data; and / or, Points in the current frame's point cloud data with height values ​​higher than those in the terrain data are marked as noise in the current frame's point cloud data.

[0048] In this embodiment, based on the static spatial and elevation features of terrain data, abnormal point clouds exceeding terrain boundaries or higher than terrain elevations are filtered out. This balances the efficiency and accuracy of noise assessment, effectively eliminating noise from the mining environment, such as dust, heat, and suspended impurities, providing clean and effective data for subsequent point cloud data applications and data caching. The original point cloud data of the current frame is traversed one by one, extracting the three-dimensional coordinates (X, Y, Z) of each point cloud and comparing them with the terrain boundary boxes of the terrain data: if the X coordinate of the point cloud exceeds the terrain X-axis boundary, or the Y coordinate exceeds the terrain Y-axis boundary, or the Z coordinate exceeds the terrain Z-axis boundary, the point cloud is directly determined to be noise; if the point cloud coordinates are within the terrain boundary box, it is determined to be a valid point in space. The Z coordinate (height value) of the point cloud is compared with the elevation value of the corresponding terrain unit in the terrain data: if the point cloud height value is higher than the elevation value, it is determined to be noise. Spatial location verification filters discrete noise, while height verification filters terrain-attached noise. Both can be used individually or in combination to comprehensively cover various types of noise in mining scenarios, such as dust, heat, and suspended impurities.

[0049] I. For the detection part of the preset segmentation model, the following steps are included: Step 1: Perform rule-based processing on the point cloud data of the current frame: 1.1 Point Cloud Data Regularization The point cloud data of the current frame is transformed into a two-dimensional distance map by spherical projection, preserving the original geometric information (depth, azimuth, elevation); the disordered point cloud data is transformed into a regular voxel grid by voxelization, realizing the structured transformation of point cloud data.

[0050] 1.2 Data Augmentation and Normalization Data augmentation operations are performed on the above two-dimensional distance map or voxel grid, including dynamic random sampling (balancing the data distribution in sparse areas of the point cloud) and random rotation / translation (improving the generalization robustness of the model); then the data is normalized to the [-1,1] interval to eliminate numerical scale differences and ensure feature consistency.

[0051] 1.3 Regularized Feature Map Output The output is a regularized feature map with dimensions (C, H, W), where channel C contains attributes such as depth information and reflection intensity information, and H and W are the height and width dimensions of the feature map, respectively.

[0052] Step 2: Perform deep feature extraction on the regularized feature map A combination of convolutional neural networks and Transformer encoders is used to perform deep feature extraction on regularized feature maps, specifically including: 2.1 2D Convolutional Neural Network Operations Perform multi-layer 2D convolutional neural network convolution operations on the first regularized feature map (obtained after processing the two-dimensional distance map), for example, extract features based on the ResNet backbone network.

[0053] 2.2 3D Convolutional Neural Network Operations The second regularized feature map (obtained after processing voxel grids) is used to extract local geometric features using a sparse 3D convolutional neural network (such as VoxelNet or HEDNet), and the receptive field is gradually expanded through hierarchical downsampling operations.

[0054] 2.3 Transformer Encoder Feature Enhancement The output features (voxel or raster features) obtained from the convolution operation are unfolded into a feature sequence. A multi-head self-attention mechanism is used to model the global contextual relationships, compensating for the insufficient global feature capture capability of convolutional neural networks and improving the completeness of feature representation. The expression is as follows: ; in, This represents the set of input query vectors. Let represent the set of key vectors at all positions in the input feature sequence. This represents the set of value vectors for all positions in the input feature sequence; Indicates calculation and The matrix product yields the attention score matrix, which reflects the original similarity between each query and all keys. Indicates the scaling factor. yes The vector dimension. This means performing Softmax normalization on the scaled attention score matrix, converting each row vector into a probability distribution, ensuring that the sum of the attention weights at all positions is 1, and outputting the attention weight matrix.

[0055] 2.4 High-dimensional feature map output The output is a high-dimensional feature map with dimensions (C1, W1, H1), where C1 is the number of high-dimensional feature channels, and W1 and H1 are the spatial resolutions of the high-dimensional feature map.

[0056] Step 3: Decoder module processing High-dimensional feature maps are optimized through upsampling and feature aggregation operations, specifically including: 3.1 Resolution Restoration By employing deconvolution or interpolation (nearest neighbor interpolation / bilinear interpolation), upsampling is performed on the high-dimensional feature map to restore the spatial resolution of the feature map and adapt it to the scale of the original point cloud data.

[0057] 3.2 Feature Fusion A skip connection mechanism is introduced to concatenate and fuse the intermediate layer features output by the encoder module with the high-level semantic features, thereby supplementing the detailed information of semantic segmentation and improving the accuracy of feature localization. For example, the UNet network structure can be used.

[0058] Step 4: Output Layer Processing The Softmax function is used to perform logistic regression on the decoder output features to generate the probability distribution of each point in the current frame's point cloud data to its respective category. The expression is as follows: ; in, Indicates that the input is At that time, the preset segmentation model predicts that the sample belongs to the category. The probability of . The value ranges from 0 to 1, and the sum of the probabilities of all categories is 1. This represents the input sample of the preset segmentation model. Indicates the category label of the sample. Indicates input sample In category The raw output score is directly output from the last linear layer of the preset segmentation model without being processed by an activation function. This indicates the number of categories, and the probability distribution satisfies the condition that the sum of the probabilities of all categories is 1, providing a quantitative basis for subsequent semantic point cloud determination. Indicates all The original scores for each category.

[0059] II. The semantic point cloud post-processing part includes the following steps: Step 1: Multi-scale raster projection of semantic point cloud and semantic attribute determination The output semantic point cloud is projected onto a preset grid space of corresponding resolution based on multiple preset grid scales to obtain multiple sets of grid cell data. For each grid cell in each set of grid cell data, the semantic attribute of the grid cell is determined according to the probability distribution of the category to which each point in the cell belongs. The determination method can be to take the category that appears most frequently in the grid cell as the semantic attribute of the cell.

[0060] Step 2: Multi-frame fusion and terrain data generation 2.1 Dynamic Target Grid Cell Removal For the above multiple sets of raster unit data, based on the semantic attributes of each raster unit, raster units that represent dynamic target categories are removed. The dynamic targets include dynamic vehicles, pedestrians, dynamic dust, hot air, etc. Raster units that only contain ground and static obstacle semantics are retained to obtain the target semantic raster data of the current frame.

[0061] 2.2 Probabilistic Fusion and High-Level Fusion 2.2.1 Probability Fusion Probabilistic fusion is performed on the corresponding grid cells in the target semantic raster data of the current frame and the target semantic raster data of the historical frames to obtain the target probability coefficient. The probability fusion method is as follows: target probability coefficient = probability coefficient corresponding to the grid cell in the current frame + probability coefficient of the corresponding grid cell in the historical frame; wherein the grid cell probability coefficient is divided into two categories: grid cells with semantic information of ground correspond to a first preset coefficient a, and grid cells with semantic information of static obstacles correspond to a second preset coefficient b. A multi-frame cumulative fusion method is adopted, and no forgetting mechanism is executed.

[0062] 2.2.2 High degree of integration A height fusion is performed on the corresponding raster cells in the target semantic raster data of the current frame and the target semantic raster data of the historical frames to obtain the target height value. The height fusion is expressed as: target height value = height value of the current frame raster cell × preset height coefficient of the current frame raster cell + height value of the corresponding raster cell in the target semantic raster data of the historical frame × preset height coefficient of the historical frame raster cell.

[0063] 2.3 Grid cell selection and terrain data generation Based on the target probability coefficient and a preset probability coefficient threshold, grid cells are selected: if the target probability coefficient does not reach the preset threshold, it is determined to be a semantically ambiguous grid cell, and no data is output; if the target probability coefficient reaches the preset threshold, it is determined to be a grid cell, indicating that after multi-frame cumulative verification, the grid cell can be identified as static terrain data. Scaling processing is performed on the grid cells to generate terrain data (i.e., DEM data). The terrain data is raster data with a preset raster size, obtained by scaling the grid cells according to a preset ratio.

[0064] Step 3: Current frame point cloud data verification and noise labeling Using the generated terrain data as a benchmark, the point cloud data of the current frame is validated to identify and label noisy data: point clouds located outside the terrain data are labeled as noise; and / or, point clouds with height values ​​higher than the corresponding area height values ​​in the terrain data are labeled as noise. This validation step ensures that the output point cloud data is free from noise interference, avoiding the introduction of noisy data into downstream high-precision map construction.

[0065] Based on the same disclosed concept, this disclosure also provides a point cloud noise identification device and an unmanned vehicle. Since the principle of solving the problem by these devices and unmanned vehicles is similar to the aforementioned point cloud noise identification method, the implementation of the device and unmanned vehicle can refer to the implementation of the aforementioned method, and the repeated parts will not be described again.

[0066] This disclosure provides a point cloud noise identification device, such as... Figure 2 As shown, it includes: The semantic segmentation module 201 is used to perform feature recognition on the point cloud data of the current frame to obtain a semantic point cloud; The dynamic target removal module 202 is used to perform rasterization processing on the semantic point cloud, and remove the raster units representing the dynamic target category according to the category to which each point in the semantic point cloud belongs, so as to determine the target semantic raster data of the current frame. The semantic raster data fusion module 203 is used to fuse the target semantic raster data of the current frame with the target semantic raster data of the historical frames to generate terrain data; The noise recognition module 204 is used to verify the point cloud data of the current frame based on the terrain data and determine the noise of the point cloud data of the current frame.

[0067] This disclosure provides an unmanned vehicle, including: a point cloud noise identification device as described in any of the above embodiments.

[0068] Through the above description of the embodiments, those skilled in the art can clearly understand that the embodiments of this disclosure can be implemented in hardware or by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions of the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, mobile hard drive, etc.) and includes several instructions to cause a computer device (such as a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of this disclosure.

[0069] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes in the drawings are not necessarily essential for implementing this disclosure.

[0070] Those skilled in the art will understand that the modules in the apparatus of the embodiments can be distributed in the apparatus of the embodiments as described in the embodiments, or they can be located in one or more devices different from this embodiment with corresponding changes. The modules of the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0071] The sequence numbers of the embodiments disclosed above are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0072] Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from its spirit and scope. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. A method for identifying point cloud noise, characterized in that, include: Perform feature recognition on the point cloud data of the current frame to obtain a semantic point cloud; The semantic point cloud is rasterized, and the raster units representing the dynamic target category are removed according to the category to which each point in the semantic point cloud belongs, so as to determine the target semantic raster data of the current frame. The target semantic raster data of the current frame is fused with the target semantic raster data of the historical frames to generate terrain data; The point cloud data of the current frame is verified based on the terrain data to determine the noise of the point cloud data of the current frame.

2. The method as described in claim 1, characterized in that, The step of fusing the target semantic raster data of the current frame with the target semantic raster data of the historical frames to generate terrain data includes: For each raster cell in the target semantic raster data of the current frame, determine the probability coefficient corresponding to the semantic information of each raster cell; The target probability coefficient is determined based on the probability coefficient of the current frame raster cell and the probability coefficient of the corresponding raster cell in the target semantic raster data of the historical frame. Based on the target probability coefficient and the preset probability coefficient threshold, the grid cell is determined from the grid cell; The grid cells are scaled to generate terrain data.

3. The method as described in claim 2, characterized in that, The semantic information of the grid cell includes: ground and static obstacles; For each raster cell in the target semantic raster data of the current frame, determine the probability coefficient corresponding to the semantic information of each raster cell, including: For each grid cell in the target semantic grid data of the current frame, the following steps are performed: if the semantic information of the grid cell is ground, use the first preset coefficient as the probability coefficient of the grid cell; if the semantic information of the grid cell is a static obstacle, use the second preset coefficient as the probability coefficient of the grid cell. Preferably, determining the grid cell from the grid cells based on the target probability coefficient and the preset probability coefficient threshold includes: when the target probability coefficient satisfies the first preset probability coefficient threshold, using the corresponding grid cell as the grid cell with semantic information of a static obstacle; and when the target probability coefficient satisfies the second preset probability coefficient threshold, using the corresponding grid cell as the grid cell with semantic information of the ground. Preferably, determining the target probability coefficient based on the probability coefficient of the current frame raster unit and the probability coefficient of the corresponding position raster unit in the target semantic raster data of the historical frame includes: summing the probability coefficient of the current frame raster unit and the probability coefficient of the corresponding position raster unit in the target semantic raster data of the historical frame according to a preset operation to determine the target probability coefficient. Preferably, the scaling process of the grid cells to generate terrain data includes: The grid cells whose semantic information is static obstacles and the grid cells whose semantic information is ground are scaled to generate terrain data.

4. The method as described in claim 1, characterized in that, The step of fusing the target semantic raster data of the current frame with the target semantic raster data of the historical frames to generate terrain data includes: For each raster cell in the target semantic raster data of the current frame, determine the height value of the raster cell; Multiply the height value of the current frame raster cell by the preset height coefficient of the current frame raster cell, and add the product of the height value of the raster cell at the corresponding position in the target semantic raster data of the historical frame and the preset height coefficient of the historical frame raster cell to determine the target height value of the raster cell. Terrain data is generated based on the target height value of the grid cell and the grid cell itself.

5. The method as described in claim 1, characterized in that, Feature recognition is performed on the point cloud data of the current frame to obtain a semantic point cloud, including: The point cloud data of the current frame is processed into regularized features to generate regularized feature maps. The regularized feature map is input into a preset segmentation model to determine the probability distribution of the category to which each point in the point cloud data of the current frame belongs; The semantic point cloud is determined based on the probability distribution of the category to which each point belongs in the point cloud data of the current frame; Preferably, the preset segmentation model is obtained in the following manner: The sample point cloud data is processed by rules to obtain training samples; and the sample point cloud data is labeled to obtain semantic point cloud ground truth. The training samples are input into the initial segmentation model for feature recognition, resulting in a semantic point cloud generated during training. Based on the loss between the semantic point cloud generated during training and the ground truth value of the semantic point cloud, the segmentation model parameters are updated to obtain the preset segmentation model.

6. The method as described in claim 5, characterized in that, The point cloud data of the current frame is processed according to rules to generate a regularized feature map, including: The point cloud data of the current frame is transformed into a two-dimensional distance map through spherical projection; the two-dimensional distance map is then subjected to data augmentation and normalization processing to output a first regularized feature map; and / or, The point cloud data of the current frame is converted into a voxel grid by voxelization, and the voxel grid is subjected to data augmentation and normalization processing to output a second regularized feature map. The first regularized feature map and the second regularized feature map include depth information and / or reflection intensity information; Preferably, the preset segmentation model includes: an encoder module and a decoder module; the encoder module includes: a convolutional neural network and a Transformer encoder, and the convolutional neural network includes: a 2D convolutional neural network and / or a 3D convolutional neural network; The step of inputting the regularized feature map into a preset segmentation model to determine the probability distribution of the category to which each point in the point cloud data of the current frame belongs includes: If the generated regularized feature map is a first regularized feature map, the first regularized feature map is input into a 2D convolutional neural network for convolution operation; and / or, if the generated regularized feature map is a second regularized feature map, the second regularized feature map is input into a 3D convolutional neural network for convolution operation; The output features obtained from the convolution operation are input into the Transformer encoder to obtain a high-dimensional feature map; The high-dimensional feature map is input, deconvolved or interpolated, and then fused with the intermediate layer features obtained from the convolution operation. The fused features are then input to the decoder module for decoding, and the output layer outputs the probability distribution of the category to which each point in the current frame's point cloud data belongs.

7. The method as described in claim 1, characterized in that, The step of rasterizing the semantic point cloud, and removing raster units representing dynamic target categories based on the category of each point in the semantic point cloud to determine the target semantic raster data of the current frame, includes: Based on multiple preset grid scales, the semantic point cloud is projected onto a preset grid space of corresponding resolution to obtain multiple sets of grid unit data; For each raster cell in each group of raster cell data, determine the semantic attributes of the raster cell based on the probability distribution of the category to which each point in the raster cell data belongs; Based on the raster unit semantic attributes determined from multiple sets of raster unit data, raster units representing dynamic target categories are removed to determine the target semantic raster data for the current frame.

8. The method as described in claim 1, characterized in that, The method further includes: If the current frame is the first frame, generate the first frame terrain data based on the target semantic raster data of the current frame; The point cloud data of the current frame is verified based on the terrain data of the first frame to identify and delete noise in the point cloud data of the current frame.

9. The method as described in claim 1, characterized in that, The step of verifying the point cloud data of the current frame based on the terrain data to determine the noise of the point cloud data of the current frame includes: Points in the current frame's point cloud data whose locations are outside the terrain data are marked as noise in the current frame's point cloud data; and / or, Points in the current frame's point cloud data with height values ​​higher than those in the terrain data are marked as noise in the current frame's point cloud data.

10. A point cloud noise identification device, characterized in that, include: The semantic segmentation module is used to perform feature recognition on the point cloud data of the current frame to obtain semantic point cloud; The dynamic target removal module is used to perform rasterization processing on the semantic point cloud, and remove the raster units that represent the dynamic target category according to the category to which each point in the semantic point cloud belongs, so as to determine the target semantic raster data of the current frame. The semantic raster data fusion module is used to fuse the target semantic raster data of the current frame with the target semantic raster data of the historical frames to generate terrain data; The noise recognition module is used to verify the point cloud data of the current frame based on the terrain data and determine the noise of the point cloud data of the current frame.

11. An unmanned vehicle, characterized in that, include: Includes the point cloud noise identification device as described in claim 10.

Images