A three-dimensional target detection method and system for rain and snow weather

Abstract

The application relates to the technical field of three-dimensional target detection, and discloses a three-dimensional target detection method and system for rain and snow weather, which comprises the following steps: acquiring original LiDAR point cloud data; constructing a double-flow parallel feature extraction network; inputting the original LiDAR point cloud data into a geometry feature maintaining main flow branch and a weather perception auxiliary flow branch of the double-flow parallel feature extraction network in parallel; extracting three-dimensional voxel geometry features based on the geometry feature maintaining main flow branch; extracting a plurality of two-dimensional weather feature masks based on the weather perception auxiliary flow branch; generating a three-dimensional cylindrical mask from the plurality of two-dimensional weather feature masks, mapping the three-dimensional cylindrical mask into denoising attention weights which are aligned with a three-dimensional voxel geometry feature space; and obtaining voxel features; compressing the voxel features into bird's eye view features, performing global context modeling and local detail enhancement, and generating boundary information of a three-dimensional target. The application can realize high-precision three-dimensional boundary prediction in a severe weather environment.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional target detection technology, and more specifically, to a three-dimensional target detection method and system for rainy and snowy weather. Background Technology

[0002] With the development of autonomous driving and intelligent transportation, environmental perception has become a key fundamental capability for safe vehicle operation. Among these, LiDAR (Light Detection and Ranging) is widely used in target detection, obstacle recognition, and path planning due to its ability to directly acquire high-precision 3D point cloud information. Existing 3D target detection methods typically extract features based on voxelization, direct point cloud learning, or bird's-eye view (BEV) projection, combining convolutional neural networks or sparse convolutional networks to achieve spatial feature modeling, achieving good detection accuracy under clear weather conditions.

[0003] However, severe weather conditions such as rain and snow significantly interfere with the measurement performance of lidar. Raindrops and snowflakes suspended in the air absorb or scatter the laser signal, causing attenuation of the echo signal and making it difficult to effectively detect distant targets. Simultaneously, nearby precipitation particles may directly reflect the laser signal, forming random noise points in the point cloud and interfering with the identification of real targets. Snowflakes, driven by wind, may also form clusters, creating false features resembling obstacles and increasing the risk of false detection. Specular reflection caused by accumulated water or snow can also lead to sparse or hollow ground point clouds, affecting scene modeling and target localization. Rain and snow environments also cause fluctuations in reflection intensity, making it difficult for traditional denoising methods based on geometry or intensity thresholds to simultaneously suppress noise and preserve effective information. Furthermore, existing 3D target detection networks are mostly designed for sunny or mildly disturbed environments and lack the ability to explicitly model the characteristics of severe weather, resulting in a significant decrease in detection performance in rainy and snowy scenarios.

[0004] Therefore, it is necessary to design a three-dimensional target detection method and system for rainy and snowy weather to solve the problems existing in the current technology. Summary of the Invention

[0005] In view of this, the present invention proposes a three-dimensional target detection method and system for rainy and snowy weather, aiming to solve the problem that the lack of explicit modeling capability for severe weather features leads to a significant decrease in detection performance in rainy and snowy scenarios.

[0006] On the one hand, this invention proposes a three-dimensional target detection method for rainy and snowy weather, including: Acquire raw LiDAR point cloud data; A dual-stream parallel feature extraction network is constructed. The original LiDAR point cloud data is input in parallel into the geometric feature preservation main branch and the weather sensing auxiliary branch of the dual-stream parallel feature extraction network. Based on the geometric feature preservation main branch, the original LiDAR point cloud data is voxelized, and three-dimensional voxel geometric features are extracted through multi-branch cross-convolution. Based on the weather sensing auxiliary branch, the original LiDAR point cloud data is projected from three perspectives to generate a two-dimensional feature map, and several two-dimensional weather feature masks are extracted. A three-dimensional cylindrical mask is generated from several two-dimensional weather feature masks; the three-dimensional cylindrical mask is mapped to denoising attention weights; and voxel features are obtained based on the denoising attention weights and three-dimensional voxel geometric features. The voxel features are compressed into bird's-eye view features, and the bird's-eye view features are preprocessed; and based on the preprocessed bird's-eye view features, the boundary information of the three-dimensional target is generated.

[0007] Furthermore, when extracting 3D voxel geometric features by preserving the mainstream branches based on the geometric features and using multi-branch cross-convolution, the process includes: In the Cartesian coordinate system, the original LiDAR point cloud data is voxelized to obtain voxel data, and a lightweight multi-branch cross-sparse convolutional backbone network is used to extract features from the voxel data. The lightweight multi-branch cross-sparse convolution backbone network includes parallel sub-manifold sparse convolution paths and regular sparse convolution paths. Based on the active voxel positions in the voxel data, convolution calculations are performed in the sub-manifold sparse convolution path; based on the neighborhood space of the voxel data, feature expansion and receptive field enlargement processing are performed in the regular sparse convolution path; the output data of the sub-manifold sparse convolution path and the regular sparse convolution path are fused to obtain the three-dimensional voxel geometric features.

[0008] Furthermore, when generating a two-dimensional feature map based on the weather-sensing auxiliary stream branch, the process includes: The original LiDAR point cloud data in Cartesian coordinates is transformed to obtain cylindrical coordinate data. Based on the cylindrical coordinate data, the cylindrical space is divided into cylindrical voxel grids. The original LiDAR point cloud data within each cylindrical voxel grid is extracted, and local features are extracted through multilayer sensing and max pooling to obtain local feature vectors. Based on all the local feature vectors, a three-dimensional cylindrical feature is constructed, and compressed projection is performed along the radial, angular, and height axes to obtain the two-dimensional feature map. The two-dimensional feature map includes a circular bird's-eye view, a distance view, and an auxiliary view.

[0009] Furthermore, when extracting several two-dimensional weather feature masks, the following are included: Based on the circular bird's-eye view, distance view, and auxiliary view, feature channel compression processing is performed to obtain the scalar logarithmic probability corresponding to each pixel position; based on the scalar logarithmic probability, probability mapping processing is performed to obtain several two-dimensional weather feature masks.

[0010] Furthermore, when generating a three-dimensional cylindrical mask from several of the aforementioned two-dimensional weather feature masks, the process includes: The two-dimensional weather feature masks are sequentially subjected to dimensional expansion, weighted linear combination, and probability mapping to generate a three-dimensional cylindrical mask.

[0011] Furthermore, mapping the three-dimensional cylindrical mask to denoising attention weights includes: Based on the non-empty voxels in the three-dimensional voxel geometric features, the Cartesian center coordinates are extracted and transformed into continuous spatial coordinates in cylindrical coordinates. Based on the continuous spatial coordinates and the three-dimensional cylindrical mask, the denoising attention weights are obtained through trilinear interpolation.

[0012] Furthermore, when obtaining voxel features, this includes: Based on the denoising attention weights and the three-dimensional voxel geometric features, element-wise intelligent multiplication is performed to obtain the voxel features.

[0013] Furthermore, the preprocessing of the bird's-eye view features includes: The preprocessing includes global context modeling and local detail enhancement.

[0014] Furthermore, when generating the boundary information of a 3D target, the following are included: The bird's-eye view features are subjected to parallel convolution processing to obtain class probabilities, 3D bounding box regression parameters, and orientation classification information. Based on the class probabilities, 3D bounding box regression parameters, and orientation classification information, combined with preset anchor boxes, the position offset is predicted to generate the boundary information of the 3D target.

[0015] Compared with existing technologies, the advantages of this invention are as follows: By constructing a dual-stream parallel feature extraction network, the geometric feature preservation main branch and the weather perception auxiliary branch are processed in parallel. This allows for full utilization of the geometric information of point clouds and precipitation noise features under severe weather conditions such as rain and snow, suppressing random noise interference caused by rain and snow particles, thereby improving the stability and reliability of 3D target detection. Through voxelization processing of the geometric main branch and multi-branch cross-convolution to extract 3D voxel geometric features, combined with the three-view projection of the weather auxiliary branch and the generation and fusion of 2D weather feature masks, target features are enhanced in both spatial and intensity dimensions while preserving detailed information, improving target recognizability in rain and snow environments. The 2D weather feature mask is adaptively weighted and fused to generate a 3D cylindrical mask, which is then mapped to denoising attention weights aligned with the voxel geometric feature space. This suppresses non-target noise while ensuring that real target point clouds are not mistakenly deleted, thus optimizing the quality of feature representation. By compressing voxel features into bird's-eye view features and combining global context modeling with local detail enhancement operations, both global spatial relationships and local target details can be acquired simultaneously, enhancing the detection network's ability to identify targets in complex scenes. Based on the enhanced bird's-eye view features, the category, location, and orientation information of 3D targets are generated, enabling high-precision 3D boundary prediction even in adverse weather conditions.

[0016] On the other hand, this application also provides a three-dimensional target detection system for rainy and snowy weather, used to apply the above-mentioned three-dimensional target detection method for rainy and snowy weather, including: The data acquisition unit is configured to acquire raw LiDAR point cloud data; The feature extraction unit is configured to construct a dual-stream parallel feature extraction network; input the original LiDAR point cloud data in parallel into the geometric feature preservation main branch and the weather sensing auxiliary branch of the dual-stream parallel feature extraction network; based on the geometric feature preservation main branch, perform voxelization processing on the original LiDAR point cloud data, and extract three-dimensional voxel geometric features through multi-branch cross-convolution; based on the weather sensing auxiliary branch, project the original LiDAR point cloud data from three perspectives to generate a two-dimensional feature map, and extract several two-dimensional weather feature masks; The feature processing unit is configured to generate a three-dimensional cylindrical mask from a plurality of the two-dimensional weather feature masks; map the three-dimensional cylindrical mask to denoising attention weights; and obtain voxel features based on the denoising attention weights and the three-dimensional voxel geometric features. The information generation unit is configured to compress the voxel features into bird's-eye view features, preprocess the bird's-eye view features, and generate boundary information of the three-dimensional target based on the preprocessed bird's-eye view features.

[0017] It is understandable that the above-mentioned three-dimensional target detection method and system for rainy and snowy weather have the same beneficial effects, and will not be elaborated further here. Attached Figure Description

[0018] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 A flowchart illustrating a three-dimensional target detection method for rainy and snowy weather provided in an embodiment of the present invention; Figure 2 The diagram shows the dual-stream parallel feature extraction network structure of the three-dimensional target detection method for rainy and snowy weather provided in this embodiment of the invention. Figure 3 The geometric feature-preserving mainstream branch structure diagram of the three-dimensional target detection method for rain and snow weather provided in the embodiments of the present invention; Figure 4 The weather perception auxiliary flow branch structure diagram is provided for the three-dimensional target detection method for rainy and snowy weather according to the embodiments of the present invention. Figure 5 This is a structural diagram of a compact, fine-grained attention enhancement module for a three-dimensional target detection method for rainy and snowy weather provided in an embodiment of the present invention. Figure 6 This is a functional block diagram of a three-dimensional target detection system for rainy and snowy weather, provided in an embodiment of the present invention. Detailed Implementation

[0019] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, embodiments and features in the embodiments of the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0020] In some embodiments of this application, see Figure 1-2 As shown, a three-dimensional target detection method for rainy and snowy weather is proposed, including: S100: Acquire raw LiDAR point cloud data; S200: Construct a dual-stream parallel feature extraction network; input the raw LiDAR point cloud data in parallel into the geometric feature preservation main branch and the weather sensing auxiliary branch of the dual-stream parallel feature extraction network; based on the geometric feature preservation main branch, perform voxelization processing on the raw LiDAR point cloud data, and extract three-dimensional voxel geometric features through multi-branch cross-convolution; based on the weather sensing auxiliary branch, project the raw LiDAR point cloud data from three perspectives to generate a two-dimensional feature map, and extract several two-dimensional weather feature masks; S300: Generate a three-dimensional cylindrical mask from several two-dimensional weather feature masks; map the three-dimensional cylindrical mask to denoising attention weights; and obtain voxel features based on the denoising attention weights and three-dimensional voxel geometric features. S400: Compresses voxel features into bird's-eye view features and preprocesses the bird's-eye view features; and generates boundary information of the 3D target based on the preprocessed bird's-eye view features.

[0021] Specifically, in step S100, raw LiDAR point cloud data is acquired. The point cloud data can be acquired in real-time by an onboard LiDAR or read from a publicly available dataset. The point cloud includes three-dimensional coordinate information (x, y, z) and reflection intensity information (intensity). In practical applications, basic preprocessing can be performed on the point cloud, including coordinate range clipping (e.g., limiting forward 70m, lateral ±40m, and height range [-3m, 1m]), outlier removal, and time synchronization processing to ensure the validity and consistency of the input data.

[0022] In step S200, a dual-stream parallel feature extraction network is constructed, and the original point cloud data is simultaneously input into the geometric feature preservation main branch and the weather sensing auxiliary branch. In the geometric feature preservation main branch, the point cloud is first voxelized, dividing the continuous space into a regular 3D grid, for example, setting the voxel size to (0.1m, 0.1m, 0.2m). Feature aggregation is performed on points within each non-empty voxel to obtain initial voxel features (including the number of points, mean coordinates, and mean intensity). Subsequently, the voxel features are input into a lightweight multi-branch cross-sparse convolution backbone network for feature extraction. This network includes sub-manifold sparse convolution paths and regular sparse convolution paths. The sub-manifold paths perform convolution operations only on active voxels to maintain sparsity, while the regular paths increase the receptive field through neighborhood expansion. Finally, 3D voxel geometric features are obtained through feature fusion. Simultaneously, in the weather sensing auxiliary branch, the original point cloud is transformed from a Cartesian coordinate system to a cylindrical coordinate system. ), and divide the cylindrical space into grids, for example The cylindrical voxel mesh sizes correspond to the radial, angular, and height directions, respectively. For each point within a cylindrical voxel, a multilayer perceptron is used to extract local features, and max pooling is applied to obtain the feature representation of that voxel. Subsequently, the 3D cylindrical features are compressed and projected along the radial, angular, and height directions to obtain a circular bird's-eye view, a distance view, and an auxiliary view. Figure 3 Two-dimensional feature maps are generated and encoded using a lightweight two-dimensional convolutional network. At the encoding output, the high-dimensional features are compressed into scalar log probabilities through a 1×1 convolution, and then mapped to probability values ​​between 0 and 1 using a sigmoid function to generate the corresponding two-dimensional weather feature mask.

[0023] In step S300, multiple two-dimensional weather feature masks are fused and mapped. First, each two-dimensional mask is expanded in the missing dimension to restore a three-dimensional structure consistent with the cylindrical voxel space. Then, an adaptive weighted fusion method is used to linearly combine the view masks and generate a unified three-dimensional cylindrical mask using the Sigmoid function, where the fusion weights are trainable parameters. Next, for each non-empty voxel in the geometric mainstream branch, its Cartesian coordinates are extracted and converted to cylindrical coordinates. The corresponding weights are sampled in the three-dimensional cylindrical mask using trilinear interpolation to obtain denoising attention weights aligned with the voxel feature space. Finally, these attention weights are multiplied element-wise with the three-dimensional voxel geometric features to obtain the denoised voxel features.

[0024] In step S400, the voxel features are compressed along the height dimension, for example, through stacking or max pooling operations, mapping the 3D features to 2D bird's-eye view features. Subsequently, global context modeling and local detail enhancement are performed on the bird's-eye view features. Specifically, coarse-grained features are obtained through channel downsampling and spatial downsampling, and a moving window-based self-attention mechanism is introduced in the low-resolution space to model long-distance dependencies. Then, upsampling restores the resolution and fuses it with the original features to obtain the enhanced bird's-eye view features. Finally, this feature is input into the detection head, and parallel convolution outputs the target class probability, 3D bounding box regression parameters, and orientation classification information, which are then combined with preset anchor boxes to generate the final 3D target boundary information.

[0025] Understandably, by constructing a dual-stream parallel network structure, combining geometric feature extraction with weather noise modeling, and utilizing a cross-representation gating mechanism to achieve the alignment and fusion of features across multiple coordinate systems, point cloud noise interference can be suppressed and target structural information can be preserved under rainy and snowy weather conditions. By replacing the traditional hard threshold denoising strategy with an attention-weighted approach, the risk of accidental deletion of target information is reduced. Furthermore, by combining global context modeling and local detail enhancement, the accuracy and robustness of 3D target detection are improved.

[0026] In some embodiments of this application, see Figure 3 As shown, when extracting 3D voxel geometric features by preserving the mainstream branches based on geometric features and using multi-branch cross-convolution, the process includes: In the Cartesian coordinate system, the original LiDAR point cloud data is voxelized to obtain voxel data, and a lightweight multi-branch cross-sparse convolutional backbone network is used to extract features from the voxel data. The lightweight multi-branch cross-sparse convolution backbone network includes parallel sub-manifold sparse convolution paths and regular sparse convolution paths. Based on the active voxel locations in the voxel data, convolution computation is performed in the submanifold sparse convolution path; based on the neighborhood space of the voxel data, feature expansion and receptive field enlargement processing are performed in the regular sparse convolution path; the output data of the submanifold sparse convolution path and the regular sparse convolution path are fused to obtain the three-dimensional voxel geometric features.

[0027] Specifically, the original LiDAR point cloud data is first voxelized in a Cartesian coordinate system. Preferably, the point cloud is cropped according to a preset spatial range and the voxel size is set accordingly. The 3D space is divided into a regular voxel grid (e.g., 0.1m × 0.1m × 0.15m). For each non-empty voxel, the feature information of all points within it is extracted, and an initial feature vector for that voxel is generated through statistical aggregation. Features include, but are not limited to, the mean coordinates, mean reflection intensity, and point count. This voxelization process transforms the disordered point cloud into a structured data form while preserving its spatial distribution characteristics. After obtaining the voxel data, it is input into a lightweight multi-branch cross-sparse convolutional backbone network (LMCCN) for feature extraction. The backbone network consists of multiple cascaded coding blocks, each including parallel sub-manifold sparse convolutional paths and regular sparse convolutional paths.

[0028] Specifically, in the submanifold sparse convolution path, convolution operations are performed only on active voxel locations in the voxel data. That is, the convolution operation is only performed on existing non-empty voxel locations without introducing new voxel locations, thereby extracting local geometric details while maintaining feature sparsity. This path can effectively avoid excessive feature diffusion in space and maintain the fine expression of object edges and structural contours.

[0029] Specifically, in the regular sparse convolution path, convolution calculations are performed based on the neighborhood space of voxel data. By setting the convolution stride and kernel size, features are spatially expanded into the neighborhood, thereby increasing the receptive field and introducing richer contextual information. This path can capture the spatial relationships between voxels and improve the semantic expressive power of features.

[0030] Specifically, the output features of the sub-manifold sparse convolution path and the regular sparse convolution path are fused. Preferably, element-wise weighted summation or channel concatenation followed by convolution can be used for fusion, thereby comprehensively utilizing the feature advantages of the two paths to obtain the three-dimensional voxel geometric features. By stacking multiple layers of coding blocks, features are abstracted layer by layer from local geometric details to global semantic information.

[0031] Understandably, by employing a multi-branch cross structure that combines sub-manifold sparse convolutional paths with regular sparse convolutional paths, the receptive field is expanded while maintaining the sparsity of point cloud features, thus achieving an effective fusion of fine-grained geometric information and contextual semantic information.

[0032] In some embodiments of this application, see Figure 4 As shown, when generating a two-dimensional feature map based on the weather-sensing auxiliary stream branch, the following steps are included: The original LiDAR point cloud data in Cartesian coordinates is transformed to obtain cylindrical coordinate data. Based on the cylindrical coordinate data, the cylindrical space is divided into cylindrical voxel grids. The original LiDAR point cloud data in each cylindrical voxel grid is extracted, and local features are extracted through multilayer sensing and max pooling is performed to obtain local feature vectors. Based on all local feature vectors, a three-dimensional cylindrical feature is constructed, and compressed projection is performed along the radial, angular and height axes to obtain a two-dimensional feature map. The two-dimensional feature map includes a circular bird's-eye view, a distance view and an auxiliary view.

[0033] In some embodiments of this application, extracting several two-dimensional weather feature masks includes: Based on the circular bird's-eye view, distance view, and auxiliary view, feature channel compression is performed to obtain the scalar log probability corresponding to each pixel position; based on the scalar log probability, probability mapping is performed to obtain several two-dimensional weather feature masks.

[0034] Specifically, the original LiDAR point cloud data is first transformed from Cartesian coordinates to cylindrical coordinates to enhance its ability to represent radial noise distribution; the cylindrical coordinates include radial distance, angular azimuth, and height information. Subsequently, the point cloud is discretized within a preset spatial range according to radial, angular, and height directions, constructing a regular cylindrical pixel mesh structure. The scene is quantized into a dimension of [dimension value missing]. Cylindrical elements, in which The cylindrical elemental mesh sizes correspond to the radial, angular, and height directions, respectively.

[0035] Specifically, for those falling into a specific grid index Point set within The point set data contained within it is extracted, local features are extracted using a PointNet-style MLP (Multilayer Perceptron), and max pooling is used to obtain the feature vector of the grid. The expression is: ; in, It is the j-th point in the point cloud in cylindrical coordinates, where u, v, and w represent respectively Axis index, It is the geometric center of the grid space.

[0036] Specifically, the point features within the voxel are aggregated using max pooling to generate the corresponding local feature vector, thus forming a structured 3D cylindrical feature representation. Preferably, the point features include at least spatial coordinate information and reflection intensity information. After obtaining the 3D cylindrical features, compression projection processing is performed along the radial, angular, and height axes to obtain multiple 2D feature maps; for example, to reduce computational complexity and capture noise patterns under different views, we use 3D cylindrical features... Three 2D feature maps are generated by compressing projections along three axes and then stitching the features together. The 2D feature map includes a circular bird's-eye view representing the spatial distribution of the ground. Distance view used to characterize radial distance variation And auxiliary views used to supplement spatial structure information To further extract high-level semantic information from each view, a lightweight two-dimensional convolutional neural network encoder is constructed for each two-dimensional feature map to perform feature encoding processing.

[0037] Specifically, to obtain the weather feature mask, a channel compression module, preferably a 1×1 convolutional layer, is set at the encoding output of each two-dimensional feature map to linearly combine the multi-channel features, compressing the feature vector at each pixel location into a single scalar log-probability value. Subsequently, the scalar log-probability is mapped using a preset probability mapping function (preferably the sigmoid function) to convert it into a probability value between 0 and 1, thereby obtaining the corresponding two-dimensional weather feature mask. The probability mapping function formula is as follows: ; ; in, A two-dimensional weather feature mask; It is a circular bird's-eye view projection feature map after encoding processing.

[0038] Specifically, a two-dimensional weather feature mask is used to represent the probability that the corresponding spatial location is a non-noise area. When the probability value is close to 1, it indicates that the area is a real target or effective background area, and its feature information should be preserved; when the probability value is close to 0, it indicates that the area is rain / snow noise or an invalid area, and its feature information should be suppressed. In this way, the weather sensing auxiliary flow branch can learn the distribution characteristics of rain / snow noise from different perspectives and provide a reliable weighting basis for subsequent cross-branch feature fusion.

[0039] Understandably, by introducing cylindrical coordinate modeling and multi-view projection mechanisms into the weather-sensing auxiliary flow, the spatial distribution characteristics of rain and snow noise can be effectively characterized, and the noise region can be finely depicted through probabilistic weather feature masks; radial noise and real targets are distinguished, and the noise recognition capability is improved.

[0040] In some embodiments of this application, generating a three-dimensional cylindrical mask from several two-dimensional weather feature masks includes: Several two-dimensional weather feature masks are sequentially subjected to dimensional expansion, weighted linear combination, and probability mapping processes to generate a three-dimensional cylindrical mask.

[0041] In some embodiments of this application, mapping a three-dimensional cylindrical mask to denoising attention weights includes: Based on the non-empty voxels in the three-dimensional voxel geometry, Cartesian center coordinates are extracted and transformed into continuous spatial coordinates in cylindrical coordinates. Based on continuous spatial coordinates and a three-dimensional cylindrical mask, the denoising attention weights are obtained through trilinear interpolation.

[0042] In some embodiments of this application, obtaining voxel features includes: Based on the denoising attention weights and 3D voxel geometric features, element-wise intelligent multiplication is performed to obtain voxel features.

[0043] Specifically, firstly, several two-dimensional weather feature masks are subjected to dimensional expansion processing, allowing each two-dimensional mask to be copied, broadcast, or interpolated in its missing dimensions, thus converting the two-dimensional planar masks into three-dimensional tensors with the same size as the three-dimensional cylindrical pixel mesh. Based on this, a weighted linear combination processing is performed on the expanded three-dimensional tensors, preferably introducing learnable scalar weight parameters and bias terms, and adaptively weighting and summing different view masks to obtain an intermediate fusion quantity reflecting the joint discrimination result of multiple views. Subsequently, the intermediate fusion quantity undergoes probability mapping processing, preferably using the Sigmoid function to compress it to a probability range of 0 to 1, generating a three-dimensional cylindrical mask. The closer the mask value is to 1, the more likely the cylindrical spatial location corresponds to a real target or effective background; the closer the mask value is to 0, the more likely the location is rain, snow noise, or a blank area. This adaptive weighted fusion method can automatically adjust the contribution ratio of the circular bird's-eye view, distance view, and auxiliary view in the fusion process according to different weather scenarios, thereby avoiding the problem of excessive suppression of the overall mask due to misjudgment of a single view and improving the ability to preserve complex target structures.

[0044] Specifically, to map the 3D cylindrical mask to denoising attention weights aligned with the 3D voxel geometric feature space, the center coordinates in Cartesian coordinates are extracted for each non-empty voxel in the mainstream branch of the geometric features, and these center coordinates are then converted into continuous spatial coordinates in cylindrical coordinates. Since continuous spatial coordinates typically fall at non-integer positions in the cylindrical voxel grid, the corresponding mask values ​​cannot be obtained through direct indexing. Therefore, trilinear interpolation sampling is performed on the 3D cylindrical mask based on the continuous spatial coordinates to calculate the denoising attention weights corresponding to the non-empty voxel using the weight contributions of the eight neighboring voxels around the target point. Trilinear interpolation makes the mapping results between different coordinate systems smoother and more continuous, reduces the jagged edges caused by discretization errors, and improves the accuracy and stability of the mask during spatial alignment.

[0045] Specifically, after obtaining the denoising attention weights, they are multiplied element-wise with the 3D voxel geometric features to obtain the voxel features, as shown in the following expression: ; in, Mainstream characteristics; 3D weighted graph; Intelligent multiplication of elements; It is a voxel characteristic.

[0046] Preferably, since the denoising attention weights are single-channel weight maps, broadcasting operations can be performed along the feature channel dimension, allowing all channel features at the same voxel location to share the same gating coefficients, thereby achieving unified modulation of voxel-level features. This soft gating mechanism continuously suppresses the feature responses in rain and snow noise regions while retaining weak features with low confidence but potentially containing valid contextual information, avoiding the false deletion of true target edges, distant sparse targets, or low-reflectivity targets by traditional hard thresholding methods. Finally, after processing by the cross-representation gating module, rain and snow noise originally mixed in with mainstream 3D voxel features is significantly suppressed, while target features such as vehicles and pedestrians are fully preserved, providing high-quality feature input for subsequent BEV feature refinement and 3D target boundary prediction.

[0047] Understandably, by constructing a 3D cylindrical mask generation mechanism based on adaptive weighted fusion, multi-view weather information is fused, avoiding the false suppression problem caused by traditional hard fusion methods; precise alignment across coordinate systems is achieved through coordinate transformation and trilinear interpolation, enabling weather perception results to accurately apply to voxel features; and a soft gating mechanism based on attention weights retains effective target features while suppressing rain and snow noise, thereby improving the robustness and detection accuracy of 3D point cloud target detection under complex weather conditions.

[0048] In some embodiments of this application, see Figure 5 As shown, preprocessing the bird's-eye view features includes: The preprocessing includes global context modeling and local detail enhancement.

[0049] Specifically, based on voxel features, compression processing is performed along the height dimension to obtain bird's-eye view features; based on bird's-eye view features, channel and spatial downsampling processing is performed to obtain coarse-grained intermediate features; based on coarse-grained intermediate features, self-attention processing is performed through a moving window to obtain self-attention features; based on self-attention features and bird's-eye view features, fusion and upsampling processing are performed to obtain bird's-eye view features after global context modeling and local detail enhancement.

[0050] Specifically, voxel features are stacked or aggregated in the height direction to map the Z-axis information in the original three-dimensional space to the channel dimension, thereby obtaining the initial bird's-eye view features on the two-dimensional plane. Furthermore, the number of channels is adjusted by setting a 1×1 convolutional layer so that the obtained bird's-eye view features meet the channel scale requirements of subsequent network processing, while achieving effective fusion of information at different heights.

[0051] Specifically, after obtaining the bird's-eye view features, to improve feature representation and reduce computational complexity, the bird's-eye view features are downsampled in terms of channel and spatial dimensions to obtain coarse-grained intermediate features; the expression is: ; in, This represents the feature map downsampling function. This indicates the channel downsampling process. This represents the bird's-eye view obtained after downsampling; This is the initial bird's-eye view.

[0052] Preferably, the downsampling process includes: compressing the channel dimension through convolution operations and reducing the spatial resolution through stride convolution or pooling operations, thereby expanding the receptive field while reducing the feature map size and achieving an aggregated representation of spatial and channel information. Through the above processing, computational overhead can be reduced while preserving the main semantic information.

[0053] Specifically, based on coarse-grained intermediate features, a self-attention processing module based on a moving window mechanism is introduced to achieve global context modeling. Specifically, the coarse-grained intermediate features are divided into multiple local windows, and self-attention computation is performed within each window to obtain the relationships between different positions within the window. Simultaneously, information interaction between different windows is achieved through window movement or cross-window connection mechanisms, thereby establishing long-distance dependencies while maintaining low computational complexity.

[0054] Preferably, a self-attention mechanism based on Swin Transformer is introduced at the bottleneck layer. Unlike full-image attention, Swin Transformer utilizes a moving window mechanism to establish cross-window interaction while maintaining linear computational complexity. Furthermore, skip connections are employed to avoid deep feature degradation. The output of the i-th self-attention module... The calculation is as follows: ; in, This represents the i-th convolutional layer. This represents the input of the i-th convolutional layer. This refers to the Swing Transformer module, whose core calculation formula is: ; in, Don't represent query vector, key vector, and value vector. This represents the dimension of the key vector. It effectively captures the complex spatial relationships between different regions within the BEV feature set. Simultaneously, it dynamically extracts and fuses BEV features based on context, effectively focusing on key information.

[0055] Specifically, during the self-attention computation process, features are mapped to query vectors, key vectors, and value vectors, and the enhanced feature representation is obtained through weighted summation. Combined with a residual connection structure, the input features are fused with the self-attention output to avoid feature degradation during deep network training. Through this self-attention processing, self-attention features with global semantic information are obtained. After obtaining the self-attention features, they are fused with the original bird's-eye view features, preferably using feature concatenation or element-wise addition to balance local detail information and global contextual information. Subsequently, an upsampling operation restores the feature map to its original spatial resolution to meet the consistency requirements of the input feature size for subsequent detection heads. Finally, the bird's-eye view features after global context modeling and local detail enhancement are obtained. These features maintain fine-grained spatial structure while enhancing the expressive power of distant targets and weak feature regions.

[0056] Understandably, after compressing voxel features into bird's-eye view features, computational overhead is reduced through channel and spatial downsampling, and efficient global context modeling is achieved by utilizing a self-attention mechanism based on moving windows. At the same time, by combining upsampling and feature fusion operations, fine-grained structural information is preserved while restoring spatial resolution, thereby enhancing the expressive power of distant small targets and weak feature regions.

[0057] In some embodiments of this application, generating the boundary information of a three-dimensional target includes: Parallel convolution processing is performed on the features of the bird's-eye view to obtain the class probability, 3D bounding box regression parameters, and orientation classification information. Based on the class probability, 3D bounding box regression parameters, and orientation classification information, combined with the preset anchor boxes, the position offset is predicted to generate the boundary information of the 3D target.

[0058] Specifically, the bird's-eye view features are processed by parallel convolution, preferably using multiple parallel convolution branches. Each branch maps the input features through a convolutional layer (e.g., a 1×1 convolutional layer), outputting class probabilities, 3D bounding box regression parameters, and orientation classification information. The class probabilities represent the confidence that the current feature location belongs to different target categories; the 3D bounding box regression parameters represent the target's positional offset, size scaling, and height information relative to a preset anchor box; and the orientation classification information represents the target's orientation category.

[0059] Furthermore, for each spatial location on the bird's-eye view feature map, a corresponding candidate bounding box is generated based on a preset set of anchor boxes. Combining class probabilities, 3D bounding box regression parameters, and orientation classification information, the offset of each anchor box relative to the real target is calculated, thereby obtaining the target's position, size, and orientation parameters in 3D space, and ultimately generating the boundary information of the 3D target. Through this method, end-to-end 3D target detection is achieved.

[0060] During model training, to simultaneously optimize geometric feature extraction and weather noise recognition capabilities, a multi-task joint loss function is constructed, expressed as follows: ; in, Total loss; The detection loss of the main detection branch; Loss of auxiliary monitoring due to weather-sensing auxiliary flow; Hyperparameters for balancing the weights of the two branch tasks.

[0061] Specifically, total loss Detection loss due to the main detection branch Auxiliary monitoring loss of weather-sensing auxiliary flow Weighted composition, To balance the hyperparameters of the two branch task weights, the total loss is composed of a weighted average of the detection loss from the main detection branch and the auxiliary supervision loss from the weather perception auxiliary flow branch. The weighting coefficients are used to balance the two parts of the loss. In the main detection branch, the detection loss includes class loss, 3D bounding box regression loss, and orientation classification loss. Preferably, the 3D bounding box regression loss adopts a loss function optimized based on distance and overlap relationships, expressed as: ; in, Indicates category loss. Indicates directional classification loss, Represents the regression loss of the 3D bounding box. and This is a hyperparameter.

[0062] In the weather sensing auxiliary stream branch, to improve the accuracy of rain and snow noise identification, a continuous soft-label strategy based on point proportion is adopted to generate supervision signals. Specifically, in cylindrical coordinates, for each non-empty grid, the number of points belonging to the real target in its internal point cloud data is counted to the total number of points, and the ratio of the two is used as the soft label value of that grid, thus obtaining a continuous label with a value between 0 and 1. The expression for the statistical point cloud data is: ; Among them, N is the total number of internal points, and K is the number of points belonging to the true target. This continuous label can reflect the mixing degree of target points and noise points in the grid. Among them, when S(u, v, w) = 1, it indicates that all points in this grid are true target points, and the features should be completely retained; when S(u, v, w) = 0, it indicates that there are no object points in this grid, and the features should be suppressed; when 0 < S(u, v, w) < 1, it indicates that this grid contains both object points and rain-snow noise points, and the network needs to learn to moderately suppress the noise component while retaining the object features.

[0063] Specifically, after obtaining the three-dimensional soft label, it is projected along the radial, angular, and height directions respectively to obtain multiple corresponding two-dimensional true value masks, which are used to supervise the weather feature masks under different views. Since the soft label is a continuous value, the Smooth L1 loss function is used to measure the difference between the predicted mask and the true value mask, so as to maintain smooth optimization when the error is small and enhance robustness when the error is large. The total loss of the auxiliary flow branch is obtained by superimposing the Smooth L1 losses under multiple views. Preferably, the Smooth L1 Loss is used as the supervision loss function of the auxiliary flow.

[0064] It can be understood that based on the enhanced bird's-eye view features, the target category, three-dimensional bounding box, and orientation information are efficiently predicted through a parallel convolution structure, realizing end-to-end three-dimensional object detection; by introducing a multi-task joint loss function, the geometric feature learning and weather noise modeling are co-optimized, improving the overall performance of the model. The continuous soft label supervision strategy based on the point occupancy ratio can more finely describe the mixing degree of the target and noise in the voxel, and is numerically semantically consistent with the subsequent soft gating mechanism, thereby enhancing the network's adaptability to complex noise environments.

[0065] In another preferred manner based on the above embodiments, refer to Figure 6 As shown, this embodiment provides a three-dimensional object detection system for rainy and snowy weather, which is used to apply the three-dimensional object detection method for rainy and snowy weather, including: A data acquisition unit configured to acquire original LiDAR point cloud data; A feature extraction unit configured to construct a two-stream parallel feature extraction network; parallelly input the original LiDAR point cloud data into the geometric feature-maintaining main branch and the weather-aware auxiliary flow branch of the two-stream parallel feature extraction network; based on the geometric feature-maintaining main branch, perform voxelization on the original LiDAR point cloud data, and extract three-dimensional voxel geometric features through multi-branch cross convolution; based on the weather-aware auxiliary flow branch, perform three-view projection on the original LiDAR point cloud data, generate two-dimensional feature maps, and extract several two-dimensional weather feature masks; The feature processing unit is configured to generate a three-dimensional cylindrical mask from several two-dimensional weather feature masks; map the three-dimensional cylindrical mask to denoising attention weights; and obtain voxel features based on the denoising attention weights and three-dimensional voxel geometric features. The information generation unit is configured to compress voxel features into bird's-eye view features, perform global context modeling and local detail enhancement on the bird's-eye view features, and generate boundary information of the three-dimensional target based on the bird's-eye view features after global context modeling and local detail enhancement.

[0066] In summary, by constructing a dual-stream parallel feature extraction network, which processes the geometric feature-preserving main branch and the weather-sensing auxiliary branch in parallel, the network can fully utilize the geometric information of point clouds and precipitation noise features under severe weather conditions such as rain and snow, suppressing random noise interference caused by rain and snow particles, thereby improving the stability and reliability of 3D target detection. By extracting 3D voxel geometric features through voxelization processing of the geometric main branch and multi-branch cross-convolution, combined with three-view projection of the weather auxiliary branch and the generation and fusion of 2D weather feature masks, target features are enhanced in both spatial and intensity dimensions while preserving detailed information, improving target recognizability in rain and snow environments. The 2D weather feature mask is adaptively weighted and fused to generate a 3D cylindrical mask, which is then mapped to denoising attention weights aligned with the voxel geometric feature space. This suppresses non-target noise while ensuring that real target point clouds are not mistakenly deleted, optimizing the quality of feature representation. By compressing voxel features into bird's-eye view features and combining global context modeling with local detail enhancement operations, both global spatial relationships and local target details can be acquired simultaneously, enhancing the detection network's ability to identify targets in complex scenes. Based on the enhanced bird's-eye view features, the category, location, and orientation information of 3D targets are generated, enabling high-precision 3D boundary prediction even in adverse weather conditions.

[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A three-dimensional target detection method for rainy and snowy weather, characterized in that, include: Acquire raw LiDAR point cloud data; Construct a two-stream parallel feature extraction network; The original LiDAR point cloud data is input in parallel into the geometric features of the dual-stream parallel feature extraction network to maintain the main branch and the weather perception auxiliary branch. Based on the geometric features, the mainstream branches are maintained, the original LiDAR point cloud data is voxelized, and three-dimensional voxel geometric features are extracted by multi-branch cross convolution. Based on the weather-sensing auxiliary stream branch, the original LiDAR point cloud data is projected from three perspectives to generate a two-dimensional feature map, and several two-dimensional weather feature masks are extracted. Generate a three-dimensional cylindrical mask from several of the aforementioned two-dimensional weather feature masks; The three-dimensional cylindrical mask is mapped to denoising attention weights; Based on the denoising attention weights and three-dimensional voxel geometric features, voxel features are obtained; The voxel features are compressed into bird's-eye view features, and the bird's-eye view features are preprocessed. Based on the preprocessed bird's-eye view features, the boundary information of the three-dimensional target is generated.

2. The three-dimensional target detection method for rainy and snowy weather according to claim 1, characterized in that, When extracting 3D voxel geometric features by maintaining the mainstream branch based on the aforementioned geometric features and using multi-branch cross-convolution, the process includes: In the Cartesian coordinate system, the original LiDAR point cloud data is voxelized to obtain voxel data, and a lightweight multi-branch cross-sparse convolutional backbone network is used to extract features from the voxel data. The lightweight multi-branch cross-sparse convolution backbone network includes parallel sub-manifold sparse convolution paths and regular sparse convolution paths. Based on the active voxel positions in the voxel data, convolution calculations are performed in the sub-manifold sparse convolution path; based on the neighborhood space of the voxel data, feature expansion and receptive field enlargement processing are performed in the regular sparse convolution path; the output data of the sub-manifold sparse convolution path and the regular sparse convolution path are fused to obtain the three-dimensional voxel geometric features.

3. The three-dimensional target detection method for rainy and snowy weather according to claim 2, characterized in that, When generating a two-dimensional feature map based on the weather-sensing auxiliary stream branch, the following steps are included: The original LiDAR point cloud data in Cartesian coordinates is transformed to obtain cylindrical coordinate data. Based on the cylindrical coordinate data, the cylindrical space is divided into cylindrical voxel grids. The original LiDAR point cloud data within each cylindrical voxel grid is extracted, and local features are extracted through multilayer sensing and max pooling to obtain local feature vectors. Based on all the local feature vectors, a three-dimensional cylindrical feature is constructed, and compressed projection is performed along the radial, angular, and height axes to obtain the two-dimensional feature map. The two-dimensional feature map includes a circular bird's-eye view, a distance view, and an auxiliary view.

4. The three-dimensional target detection method for rainy and snowy weather according to claim 3, characterized in that, When extracting several two-dimensional weather feature masks, the following are included: Based on the circular bird's-eye view, distance view, and auxiliary view, feature channel compression processing is performed to obtain the scalar logarithmic probability corresponding to each pixel position; based on the scalar logarithmic probability, probability mapping processing is performed to obtain several two-dimensional weather feature masks.

5. The three-dimensional target detection method for rainy and snowy weather according to claim 4, characterized in that, When generating a three-dimensional cylindrical mask from several of the aforementioned two-dimensional weather feature masks, the following steps are included: The two-dimensional weather feature masks are sequentially subjected to dimensional expansion, weighted linear combination, and probability mapping to generate a three-dimensional cylindrical mask.

6. The three-dimensional target detection method for rainy and snowy weather according to claim 5, characterized in that, Mapping the three-dimensional cylindrical mask to denoising attention weights includes: Based on the non-empty voxels in the three-dimensional voxel geometric features, the Cartesian center coordinates are extracted and transformed into continuous spatial coordinates in cylindrical coordinates. Based on the continuous spatial coordinates and the three-dimensional cylindrical mask, the denoising attention weights are obtained through trilinear interpolation.

7. The three-dimensional target detection method for rainy and snowy weather according to claim 6, characterized in that, When obtaining voxel features, the following are included: Based on the denoising attention weights and the three-dimensional voxel geometric features, element-wise intelligent multiplication is performed to obtain the voxel features.

8. The three-dimensional target detection method for rainy and snowy weather according to claim 7, characterized in that, Preprocessing the bird's-eye view features includes: The preprocessing includes global context modeling and local detail enhancement.

9. The three-dimensional target detection method for rainy and snowy weather according to claim 8, characterized in that, When generating boundary information for a 3D target, the following are included: The bird's-eye view features are subjected to parallel convolution processing to obtain class probabilities, 3D bounding box regression parameters, and orientation classification information. Based on the class probabilities, 3D bounding box regression parameters, and orientation classification information, combined with preset anchor boxes, the position offset is predicted to generate the boundary information of the 3D target.

10. A three-dimensional target detection system for rainy and snowy weather, used to apply the three-dimensional target detection method for rainy and snowy weather as described in any one of claims 1-9, characterized in that, include: The data acquisition unit is configured to acquire raw LiDAR point cloud data; The feature extraction unit is configured to construct a two-stream parallel feature extraction network; The original LiDAR point cloud data is input in parallel into the geometric feature preservation main branch and the weather perception auxiliary branch of the dual-stream parallel feature extraction network; based on the geometric feature preservation main branch, the original LiDAR point cloud data is voxelized, and three-dimensional voxel geometric features are extracted through multi-branch cross convolution. Based on the weather-sensing auxiliary stream branch, the original LiDAR point cloud data is projected from three perspectives to generate a two-dimensional feature map, and several two-dimensional weather feature masks are extracted. The feature processing unit is configured to generate a three-dimensional cylindrical mask from a plurality of the two-dimensional weather feature masks. The three-dimensional cylindrical mask is mapped to denoising attention weights; Based on the denoising attention weights and three-dimensional voxel geometric features, voxel features are obtained; The information generation unit is configured to compress the voxel features into bird's-eye view features and preprocess the bird's-eye view features; Based on the preprocessed bird's-eye view features, the boundary information of the three-dimensional target is generated.

Images