A weather-adaptive point cloud 3D target detection method and apparatus
By using an interference perception network and a hybrid expert target detection model to dynamically select processing paths, the problems of accuracy and robustness in point cloud detection under adverse weather conditions are solved, achieving efficient and reliable target detection in autonomous driving environments.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- UNIV OF SCI & TECH BEIJING
- Filing Date
- 2026-02-12
- Publication Date
- 2026-05-26
Smart Images
Figure CN122090435A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of target detection technology, and in particular to a method and apparatus for detecting three-dimensional targets in point clouds that is weather-adaptive. Background Technology
[0002] With the rapid development of autonomous driving and intelligent robotics, LiDAR-based 3D target detection technology has become a core component of environmental perception. This technology aims to accurately identify and locate the 3D position, size, and orientation of vehicles, pedestrians, and obstacles from sparse and disordered point cloud data. Under ideal weather conditions, deep learning-based point cloud detection models have made significant progress, enabling high-precision and real-time detection.
[0003] However, under adverse weather conditions such as rain, snow, and fog, lidar measurements are severely interfered with by the scattering and absorption of atmospheric particles, mainly manifested in: 1) the introduction of a large number of discrete noise points; and 2) the loss of effective point clouds or signal attenuation on the surface of real objects. This interference severely damages the geometric structure and feature distribution of point clouds, causing a sharp decline in the performance of standard detection models trained in clear weather, constituting a major bottleneck for practical applications. To improve the robustness of point cloud target detection under adverse weather conditions, existing technologies mainly follow these typical approaches: 1. Data augmentation and domain adaptation methods: These methods artificially simulate adverse weather noise in the training data (e.g., using physical models to render rain, snow, and fog points), or employ domain adaptation techniques to adapt the model from the "clear weather domain" to the "adverse weather domain." These methods improve generalization ability to some extent, but the difference between simulated noise and real complex noise distribution remains a major challenge. 2. Denoising and repair-based detection methods: These methods design an independent preprocessing module to denoise, complete, or repair the interfered point cloud before inputting the purified point cloud into the standard detection model. These methods decouple the problem, but involve multiple processing steps, increasing system latency and computational burden. 3. Multimodal fusion-based methods: These attempt to fuse data from lidar, cameras, and millimeter-wave radar sensors, leveraging the complementarity of multi-source information to overcome the disadvantages of single sensors in specific weather conditions. While effective, this method increases system hardware costs, calibration complexity, and data fusion difficulty.
[0004] Current methods for point cloud target detection involve the following steps: first, performing severe weather simulation rendering on the collected point cloud to generate noisy training data; then, using a deep learning network to remove and optimize the noise in the rendered point cloud; finally, using an upsampling network to fill in the missing valid points caused by noise; and finally, inputting the processed point cloud into a 3D target detection model to obtain the result. However, this method is redundant in its point cloud denoising and completion calculations. This one-size-fits-all approach leads to a huge waste of computational resources, severely impairing the system's real-time performance and failing to meet the high real-time requirements of autonomous driving applications. Furthermore, this method lacks optimization for feature extraction based on the degree of interference. Regardless of whether the actual weather interference on the input point cloud is mild, moderate, or severe, the final detection stage still uses a generic feature extraction network. This network cannot adapt to point cloud features that differ fundamentally due to varying degrees of original interference, easily leading to loss of detail or residual noise, making it difficult to achieve the optimal balance between accuracy and robustness. Therefore, a novel weather-adaptive point cloud 3D target detection model is needed. Summary of the Invention
[0005] To address the technical problem that existing point cloud feature extraction methods lack optimization for varying levels of interference, leading to loss of feature details or residual noise, and further reducing the accuracy of recognition results in complex weather environments, this invention provides a weather-interference-adaptive point cloud 3D target detection method and apparatus. The technical solution is as follows:
[0006] On the one hand, a weather-interference-adaptive point cloud 3D target detection method is provided. This method is implemented by a weather-interference-adaptive point cloud 3D target detection device, and includes: S1. Construct a point cloud 3D target detection model based on weather interference adaptation; the model includes: interference perception network and hybrid expert target detection model; S2. Obtain raw point cloud data; convert the raw point cloud data into a distance image and input it into an interference perception network to classify the degree of interference and obtain a deterministic decision result on the interference level. S3. Based on the deterministic decision results of the interference level, a sparse activation mechanism is used to match the target detection expert model corresponding to the deterministic decision results of the interference level from the hybrid expert target detection model; S4. Input the original point cloud data into the matched target detection expert model for target detection, and output the 3D target detection result corresponding to the original point cloud data. S5. Based on the three-dimensional target detection results and interference level deterministic decision results corresponding to the original point cloud data, a phased training strategy is adopted to train the point cloud three-dimensional target detection model through the constructed total loss function to obtain the trained point cloud three-dimensional target detection model. S6. Acquire the point cloud data to be detected; input the point cloud data to be detected into the trained point cloud 3D target detection model to obtain the 3D target detection result corresponding to the point cloud data to be detected; wherein, the 3D target detection result includes: target category, position, size and orientation information.
[0007] On the other hand, a weather-interference-adaptive point cloud 3D target detection device is provided, which is applied to the weather-interference-adaptive point cloud 3D target detection method. The device includes: A construction unit is used to build a point cloud 3D target detection model based on weather interference adaptation; the model includes: an interference sensing network and a hybrid expert target detection model; The acquisition unit is used to acquire raw point cloud data; it converts the raw point cloud data into a distance image and inputs it into an interference perception network to classify the degree of interference and obtain a deterministic decision result on the interference level. The matching unit is used to match the target detection expert model corresponding to the deterministic decision result of the interference level from the hybrid expert target detection model using a sparse activation mechanism based on the deterministic decision result of the interference level. The first detection unit is used to input the original point cloud data into the matched target detection expert model for target detection and output the three-dimensional target detection result corresponding to the original point cloud data. The training unit is used to train the point cloud 3D target detection model based on the 3D target detection results and interference level deterministic decision results corresponding to the original point cloud data, using a phased training strategy and a constructed total loss function, to obtain a trained point cloud 3D target detection model. The second detection unit is used to acquire point cloud data to be detected; input the point cloud data to be detected into a trained point cloud 3D target detection model to obtain the 3D target detection result corresponding to the point cloud data to be detected; wherein, the 3D target detection result includes: target category, position, size and orientation information.
[0008] On the other hand, a weather-interference-adaptive point cloud 3D target detection device is provided, the weather-interference-adaptive point cloud 3D target detection device comprising: a processor; a memory, the memory storing computer-readable instructions, which, when executed by the processor, implement any of the methods described above for weather-interference-adaptive point cloud 3D target detection.
[0009] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, the at least one instruction being loaded and executed by a processor to implement any of the above-described methods for weather-interference-adaptive point cloud 3D target detection.
[0010] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: This invention achieves a dynamic optimal balance between accuracy and efficiency, significantly improving model real-time performance. It introduces an interference perception network with real-time decision-making capabilities, enabling the model to dynamically select the most suitable processing path based on the actual interference level of each frame's point cloud. Under conditions of good or mild interference, which constitute the majority of the runtime, the model activates a lightweight one-stage detection expert with minimal computational cost. This avoids the redundant and time-consuming general denoising and upsampling operations required in existing technologies, thus saving significant computational resources. This results in a substantial reduction in the model's average processing latency and a significant increase in overall throughput, better meeting the demanding real-time requirements of applications such as autonomous driving.
[0011] This invention comprehensively improves detection accuracy and robustness under complex weather interference. It abandons the "one-size-fits-all" approach of generic detectors and designs three deeply specialized detection expert models tailored to different interference levels and data characteristics. Under moderate interference, the attention-enhanced two-stage expert model utilizes an attention mechanism to achieve a balance between noise suppression and feature enhancement. Its two-stage refinement strategy significantly improves localization and classification accuracy in complex noisy environments. Under severe interference, the cascaded reconstruction-based detection expert model employs a series of measures, including point cloud purification, key point feature enhancement based on voxel set abstraction modules, local completion, and point-voxel deep fusion, to greatly enhance the ability to reconstruct and identify targets from severely degraded data, overcoming the performance bottleneck of general networks in extreme scenarios. This "specialized approach" design philosophy allows each expert model to achieve near-optimal performance within its assigned interference range, thereby improving the overall accuracy and robustness of the model under all-weather conditions.
[0012] The embodiments of this invention employ an end-to-end collaborative training method to achieve global optimization of perception, decision-making, and execution. The embodiments of this invention use an end-to-end training framework that combines Gumbel-Softmax and load balancing loss, enabling the interference perception network to learn simple interference classification. This training method effectively avoids the expert degradation problem and ensures the long-term stability of the model.
[0013] The embodiments of this invention improve the interpretability of the decision-making process; unlike existing technologies that rely on complex and black-box preprocessing pipelines, the decision-making process of these embodiments is clear and transparent. The interference level classification output by the interference perception network provides an intuitive and understandable basis for decision-making. This makes the model's behavior easier to analyze, debug, and trust, enhancing its acceptability and reliability in safety-critical areas such as autonomous driving.
[0014] The embodiments of the present invention effectively overcome the shortcomings of the prior art, such as being static, rigid, and generic. Under complex weather interference, it can improve detection accuracy, processing speed, and model robustness. Attached Figure Description
[0015] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0016] Figure 1 This is a flowchart of a weather-interference-adaptive point cloud 3D target detection method provided in an embodiment of the present invention; Figure 2 This is a flowchart of an interference sensing network provided in an embodiment of the present invention; Figure 3 This is a block diagram of a weather-interference-adaptive point cloud 3D target detection device provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of a weather-interference-adaptive point cloud 3D target detection device provided in an embodiment of the present invention. Detailed Implementation
[0017] The technical solution of the present invention will now be described with reference to the accompanying drawings.
[0018] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.
[0019] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.
[0020] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.
[0021] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.
[0022] This invention provides a weather-interference-adaptive point cloud 3D target detection method. This method can be implemented using a weather-interference-adaptive point cloud 3D target detection device, which can be a terminal or a server. Figure 1 The flowchart shown is for a weather-interference-adaptive point cloud 3D target detection method. The processing flow of this method may include the following steps:
[0023] S1. Construct a point cloud 3D target detection model based on weather interference adaptation; the model includes: interference perception network and hybrid expert target detection model.
[0024] Optionally, the interference-aware network includes: a feature encoder, a multi-level feature fusion layer, and a dual-task prediction head; Among them, the feature encoder uses a convolutional neural network to extract multi-level features of the distance image; In one feasible implementation, the convolutional neural network consists of five cascaded convolutional modules DMSConvBolck; each convolutional module DMSConvBolck includes four parallel convolutional layers, and by configuring convolutional kernels of different sizes and dilation rates, it can capture structural features of scenes of different sizes and shapes.
[0025] In this process, the spatial resolution remains constant throughout the convolution process, while the channel dimension is gradually increased to enhance the feature representation capability.
[0026] Among them, the multi-level feature fusion layer is used to splice the features output by the feature encoder to construct a multi-level feature map that combines low-level original signal information with high-level abstract features. The dual-task prediction head includes: a noise point segmentation head and an interference level classification head; Among them, the noise point segmentation head is used to output a pixel-level noise mask image; The interference level classification header is used to calculate the probability of weather interference categories.
[0027] In one feasible implementation, the interference perception network integrates noise point segmentation and interference assessment functions into a lightweight network architecture through multi-size dilation rate convolution feature extraction, multi-level feature fusion, and dual-task feature sharing design. This enables fast and accurate online decision-making and provides a reliable basis for subsequent expert model selection.
[0028] Optionally, a hybrid expert object detection model may be used, including: Lightweight one-stage detection expert model, attention-enhanced two-stage expert model, and cascaded reconstruction detection expert model; Among them, the lightweight one-stage detection expert model corresponds to mild interference; In one feasible implementation, a lightweight one-stage detection expert model is designed for mild interference or good weather conditions. It employs a PointPillars-based one-stage architecture to rapidly convert point cloud data into pseudo-images, which are then processed by a 2D convolutional neural network to output the target bounding box. Its advantages include extremely high speed, sufficient accuracy to guarantee accuracy when data quality is high, and maximized efficiency.
[0029] Among them, the attention-enhanced two-stage expert model corresponds to moderate interference; In one feasible implementation, the attention-enhanced two-stage expert model is designed for moderate interference. It employs a two-stage architecture, the core of which is the integration of channel and spatial attention mechanisms in the feature extraction stage, enabling the network to adaptively focus on effective regions and suppress noise. Its second stage utilizes multi-scale features to refine candidate boxes in parallel, rather than time-consuming point feature re-extraction, achieving an excellent balance between accuracy and efficiency.
[0030] Among them, the cascaded reconstruction detection expert model corresponds to severe interference.
[0031] In one feasible implementation, a cascaded reconstruction-based detection expert model is designed for heavy interference. It employs a complex cascaded architecture, following the principle of first cleaning, then enhancement, and finally fine-tuning. First, lightweight denoising is performed on the point cloud data to obtain denoised point cloud data. Based on the denoised point cloud data, a voxel set abstraction module is used to fuse multi-scale voxel context information to generate voxel features. Based on the BEV feature map generated from the voxel features, 3D candidate boxes are obtained through anchor box matching and classification regression branches. Local completion is performed on the point cloud within the 3D candidate boxes. Fine-tuning detection is then performed through deep fusion of point-voxel features. This expert model provides powerful feature reconstruction and recovery capabilities at the cost of higher computational cost, ensuring detection accuracy even under extreme weather conditions.
[0032] S2. Obtain raw point cloud data; convert the raw point cloud data into a distance image and input it into an interference perception network to classify the degree of interference and obtain a deterministic decision result on the interference level.
[0033] Optionally, the deterministic decision results for the interference level include: mild interference, moderate interference, and severe interference; in, Figure 2 This is a flowchart of an interference sensing network provided in an embodiment of the present invention.
[0034] Optionally, the specific implementation process of S2 includes: S21-S24: S21. Project the point cloud data to be detected to generate a two-dimensional distance image; input the two-dimensional distance image into the feature encoder in the interference perception network, and perform parallel feature extraction through 5 cascaded convolutional modules DMSConvBolck, and output the features extracted by each convolutional module. S22. Input the features extracted by each convolutional module into a multi-level feature fusion layer for feature concatenation to obtain a multi-level feature map; Among them, based on multi-level feature maps, an additional convolutional module DMSConvBolck is introduced, and all the convolutional kernels inside it are set to 1×1. This module reduces the number of channels in the multi-level feature maps, thereby achieving efficient integration of features and removal of redundant information.
[0035] S23. Input the multi-level feature map into the noise point segmentation head in the dual-task prediction head, use a 1×1 convolutional kernel as a weight-sharing classifier, and map the channel dimension of the multi-level feature map to 1 dimension while keeping the spatial resolution unchanged, and output a pixel-level noise mask map. S24. The pixel-level noise mask and multi-level feature maps are input into the interference level classification head and concatenated to obtain the concatenated feature map. Average pooling is performed on the concatenated feature map to obtain a one-dimensional feature vector. A two-layer multilayer perceptron is used to perform global feature fusion and dimension transformation on the one-dimensional feature vector, outputting a three-dimensional vector. Based on the three-dimensional vector, the probability of the weather interference category is calculated using the Softmax function. Based on the probability of the weather interference category, a deterministic decision result for the interference level is obtained.
[0036] S3. Based on the deterministic decision results of the interference level, a sparse activation mechanism is used to match the target detection expert model corresponding to the deterministic decision results of the interference level from the hybrid expert target detection model; S4. Input the original point cloud data into the matched target detection expert model for target detection, and output the 3D target detection results corresponding to the original point cloud data.
[0037] S5. Based on the 3D target detection results and interference level deterministic decision results corresponding to the original point cloud data, a phased training strategy is adopted. The point cloud 3D target detection model is trained through the constructed total loss function to obtain the trained point cloud 3D target detection model.
[0038] Optionally, the phased training includes: an independent network pre-training phase and an end-to-end joint fine-tuning phase; wherein, the independent network pre-training phase includes: interference-aware network warm-up training and independent pre-training of the expert model.
[0039] Optionally, the interference sensing network warm-up training process includes: A dataset containing noise point annotations and pseudo-labels of interference levels is obtained. Based on the deterministic decision results of interference levels, a multi-task loss function is constructed using binary cross-entropy loss and standard cross-entropy loss to train the interference perception network. In one feasible implementation, the multi-task loss function is expressed by the following formula (1): (1) Among them, L noise The noise point segmentation loss, used to supervise the training process of the noise point segmentation head, is expressed by the following formula (2): (2) Among them, M pred M represents the pixel-level noise probability map predicted by the interference sensing network. gt The corresponding real noise point binary mask label is obtained through physical simulation; BCEWithLogitsLoss is the binary cross loss, which is used to enable the interference sensing network to accurately distinguish between valid points and weather noise points.
[0040] Among them, L gate The interference classification loss is used to guide the training of the interference level classification head of the interference sensing network, so that its predictions approach the pseudo-labels based on physical rules, and is expressed by the following formula (3): (3) Among them, P gate Y represents the probability distribution of the deterministic decision result of the interference level output by the interference sensing network; pseudo This represents a pseudo-label of the interference level generated based on multiple indicators, including the proportion of noise points and intensity attenuation. During the warm-up training phase, this loss is crucial for stabilizing the initial prediction behavior of the interference sensing network.
[0041] The independent pre-training process of the expert model includes: Obtain the datasets corresponding to the three object detection expert models in the hybrid expert object detection model. Based on the 3D object detection results corresponding to the original point cloud data, construct the target loss function for each object detection expert model using classification loss and regression loss. Train the three object detection expert models independently based on the corresponding datasets and target loss functions. The target loss function corresponding to each target detection expert model is expressed by the following formula (4): (4) Among them, L cls L represents the classification loss, used to mitigate the extreme imbalance between foreground and background classes; reg The regression loss is used to optimize the position, size, and orientation of the predicted bounding box; In one feasible implementation, the total loss function of the target detection expert model is expressed by the following formula (5): (5) in, Let i represent the target loss function corresponding to each object detection expert model, and let i represent the number of object detection expert models.
[0042] The end-to-end joint fine-tuning process includes: Based on the multi-task loss function, the target loss functions corresponding to the three target detection expert models, and the load balancing loss function, an overall loss function is constructed; end-to-end joint fine-tuning is performed based on the overall loss function; wherein, the overall loss function is expressed by the following formula (6): (6) Among them, L total Represents the overall loss function from end to end; This represents the first balancing weight coefficient; This represents the second balancing weight coefficient; L represents the third balancing weight coefficient; multi L represents the multi-task loss function; det L represents the total loss function of the three object detection expert models; balance This represents the load balancing loss function.
[0043] In one feasible implementation, the load balancing loss function serves as a key regularization term in the sparse activation hybrid expert system. Its core purpose is to prevent the routing network from excessively favoring a particular expert network, thus avoiding performance degradation in other expert networks due to insufficient training. The load balancing loss function is expressed by the following formula (7):
[0044] (7) in, Expert model E k The probability of being selected within this batch; denoted as the ideal probability of selection for each expert under a uniform distribution; N represents the sample size in a batch; KL represents the KL divergence; p represents the probability of selection; and u represents the ideal probability of selection.
[0045] In one feasible implementation, the total loss function can simultaneously ensure the accuracy of interference perception, the load balancing of the expert model, and the accuracy of final target detection through the synergistic effect of multiple sub-items.
[0046] In the forward propagation process during training, the Gumbel-Softmax method is used to perform differentiable sampling of the classification output of the interference sensing network, generating soft routing weights. This enables the effective backpropagation of gradients from the detection loss to the routing network, guiding it to learn routing decisions that maximize overall detection accuracy. Simultaneously, a load balancing loss is introduced to prevent the routing network from becoming overly biased towards a single expert model, avoiding undertraining of other expert models (i.e., the expert "degeneration" problem). The load balancing loss plays a crucial regulatory role at this stage, effectively preventing routing decisions from collapsing to a single expert model and ensuring a balanced training load across all expert models.
[0047] In this embodiment of the invention, through end-to-end joint optimization, the interference sensing network can learn the optimal routing decision strategy based on the final detection accuracy target, and the feature extraction capabilities of each expert model are further refined and synergistically improved.
[0048] In this embodiment of the invention, during the inference phase, the interference perception network outputs the confidence level of each interference category, selects the deterministic decision result of the interference level with the highest confidence level, and sparsely activates the corresponding single expert model. The activated expert model receives point cloud data and executes the target detection process, while other expert models do not participate in the calculation. This process requires no loss calculation and can achieve efficient and adaptive real-time inference.
[0049] S6. Obtain the point cloud data to be detected; input the point cloud data to be detected into the trained point cloud 3D target detection model to obtain the 3D target detection result corresponding to the point cloud data to be detected; wherein, the 3D target detection result includes: target category, position, size and orientation information.
[0050] In one feasible implementation, the point cloud data to be detected is converted into a distance image and input into an interference perception network for interference level classification to obtain a deterministic decision result of the interference level; based on the deterministic decision result of the interference level, a sparse activation mechanism is used to match the target detection expert model corresponding to the deterministic decision result of the interference level from the hybrid expert target detection model. In this sparse activation mechanism, for each input sample, the k experts with the highest weights in the expert pool are dynamically activated (k is usually 1 or 2, and k is 1 in this embodiment of the invention), rather than activating all experts. The remaining experts do not participate in the calculation of the current input. The final output is obtained by weighted fusion of the results of the activated experts, thereby decoupling the model capacity and the computational cost, and controlling the computational cost while improving the model's capabilities.
[0051] Specifically, when the deterministic decision result for interference level is mild interference, a lightweight one-stage detection expert model is matched. The point cloud data to be detected is input into the lightweight one-stage detection expert model for target detection. By adopting a PointPillars-based one-stage architecture, the point cloud is discretized into vertical pillars, and pillar point cloud data is output. The features of each pillar point are processed through linear layers, batch normalization, and ReLU activation function, and the features of all points in the pillar are aggregated to obtain the global features of each pillar. The global features are mapped to dense 2D pseudo-images, and pseudo-image feature maps are output. Based on the pseudo-image feature maps, high-level semantic features are extracted step by step through 2D convolutional neural networks and multiple sets of convolutional blocks. Based on the high-level semantic features, features of different resolutions are unified in scale and concatenated through transposed convolution to obtain strong expressive features that fuse multi-scale information, and high-dimensional concatenated features are output. Two prediction heads are set: one prediction head is the classification head of the category to which the target box belongs, and the other prediction head is the regression head of the target box position. Both prediction heads employ a two-layer multilayer perceptron structure, which maps the concatenated features to the required vector length for prediction through progressive dimensionality reduction, and outputs the class confidence and 3D target box position parameters, respectively.
[0052] Specifically, when the deterministic decision result for interference level is moderate interference, a two-stage expert model with enhanced attention is used. The point cloud data to be detected is input into the two-stage expert model for target detection. The two-stage architecture integrates channel and spatial attention mechanisms in the feature extraction stage, enabling the network to adaptively focus on the effective region and suppress noise. The specific implementation process includes:
[0053] First, the point cloud data is uniformly divided into a fixed-size voxel grid to obtain voxel point cloud data. Then, a channel attention mechanism is used to obtain channel-level responses by max pooling along the point dimension. Channel weights are generated by passing through two fully connected layers and ReLU activation. These weights are then multiplied element-wise with the original features to obtain channel-enhanced voxel features. A spatial attention mechanism is employed to generate a spatial response map by max pooling the channel-enhanced voxel features along the channel dimension. The spatial response map is then generated by convolution and sigmoid activation, and multiplied element-wise with the input features to obtain the spatially enhanced voxel features. The spatially enhanced voxel features are then aggregated by max pooling, arranged according to their spatial positions, and output as a global feature map. Two-stage processing is performed based on the global feature map. Specifically, the first stage performs coarse regression of the target box, extracts multi-scale features from the global feature map through multiple sets of convolutional blocks and concatenates them to obtain a multi-scale fused feature map, and outputs coarse localization 3D candidate boxes based on the multi-scale fused feature map. The second stage performs fine regression of the target bounding boxes, employing a pyramid sampling aggregation structure. It upsamples and concatenates the multi-scale fusion feature maps obtained in the first stage, fusing high- and low-level semantic information and spatial resolution to output a unified-scale cross-layer fusion feature map. The cross-layer fusion feature map undergoes secondary classification and coordinate correction. Through two 3×3 convolutional layers and one fully connected layer, the class confidence of the candidate bounding boxes is output. Using the 3D candidate bounding boxes output by coarse regression as new anchor points, parallel convolutional and fully connected branches are used to output the 3D coordinate residuals of the candidate bounding boxes relative to the new anchor points. Minimizing the residual error achieves accurate localization.
[0054] Specifically, when the deterministic decision result for the interference level is severe interference, a cascaded reconstruction detection expert model is matched, and the point cloud data to be detected is input into the cascaded reconstruction detection expert model for target detection. The specific implementation process includes: adopting a complex cascaded architecture, following the principle of first cleaning, then enhancing, and then fine-checking; and combining the noise mask image output by the noise point segmentation head of the interference sensing network to obtain the denoised point cloud data. A 3D voxelization partitioning and sparse convolutional coding structure is used to divide the denoised point cloud data into fixed-size voxels to obtain multi-scale voxel features. A voxel set abstraction module is used to select representative key points by sampling the farthest point, and the multi-scale voxel features and original point features are aggregated to output key point features. Based on the BEV feature map generated from voxel features, 3D candidate boxes are obtained through anchor box matching and classification regression branches. A point cloud completion network is then used to locally complete the point cloud within the candidate boxes. The implementation process of the point cloud completion network includes: firstly, using attention-based dual-branch encoding, robust global features are extracted from the sparse point cloud within the candidate boxes using a multilayer perceptron global encoder, and long-range geometric dependencies are captured using an attention-based global encoder to obtain complementary features from the two branches, outputting local point cloud features; based on the local point cloud features, cascaded attention is used for feature fusion, outputting the fused global features; finally, multi-stage refinement decoding is used, progressively upsampling and graph convolutional local feature encoding to gradually complete the missing regions, outputting the complete point cloud data within the candidate boxes.
[0055] A point-voxel feature deep fusion method is adopted to aggregate the completed local point cloud features with voxel semantic features. Combined with the region of interest pooling module to encode multi-scale context information, the region of interest features are obtained. Based on these features, the candidate box position parameters and category confidence are corrected through a two-layer multilayer perceptron.
[0056] This invention achieves a dynamic optimal balance between accuracy and efficiency, significantly improving model real-time performance. It introduces an interference perception network with real-time decision-making capabilities, enabling the model to dynamically select the most suitable processing path based on the actual interference level of each frame's point cloud. Under conditions of good or mild interference, which constitute the majority of the runtime, the model activates a lightweight one-stage detection expert with minimal computational cost. This avoids the redundant and time-consuming general denoising and upsampling operations required in existing technologies, thus saving significant computational resources. This results in a substantial reduction in the model's average processing latency and a significant increase in overall throughput, better meeting the demanding real-time requirements of applications such as autonomous driving.
[0057] This invention comprehensively improves detection accuracy and robustness under complex weather interference. It abandons the "one-size-fits-all" approach of generic detectors and designs three deeply specialized detection expert models tailored to different interference levels and data characteristics. Under moderate interference, the attention-enhanced two-stage expert model utilizes an attention mechanism to achieve a balance between noise suppression and feature enhancement. Its two-stage refinement strategy significantly improves localization and classification accuracy in complex noisy environments. Under severe interference, the cascaded reconstruction-based detection expert model employs a series of measures, including point cloud purification, key point feature enhancement based on voxel set abstraction modules, local completion, and point-voxel deep fusion, to greatly enhance the ability to reconstruct and identify targets from severely degraded data, overcoming the performance bottleneck of general networks in extreme scenarios. This "specialized approach" design philosophy allows each expert model to achieve near-optimal performance within its assigned interference range, thereby improving the overall accuracy and robustness of the model under all-weather conditions.
[0058] The embodiments of this invention employ an end-to-end collaborative training method to achieve global optimization of perception, decision-making, and execution. The embodiments of this invention use an end-to-end training framework that combines Gumbel-Softmax and load balancing loss, enabling the interference perception network to learn simple interference classification. This training method effectively avoids the expert degradation problem and ensures the long-term stability of the model.
[0059] The embodiments of this invention improve the interpretability of the decision-making process; unlike existing technologies that rely on complex and black-box preprocessing pipelines, the decision-making process of these embodiments is clear and transparent. The interference level classification output by the interference perception network provides an intuitive and understandable basis for decision-making. This makes the model's behavior easier to analyze, debug, and trust, enhancing its acceptability and reliability in safety-critical areas such as autonomous driving.
[0060] The embodiments of the present invention effectively overcome the shortcomings of the prior art, such as being static, rigid, and generic. Under complex weather interference, it can improve detection accuracy, processing speed, and model robustness.
[0061] Figure 3 This is a block diagram of a weather-interference-adaptive point cloud 3D target detection device provided in an embodiment of the present invention. This device is used in a weather-interference-adaptive point cloud 3D target detection method. (Refer to...) Figure 3 The device includes a construction unit 310, an acquisition unit 320, a matching unit 330, a first detection unit 340, a training unit 350, and a second detection unit 360. Wherein:
[0062] The construction unit 310 is used to construct a point cloud 3D target detection model based on weather interference adaptation; the model includes: an interference sensing network and a hybrid expert target detection model; The acquisition unit 320 is used to acquire raw point cloud data; the raw point cloud data is converted into a distance image and input into the interference perception network for interference degree classification, and the interference level deterministic decision result is obtained; The matching unit 330 is used to match the target detection expert model corresponding to the deterministic decision result of the interference level from the hybrid expert target detection model using a sparse activation mechanism based on the deterministic decision result of the interference level. The first detection unit 340 is used to input the original point cloud data into the matched target detection expert model for target detection and output the three-dimensional target detection result corresponding to the original point cloud data. Training unit 350 is used to train the point cloud 3D target detection model based on the 3D target detection results and interference level deterministic decision results corresponding to the original point cloud data, using a phased training strategy and a constructed total loss function, to obtain a trained point cloud 3D target detection model. The second detection unit 360 is used to acquire point cloud data to be detected; input the point cloud data to be detected into a trained point cloud 3D target detection model to obtain the 3D target detection result corresponding to the point cloud data to be detected; wherein, the 3D target detection result includes: target category, position, size and orientation information.
[0063] Optionally, the deterministic decision result of the interference level includes: mild interference, moderate interference, and severe interference; Optionally, the hybrid expert object detection model includes: Lightweight one-stage detection expert model, attention-enhanced two-stage expert model, and cascaded reconstruction detection expert model; Among them, the lightweight one-stage detection expert model corresponds to mild interference; Among them, the attention-enhanced two-stage expert model corresponds to moderate interference; Among them, the cascaded reconstruction detection expert model corresponds to severe interference.
[0064] Optionally, the interference sensing network includes: a feature encoder, a multi-level feature fusion layer, and a dual-task prediction head; Among them, the feature encoder uses a convolutional neural network to extract multi-level features of the distance image; Among them, the multi-level feature fusion layer is used to splice the features output by the feature encoder to construct a multi-level feature map that combines low-level original signal information with high-level abstract features. The dual-task prediction head includes: a noise point segmentation head and an interference level classification head; Among them, the noise point segmentation head is used to output a pixel-level noise mask image; The interference level classification header is used to calculate the probability of weather interference categories.
[0065] Optionally, the acquisition unit 320 is used for: The point cloud data to be detected is projected to generate a two-dimensional distance image; the two-dimensional distance image is input into the feature encoder in the interference perception network, and parallel feature extraction is performed through 5 cascaded convolutional modules DMSConvBolck, and the features extracted by each convolutional module are output. The features extracted by each convolutional module are input into a multi-level feature fusion layer for feature concatenation to obtain a multi-level feature map; The multi-level feature map is input into the noise point segmentation head in the dual-task prediction head. A 1×1 convolutional kernel is used as a weight-sharing classifier. While keeping the spatial resolution unchanged, the channel dimension of the multi-level feature map is mapped to 1 dimension, and a pixel-level noise mask map is output. A pixel-level noise mask and a multi-level feature map are input into an interference level classification head and concatenated to obtain a concatenated feature map. Average pooling is then performed on the concatenated feature map to obtain a one-dimensional feature vector. A two-layer multilayer perceptron is used to perform global feature fusion and dimensionality transformation on the one-dimensional feature vector, outputting a three-dimensional vector. Based on the three-dimensional vector, the probability of the weather interference category is calculated using the Softmax function. Based on the probability of the weather interference category, a deterministic decision result for the interference level is obtained.
[0066] Optionally, the phased training includes: an independent network pre-training phase and an end-to-end joint fine-tuning phase; wherein, the independent network pre-training phase includes: interference-aware network warm-up training and independent pre-training of expert models.
[0067] Optionally, the preheating training process of the interference sensing network includes: A dataset containing noise point annotations and pseudo-labels of interference levels is obtained. Based on the deterministic decision results of interference levels, a multi-task loss function is constructed using binary cross-entropy loss and standard cross-entropy loss to train the interference perception network. The independent pre-training process of the expert model includes: Obtain the datasets corresponding to the three object detection expert models in the hybrid expert object detection model. Based on the 3D object detection results corresponding to the original point cloud data, construct the target loss function for each object detection expert model using classification loss and regression loss. Train the three object detection expert models independently based on the corresponding datasets and target loss functions. The end-to-end joint fine-tuning process includes: Based on the multi-task loss function, the target loss functions corresponding to the three target detection expert models, and the load balancing loss function, an overall loss function is constructed; end-to-end joint fine-tuning is performed based on the overall loss function; wherein, the overall loss function is expressed by the following formula (1): (1) Among them, L total Represents the overall loss function from end to end; This represents the first balancing weight coefficient; This represents the second balancing weight coefficient; L represents the third balancing weight coefficient; multi L represents the multi-task loss function; det L represents the total loss function of the three object detection expert models; balance This represents the load balancing loss function.
[0068] This invention achieves a dynamic optimal balance between accuracy and efficiency, significantly improving model real-time performance. It introduces an interference perception network with real-time decision-making capabilities, enabling the model to dynamically select the most suitable processing path based on the actual interference level of each frame's point cloud. Under conditions of good or mild interference, which constitute the majority of the runtime, the model activates a lightweight one-stage detection expert with minimal computational cost. This avoids the redundant and time-consuming general denoising and upsampling operations required in existing technologies, thus saving significant computational resources. This results in a substantial reduction in the model's average processing latency and a significant increase in overall throughput, better meeting the demanding real-time requirements of applications such as autonomous driving.
[0069] This invention comprehensively improves detection accuracy and robustness under complex weather interference. It abandons the "one-size-fits-all" approach of generic detectors and designs three deeply specialized detection expert models tailored to different interference levels and data characteristics. Under moderate interference, the attention-enhanced two-stage expert model utilizes an attention mechanism to achieve a balance between noise suppression and feature enhancement. Its two-stage refinement strategy significantly improves localization and classification accuracy in complex noisy environments. Under severe interference, the cascaded reconstruction-based detection expert model employs a series of measures, including point cloud purification, key point feature enhancement based on voxel set abstraction modules, local completion, and point-voxel deep fusion, to greatly enhance the ability to reconstruct and identify targets from severely degraded data, overcoming the performance bottleneck of general networks in extreme scenarios. This "specialized approach" design philosophy allows each expert model to achieve near-optimal performance within its assigned interference range, thereby improving the overall accuracy and robustness of the model under all-weather conditions.
[0070] The embodiments of this invention employ an end-to-end collaborative training method to achieve global optimization of perception, decision-making, and execution. The embodiments of this invention use an end-to-end training framework that combines Gumbel-Softmax and load balancing loss, enabling the interference perception network to learn simple interference classification. This training method effectively avoids the expert degradation problem and ensures the long-term stability of the model.
[0071] The embodiments of this invention improve the interpretability of the decision-making process; unlike existing technologies that rely on complex and black-box preprocessing pipelines, the decision-making process of these embodiments is clear and transparent. The interference level classification output by the interference perception network provides an intuitive and understandable basis for decision-making. This makes the model's behavior easier to analyze, debug, and trust, enhancing its acceptability and reliability in safety-critical areas such as autonomous driving.
[0072] The embodiments of the present invention effectively overcome the shortcomings of the prior art, such as being static, rigid, and generic. Under complex weather interference, it can improve detection accuracy, processing speed, and model robustness.
[0073] Figure 4 This is a schematic diagram of the structure of a weather-interference-adaptive point cloud 3D target detection device provided in an embodiment of the present invention, as shown below. Figure 4 As shown, a weather-interference-adaptive point cloud 3D target detection device may include the above-mentioned... Figure 3 The illustrated weather-interference adaptive point cloud 3D target detection device. Optionally, the weather-interference adaptive point cloud 3D target detection device 410 may include a first processor 2001.
[0074] Optionally, the weather-interference-adaptive point cloud 3D target detection device 410 may also include a memory 2002 and a transceiver 2003.
[0075] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.
[0076] The following is combined with Figure 4 The components of the weather-adaptive point cloud 3D target detection device 410 are described in detail below: The first processor 2001 is the control center of the weather interference adaptive point cloud 3D target detection device 410. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).
[0077] Optionally, the first processor 2001 can perform various functions of the weather interference adaptive point cloud 3D target detection device 410 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.
[0078] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 4 CPU0 and CPU1 are shown in the diagram.
[0079] In a specific implementation, as one example, the weather-interference-adaptive point cloud 3D target detection device 410 may also include multiple processors, for example... Figure 4 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).
[0080] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.
[0081] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or exist independently, and may be connected via the interface circuit of the weather interference adaptive point cloud 3D target detection device 410. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.
[0082] The transceiver 2003 is used to communicate with network devices or with terminal devices.
[0083] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 4 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.
[0084] Optionally, the transceiver 2003 can be integrated with the first processor 2001 or exist independently, and can be connected to the interface circuit of the weather interference adaptive point cloud 3D target detection device 410. Figure 4 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.
[0085] It should be noted that, Figure 4 The structure of the weather interference adaptive point cloud 3D target detection device 410 shown does not constitute a limitation on the router. Actual weather interference adaptive point cloud 3D target detection devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0086] Furthermore, the technical effect of the weather interference adaptive point cloud 3D target detection device 410 can be referred to the technical effect of the weather interference adaptive point cloud 3D target detection method described in the above method embodiments, and will not be repeated here.
[0087] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or it may be any conventional processor, etc.
[0088] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).
[0089] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.
[0090] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.
[0091] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.
[0092] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0093] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.
[0094] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0095] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0096] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0097] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
[0098] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0099] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A weather-interference-adaptive point cloud 3D target detection method, characterized in that, The method includes: S1. Construct a point cloud 3D target detection model based on weather interference adaptation; the model includes: interference perception network and hybrid expert target detection model; S2. Obtain raw point cloud data; convert the raw point cloud data into a distance image and input it into an interference perception network to classify the degree of interference and obtain a deterministic decision result on the interference level. S3. Based on the deterministic decision results of the interference level, a sparse activation mechanism is used to match the target detection expert model corresponding to the deterministic decision results of the interference level from the hybrid expert target detection model; S4. Input the original point cloud data into the matched target detection expert model for target detection, and output the 3D target detection result corresponding to the original point cloud data. S5. Based on the three-dimensional target detection results and interference level deterministic decision results corresponding to the original point cloud data, a phased training strategy is adopted to train the point cloud three-dimensional target detection model through the constructed total loss function to obtain the trained point cloud three-dimensional target detection model. S6. Acquire the point cloud data to be detected; input the point cloud data to be detected into the trained point cloud 3D target detection model to obtain the 3D target detection result corresponding to the point cloud data to be detected; wherein, the 3D target detection result includes: target category, position, size and orientation information.
2. The weather-interference-adaptive point cloud 3D target detection method according to claim 1, characterized in that, The deterministic decision results for the interference level include: mild interference, moderate interference, and severe interference.
3. The weather-interference-adaptive point cloud 3D target detection method according to claim 1, characterized in that, The hybrid expert object detection model includes: Lightweight one-stage detection expert model, attention-enhanced two-stage expert model, and cascaded reconstruction detection expert model; Among them, the lightweight one-stage detection expert model corresponds to mild interference; Among them, the attention-enhanced two-stage expert model corresponds to moderate interference; Among them, the cascaded reconstruction detection expert model corresponds to severe interference.
4. The weather-interference-adaptive point cloud 3D target detection method according to claim 1, characterized in that, The interference perception network includes: a feature encoder, a multi-level feature fusion layer, and a dual-task prediction head; Among them, the feature encoder uses a convolutional neural network to extract multi-level features of the distance image; Among them, the multi-level feature fusion layer is used to splice the features output by the feature encoder to construct a multi-level feature map that combines low-level original signal information with high-level abstract features. The dual-task prediction head includes: a noise point segmentation head and an interference level classification head; Among them, the noise point segmentation head is used to output a pixel-level noise mask image; The interference level classification header is used to calculate the probability of weather interference categories.
5. The weather-interference-adaptive point cloud 3D target detection method according to claim 4, characterized in that, S2 transforms the raw point cloud data into a distance image and inputs it into an interference perception network for interference level classification, obtaining a deterministic decision result for the interference level, including: S21. Project the point cloud data to be detected to generate a two-dimensional distance image; input the two-dimensional distance image into the feature encoder in the interference perception network, and perform parallel feature extraction through 5 cascaded convolutional modules DMSConvBolck, and output the features extracted by each convolutional module. S22. Input the features extracted by each convolutional module into a multi-level feature fusion layer for feature concatenation to obtain a multi-level feature map; S23. Input the multi-level feature map into the noise point segmentation head in the dual-task prediction head, use a 1×1 convolutional kernel as a weight-sharing classifier, and map the channel dimension of the multi-level feature map to 1 dimension while keeping the spatial resolution unchanged, and output a pixel-level noise mask map. S24. Input the pixel-level noise mask and multi-level feature map into the interference level classification head for concatenation to obtain the concatenated feature map; perform average pooling on the concatenated feature map to obtain a one-dimensional feature vector; use a two-layer multilayer perceptron to perform global feature fusion and dimension transformation on the one-dimensional feature vector to output a three-dimensional vector; based on the three-dimensional vector, calculate the probability of the weather interference category using the Softmax function; based on the probability of the weather interference category, obtain the deterministic decision result of the interference level.
6. The weather-interference-adaptive point cloud 3D target detection method according to claim 1, characterized in that, The phased training includes: an independent network pre-training phase and an end-to-end joint fine-tuning phase; wherein, the independent network pre-training phase includes: interference-aware network warm-up training and independent pre-training of expert models.
7. The weather-interference-adaptive point cloud 3D target detection method according to claim 6, characterized in that, The preheating training process of the interference sensing network includes: A dataset containing noise point annotations and pseudo-labels of interference levels is obtained. Based on the deterministic decision results of interference levels, a multi-task loss function is constructed using binary cross-entropy loss and standard cross-entropy loss to train the interference perception network. The independent pre-training process of the expert model includes: Obtain the datasets corresponding to the three object detection expert models in the hybrid expert object detection model. Based on the 3D object detection results corresponding to the original point cloud data, construct the target loss function for each object detection expert model using classification loss and regression loss. Train the three object detection expert models independently based on the corresponding datasets and target loss functions. The end-to-end joint fine-tuning process includes: Based on the multi-task loss function, the target loss functions corresponding to the three target detection expert models, and the load balancing loss function, an overall loss function is constructed; end-to-end joint fine-tuning is performed based on the overall loss function; wherein, the overall loss function is expressed by the following formula (1): （1） Among them, L total Represents the overall loss function from end to end; This represents the first balancing weight coefficient; This represents the second balancing weight coefficient; L represents the third balancing weight coefficient; multi L represents the multi-task loss function; det L represents the total loss function of the three object detection models; balance This represents the load balancing loss function.
8. A weather-interference-adaptive point cloud 3D target detection device, wherein the weather-interference-adaptive point cloud 3D target detection device is used to implement the weather-interference-adaptive point cloud 3D target detection method as described in any one of claims 1-7, characterized in that, The device includes: A construction unit is used to build a point cloud 3D target detection model based on weather interference adaptation; the model includes: an interference sensing network and a hybrid expert target detection model; The acquisition unit is used to acquire raw point cloud data; it converts the raw point cloud data into a distance image and inputs it into an interference perception network to classify the degree of interference and obtain a deterministic decision result on the interference level. The matching unit is used to match the target detection expert model corresponding to the deterministic decision result of the interference level from the hybrid expert target detection model using a sparse activation mechanism based on the deterministic decision result of the interference level. The first detection unit is used to input the original point cloud data into the matched target detection expert model for target detection and output the three-dimensional target detection result corresponding to the original point cloud data. The training unit is used to train the point cloud 3D target detection model based on the 3D target detection results and interference level deterministic decision results corresponding to the original point cloud data, using a phased training strategy and a constructed total loss function, to obtain a trained point cloud 3D target detection model. The second detection unit is used to acquire point cloud data to be detected; input the point cloud data to be detected into a trained point cloud 3D target detection model to obtain the 3D target detection result corresponding to the point cloud data to be detected; wherein, the 3D target detection result includes: target category, position, size and orientation information.
9. A weather-interference-adaptive point cloud 3D target detection device, characterized in that, The weather-interference-adaptive point cloud 3D target detection device includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 7.