Target identification method based on weak feature enhancement in complex environment

Abstract

A target identification method based on weak feature enhancement in a complex environment comprises the following steps: firstly, carrying out multi-scale feature extraction and defuzzification feature enhancement on multi-sensor data by fusing multi-sensor information of a laser radar, a millimeter wave radar, a visible light camera, an infrared camera and the like; then multi-scale feature adaptive fusion and heterogeneous scale feature alignment are carried out; and realizing enhanced fusion of multi-dimensional weak features by using a multi-scale enhanced learning fusion network based on cross attention. And finally, accurate identification of the target is completed based on a deep learning target detection algorithm. The method can effectively cope with complex conditions such as smog, dust, light rain and night, and realizes stable and accurate identification of targets such as personnel and vehicles.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent mobile robots or unmanned vehicles, in particular to a target recognition method based on weak feature enhancement in complex environments. BACKGROUND

[0002] In the field and urban environment, there are many moving targets such as personnel and vehicles. In order to achieve high-speed autonomous travel of intelligent mobile robots or unmanned vehicles in complex and diverse scenes, it is necessary to accurately identify the above dynamic targets.

[0003] However, in complex environments such as smoke, dust, light rain, night, etc., single perception sensors are prone to failure, failure, pollution, obstruction, noise, etc. In addition, the target itself also has phenomena such as blur, distortion and weak features. Therefore, how to accurately identify the target in complex environments has become a key technical problem to be solved in the field of intelligent robots and unmanned vehicles. SUMMARY

[0004] The present application provides a target recognition method based on weak feature enhancement in complex environments, which can accurately identify targets such as personnel and vehicles in complex environments such as smoke, dust, light rain, night, etc. The method fuses multi-sensor information such as laser radar, millimeter wave radar, visible light camera, infrared camera, etc. to realize target recognition in complex environments, which specifically includes the following steps: First, multi-scale feature extraction and deblurring feature enhancement are performed on multi-sensor such as laser radar, millimeter wave radar, visible light camera and infrared camera; on this basis, multi-scale feature adaptive fusion and heterogeneous scale feature alignment processing are performed; then, a multi-scale enhancement learning fusion network based on cross attention is used to enhance and fuse weak features in multi-dimensional scale; finally, based on the fused and enhanced features, a deep learning target detection and recognition algorithm is used to complete accurate target recognition.

[0005] The specific steps include: S1, on the basis of multi-sensor joint calibration, different network structures are used to extract multi-scale features and deblurring feature enhancement from multi-sensor data; S2, the extracted multi-scale features are adaptively fused and aligned with heterogeneous scale features; S3, on the basis of multi-scale feature fusion, for the problem of weak ground feature caused by strong confrontation in harsh environment, a multi-scale enhancement learning fusion network is used to enhance and fuse weak features; S4, the features obtained after enhancement and fusion are used to realize target recognition by using a deep learning based target detection and recognition algorithm.

[0006] Further, the multi-sensor includes: laser radar, millimeter wave radar, visible light camera, infrared camera, etc.

[0007] Further, in the step S1, for the visible light image, an end-to-end residual aggregation network based on convolutional neural network is constructed for image deblurring enhancement processing, the network structure of the residual aggregation network includes four parts: low-quality picture decomposition, low-dimensional feature extraction, residual aggregation module and residual map generation; the specific processing steps include: Let respectively represent the low-quality picture, the low-frequency feature and the high-frequency feature, the high-frequency feature includes one or more of dust, rain, snow, fog, haze and noise in the image; (1) First, the low-quality picture is filtered using median filtering to separate the low-frequency information from the low-quality picture Then, the low-quality picture is subtracted from the low-frequency information to obtain the high-frequency information :

[0008] (2) Then, the low-dimensional feature extractor is used to extract the low-dimensional feature from the high-frequency information:

[0009] Wherein, Convolution-Batch Normalization-relu activation function combination operation, the operation input high-frequency information , output feature map ; (3) Extract high-dimensional features through stacked multiple residual aggregation modules, the feature maps extracted by the residual aggregation modules are transmitted to the last residual transformation module using dense connection, and more efficient high-dimensional features are extracted through feature compression and distillation:

[0010] Wherein, Residual aggregation module is used to extract features, Stacked block residual aggregation module, High-dimensional features output by stacking multiple residual aggregation modules and using local hierarchical features for extraction and utilization; Each of the residual aggregation modules comprises a residual feature extractor, a dense transformation layer and a local feature fusion, wherein: the residual feature extractor extracts different feature maps using a residual module; the dense transformation layer combines the feature maps extracted by the residual feature extractor together, compresses and distills the features using a convolution operation with a convolution of 1*1; and the local feature fusion connects the input of each residual aggregation module to the input of the dense transformation layer to allow the low-dimensional features to flow to the high-dimensional features. (4) estimating the residual map using the residual map generator:

[0011] wherein, represents a combination operation of convolution-batch normalization, which inputs the high-dimensional features , estimates the residual map ; (5) subtracting the estimated residual map from the low-quality image to obtain a high-quality image after clarification: .

[0012] Further, in step S2, the step of adaptively fusing multi-scale features comprises: On the basis of feature extraction, the adaptive feature fusion search method is used to fuse the two-dimensional features of infrared features and visible light features, and the three-dimensional features of millimeter wave radar point cloud features and laser radar point cloud features. For each layer of the three non-parameter fusion schemes , and , specifically:

[0013]

[0014]

[0015] wherein, is a certain feature in the feature library, such as infrared features, millimeter wave radar point cloud features, laser radar point cloud features, and visible light features, is a Max_Pooling maximum pooling operation, is a sigmoid function, and the fusion between features is only performed under the condition that the features are of the same scale or the network is of the same layer. An adaptive feature fusion search framework is constructed, and the core idea is to model the fusion process of features as the weighted sum of all candidate fusion operations, which is represented as:

[0016] wherein, represent the weighted fusion output from the features .and . The set is for all the optional fusion operations, including , and three parameter-free fusion schemes in this case; for the persistent search space, this framework assigns a learnable weight to each operation and normalizes it through function. The weighted fusion output is calculated as: ; wherein, and both refer to the fusion operation in the set ; and are the learnable weight parameters corresponding to the operation and .

[0017] Further, in the step S2, the step of aligning the heterogeneous features is specifically as follows: According to the camera imaging model, the three-dimensional space point in the point cloud data is projected to the image plane to obtain the corresponding two-dimensional projection coordinate point ; For each projection coordinate point , the feature vector corresponding to the position is obtained by bilinear interpolation method from the neighborhood image features of the image features at the position; The feature vector at the position is spliced with the feature vector of the three-dimensional space point to obtain the fusion feature vector of the three-dimensional space point ; All three-dimensional space points in the point cloud data are traversed to finally obtain the fusion features of the image and the point cloud .

[0018] Further, in the step S3, the cross-attention-based multi-scale enhanced learning fusion network is constructed, which takes the cross-attention mechanism as the core, aims to fuse the correlation between multi-dimensional scale features and adaptively make up for the limitations between different scale features, so as to realize the enhancement of weak features; in the network structure, the multi-head mechanism based on multi-head is used to calculate to improve the efficiency; The specific fusion step includes: Two input sequences are provided: a three-dimensional fusion feature sequence S1 and a two-dimensional fusion feature sequence S2; First, Key values and Value values of S1 and Query values of S2 are calculated respectively, then an attention matrix is calculated according to the Key values and the Query values; Finally, the Value values are multiplied by the attention matrix to obtain a fused output sequence, the length of which remains consistent with S2, and the calculation formula is as follows:

[0019] wherein , , are weight matrices for calculating Query, Key and Value values respectively, and are realized through a multi-layer perception machine (MLP).

[0020] Further, a single-stage YOLO target detection algorithm is selected for target recognition in the step S4, and a YOLO model used first performs feature extraction through a multi-layer convolutional network, and then completes regression prediction through a fully connected layer. Specifically, the convolutional network divides an input picture into grid points, each of which is responsible for predicting candidate boxes, and outputs probability values of object categories Each candidate box prediction value includes five parts, wherein, the coordinates represent the offset of the center of an object from the top-left corner of the grid point to which the object belongs, and predicts the width and height proportion of the object relative to the entire picture, and the probability value represents the probability that there is an object in the candidate box. All prediction values are encoded into a dimensional tensor.

[0021] The prediction probability of each candidate box for an object is represented by the following formula: ; The loss function used in training is shown in the following formula:

[0022]

[0023]

[0024]

[0025] .

[0026] Further, in step S4, the YOLO algorithm adopts CSPDarknet53 as the backbone feature extraction network; Dropblock is used for random discarding of local regions; in order to improve the small target recognition ability, Mosaic algorithm is used for data enhancement, and path aggregation network structure PAFPN is used to construct a feature pyramid; at the same time, CIOU loss is introduced as the boundary box regression loss, so as to realize more accurate positioning and improve the accuracy and reliability of the algorithm in complex environment.

[0027] Compared with the prior art, the present disclosure has the following beneficial effects: ①An enhancement method for weak target features is proposed, which can realize accurate identification of personnel, vehicles and other targets in complex environments such as smoke, dust, light rain and night, and improve the perception robustness of the system in harsh conditions; ②A module structure with residual aggregation function is designed, which effectively extracts and aggregates multi-level high-dimensional features by stacking multiple residual aggregation modules, and enhances the richness and discriminability of feature expression; ③An adaptive feature fusion search framework is constructed, which can automatically select the optimal fusion strategy according to the input features, realize efficient fusion of multi-source heterogeneous features, and improve the feature utilization efficiency and fusion quality; ④A multi-scale enhanced learning fusion network is proposed, which effectively deals with the weak feature problem of ground features caused by target display degradation, blur, distortion and deformation in strong adversarial environment, realizes the enhancement and fusion of weak features, and improves the distinguishability of targets in complex scenes. BRIEF DESCRIPTION OF DRAWINGS

[0028] The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description of exemplary embodiments of the present disclosure taken in conjunction with the accompanying drawings. In the drawings, like reference numerals are generally used to refer to like or similar components throughout the various figures.

[0029] Figure 1 is a schematic block diagram of an exemplary PointNet++ structure; Figure 2 is a schematic block diagram of a Vgg16 network structure; Figure 3 is a flowchart of image deblurring enhancement based on a residual aggregation network; Figure 4 is a schematic diagram of the structure of DarkNet53 network; Figure 5 is a multi-sensor adaptive feature fusion structure diagram; Figure 6 is a schematic diagram of heterogeneous feature alignment structure; Figure 7A flowchart for weak feature enhancement fusion based on a multi-scale enhanced learning fusion network; Figure 8 A YOLO model structure schematic diagram; Figure 9 A Mish activation function schematic diagram; Figure 10 A CSP structure schematic diagram; Figure 11 A flowchart of the method described in the present disclosure. DETAILED DESCRIPTION

[0030] The preferred embodiments of the present disclosure will be described in detail in conjunction with the accompanying drawings. It should be understood that, although specific embodiments of the present disclosure are shown in the drawings, this does not constitute a limitation on the scope of protection of the present disclosure. The present disclosure can also be implemented in other alternative ways not detailed herein. The purpose of providing these embodiments is to make the technical solutions of the present disclosure clearer and more complete, and to help those skilled in the art to fully understand the scope covered by the present disclosure.

[0031] The present disclosure provides a target recognition method based on weak feature enhancement in a complex environment, which can accurately recognize personnel, vehicles and other targets in complex environments such as smoke, dust, light rain, night, etc. The method realizes the recognition of targets in complex environments by fusing multi-sensor information such as laser radar, millimeter wave radar, visible light camera, infrared camera, etc.

[0032] The method first uses multi-sensors such as laser radar, millimeter wave radar, visible light camera, infrared camera, etc. to perform multi-scale feature extraction and deblurring feature enhancement; on this basis, multi-scale feature adaptive fusion and heterogeneous scale feature alignment processing are performed; then, a multi-scale enhanced learning fusion network based on cross attention is used to enhance and fuse weak features in multiple dimensions; finally, based on the deep learning target detection and recognition algorithm, the fused and enhanced features are used to accurately recognize the target.

[0033] In an exemplary embodiment, a target recognition method based on weak feature enhancement in a complex environment mainly includes the following steps: 1. Step 1: Multi-sensor multi-scale feature extraction and deblurring feature enhancement On the basis of joint calibration of multi-sensors, different network structures are used for feature extraction of laser radar, millimeter wave radar, visible light camera, infrared camera, etc.

[0034] (1) For the laser radar and millimeter wave radar data, PointNet++ is used for feature extraction. PointNet++ can learn local features through gradually increasing context scales, and adaptively fuse multi-scale point cloud features for the problem of inconsistent point cloud density collected at different positions. The structure of PointNet++ is shown in Figure 1

[0035] (2) For infrared camera information, a convolutional neural network is used for feature extraction, which has two advantages: first, the weight and bias parameters used by the convolutional layer are less; second, it has translational invariance, that is, even if the target in the infrared image is translated, CNN can still accurately identify the target, and the output result is consistent with that before translation.

[0036] In this embodiment, VGGNet (Vgg16) is used as the feature extraction network, which deeply explores the relationship between the depth of the convolutional neural network and its performance. By repeatedly stacking small 3x3 convolution kernels and 2x2 max pooling layers, a 16-layer deep convolutional neural network is constructed. Its advantages include: first, the structure is simple, and all use the same size of convolution kernel and pooling kernel; second, a large number of 3x3 convolution in series are used, which not only has fewer parameters, but also can introduce more nonlinear transformation. The structure of VGGNet is shown in Figure 2

[0037] (3) For visible light image information, considering the problems of image blur and insufficient brightness caused by bad environment, this embodiment designs a deblurring image enhancement module. The module extracts detailed features by constructing a residual aggregation module, and generates high-dimensional features by using dense connection and fusing low-dimensional features. The structure is shown in Figure 3 (1) wherein, denotes the input low-quality image, denotes the clean background image, denotes the estimated residual image. Since image sharpening is an ill-posed problem, an end-to-end convolutional neural network method is constructed to estimate the residual image. The network structure of the residual aggregation network mainly includes four parts: low-quality picture decomposition, low-dimensional feature extraction, residual aggregation module and residual image generation. Let denote the low-quality picture, the low-frequency feature and the high-frequency feature, respectively.

[0038] Dust, rain, snow, fog, haze, noise and other high-frequency information in the image. First, the median filter is used for filtering operation on the low-quality picture, and the low-frequency information ​​​Then, the low-quality picture Subtract the low-frequency information The high-frequency information can be obtained .

[0039] (2) Subsequently, low-dimensional features are extracted from the high-frequency information using a low-dimensional feature extractor: (3) wherein, represents a combined operation composed of convolution, batch normalization, and a relu activation function, which inputs the high-frequency information and outputs a feature map .

[0040] Further, in order to extract more rich feature information, a residual aggregation module is constructed in the embodiment, and by stacking multiple residual aggregation modules, rich high-dimensional features can be extracted. The feature maps extracted by each residual aggregation module are transmitted to the last residual transformation module through dense connection, and more efficient high-dimensional features are extracted by means of feature compression and distillation: (4) wherein, represents a process of extracting features using a residual aggregation module, represents the number of stacked residual aggregation modules. By stacking multiple layers, local hierarchical features are fully extracted and utilized, and high-dimensional features are output .

[0041] In the embodiment, each residual aggregation module is composed of three parts: a residual feature extractor, a dense transformation layer, and local feature fusion. Specifically, the residual feature extractor extracts feature maps of different levels using a residual module; the dense transformation layer combines the feature maps extracted by the residual feature extractor, and adopts 1*1 convolution to realize feature compression and distillation; the local feature fusion connects the input of the current module to the input of the dense transformation layer to facilitate the flow of low-dimensional features to high-dimensional features. Subsequently, a residual map is estimated using a residual map generator: (5) wherein, represents a combined operation composed of convolution and batch normalization, which inputs the high-dimensional feature and estimates a residual map . Finally, the estimated residual map is subtracted from the low-quality picture to obtain a high-quality image after clarification: (6) Finally, the generated input DarNet53 network will be generated, and the network structure is as shown in Figure 4 .

[0042] 2、Step 2: Multi-scale feature adaptive fusion and heterogeneous scale feature alignment 1) Multi-scale feature adaptive fusion On the basis of feature extraction, the adaptive feature fusion search method is used to fuse the two-dimensional features of infrared features and visible light features, and the three-dimensional features of millimeter wave radar point cloud features and laser radar point cloud features. The fusion example is as follows. For each layer, three non-parameter fusion schemes 、 and are as follows: (7) (8) (9) Among them is a certain feature in the feature library of infrared features, millimeter wave radar point cloud features, laser radar point cloud features, and visible light features, is a Max_Pooling maximum pooling operation, is a sigmoid function. The fusion between features is only performed under the condition that the features are of the same scale or the network is of the same layer. In order to enable the feature fusion process to adopt the optimal fusion scheme, an adaptive feature fusion search framework is constructed. For the feature fusion process of the feature , the following formula is used: (10) Among them represents the weighted fusion output from the features and , and the set is all the selectable fusion operations. In this case, three non-parameter fusion schemes , and are included; for continuous search space, this framework assigns a learnable weight to each operation , and normalizes it through the function. The weighted fusion output can be calculated as: ; (11) Among them, and both refer to the fusion operation in the set ; and are the operations and The corresponding learnable weight parameters, whose fusion process is as shown in Figure 5 .

[0043] 2) Isometric scale feature alignment and fusion Multi-sensor feature alignment mainly includes 2D-3D scale correspondence, bilinear pooling interpolation and feature fusion steps. The main purpose of 2D-3D correspondence is to establish the geometric correspondence between image features and point cloud features through the camera projection imaging model; bilinear pooling interpolation not only extracts the local neighborhood information on the image, improves the problem caused by the sparsity of the point cloud, but also effectively alleviates the problem of low matching degree between discrete pixel points on the feature map and continuous projection points of the point cloud.

[0044] In the multi-sensor feature alignment process, the point cloud is first projected onto the image plane, and the point-to-pixel (P2P) mapping is generated (based on the multi-sensor joint calibration relationship). Then, the two-dimensional feature mapping is converted into a point-by-point two-dimensional feature according to the P2P mapping. The specific operation steps are as follows: 1. According to the camera imaging model, calculate the projection coordinate point of the three-dimensional space point in the point cloud data ; 2. Perform bilinear pooling interpolation on the neighborhood image features of the projection coordinate point to obtain the feature vector corresponding to the position ; 3. Connect the feature vector of the position and the feature vector of the three-dimensional space point to obtain the fusion feature vector of the three-dimensional space point ; 4. Repeat the above steps for all three-dimensional space points in the point cloud data to finally obtain the fusion feature of the image and the point cloud . Its structure is shown in .

[0045] Its structure is shown in Figure 6 .

[0046] 3. Step 3: Weak feature enhancement fusion based on multi-scale enhanced learning fusion network On the basis of multi-scale feature fusion, in view of the problem of weakening of ground feature caused by display decline, blur, distortion and deformation caused by strong confrontation in harsh environment, this embodiment introduces a multi-scale enhanced learning fusion network to further strengthen and fuse the weak features.

[0047] ​​The network adopts a multi-scale enhanced learning architecture based on cross-attention, which can learn the correlation between multi-dimensional scale features with a cross-attention mechanism and adaptively make up for the limitations between different scale features, thereby realizing the enhanced fusion of weak features. The structure is as shown in Figure 7 .

[0048] In terms of fusion mode, the network is realized based on a multi-head attention mechanism, which has a high parallel computing capability. Compared with a recurrent neural network, this method has stronger ability to capture the long-term dependence between the two-dimensional and three-dimensional fusion features generated in the previous stage. Unlike the way of using a fixed window as a receptive field in a convolutional neural network, the observation range of this method is global, which can more effectively explore the internal relationship between the two-dimensional fusion features and the three-dimensional fusion features (which have been aligned to the shape of two-dimensional features). The specific calculation process is as follows: 1. Input two sequences S1 (three-dimensional fusion features) and S2 (two-dimensional fusion features); 2. Calculate the Key value and Value value of S1; 3. Calculate the Query value of S2; 4. Calculate the attention matrix according to the Key value and Query value; 5. Apply the Value value to the attention matrix; 6. Output the fused sequence result, which maintains the same length as S2.

[0049] The calculation formula is as follows: (12) where , , are the equations for calculating Query, Key, and Value values, respectively, which are realized by multi-layer perceptron (MLP) in this embodiment.

[0050] 4. Step 4: Target detection and recognition based on enhanced feature deep learning The features generated by enhanced fusion are input into the target detector. In this embodiment, a deep learning-based target detection algorithm is used, and the detection accuracy, model size, and real-time requirements of the algorithm are considered comprehensively, and the single-stage YOLO algorithm is selected. The model structure diagram of YOLO is as shown in Figure 8 .

[0051] The YOLO algorithm first divides the input image into grid units by a convolutional network, and each grid is responsible for predicting candidate boxes and their corresponding probability values of object categories. The prediction value of each candidate box is mainly composed of Five parts. Among them, represents the offset of the target center relative to the upper left corner of the current grid, represents the width and height ratio of the predicted object relative to the entire picture, represents the probability value that there is a target in this candidate box. All prediction results are finally encoded into a tensor. Finally, the prediction probability of each candidate box predicting the object can be represented by the following formula: (13) The YOLO algorithm first extracts image features through multiple convolution layers, and then performs regression prediction through two fully connected layers. The loss function used in its training process is composed of multiple parts, as follows: (14) (15) (16) (17) (18) In this embodiment, the YOLO algorithm uses CSPDarknet53 as the feature extraction network, which has a larger receptive field. CSPDarknet53 is an improved Backbone structure based on the YOLOv3 backbone network Darknet53, which contains 5 CSP modules. The structural diagram of CSP is as Figure 10 shown, which is composed of three CBM modules and a number of Res unit residual components. Among them, the CBM module replaces the Leaky ReLU activation function in the original YOLOv3 CML module with the Mish activation function, enhancing the non-linear expression ability of the network. The expression and visualization of the Mish activation function are as Figure 9 shown: 。

[0052] The size of the convolution kernel in front of each CSP module is 3x3, and the step is 2, which is used to realize the down-sampling of the feature map.

[0053] Furthermore, this embodiment introduces the Dropblock module into YOLO, which functions similarly to the common Dropout, both serving as regularization techniques to alleviate overfitting. Dropout randomly discards some information from convolutional layers, but convolutional layers are not sensitive to this random discarding because they typically employ a three-layer concatenated structure of "convolution-activation-pooling." The pooling layer itself acts on adjacent units, so even with random discarding, the convolutional layer can still learn similar information from neighboring activation units. Therefore, Dropout has limited effectiveness in convolutional layers, while Dropblock achieves regularization by discarding entire local regions. While the Cutout method zeroes out a portion of the input image, Dropblock applies the Cutout concept to each feature map layer and uses a dynamic discard ratio during training: initially using a small ratio, increasing linearly with training progress, ultimately achieving better results than Cutout.

[0054] For small target detection, this embodiment employs the Mosaic algorithm for data augmentation and PAFPN to construct a feature pyramid. The CIOU loss function is also introduced to achieve more accurate bounding box localization, thereby improving the accuracy and robustness of the algorithm in complex environments. Ultimately, it achieves accurate identification of targets such as people and vehicles in complex environments such as smoke, dust storms, light rain, and nighttime.

[0055] It should be noted that the above description is only a specific embodiment of the present invention. For those skilled in the art, based on the application methods and principles disclosed in the present invention, it is easy to make various types of improvements or modifications, and not limited to the methods described in the above specific embodiments of the present invention. Therefore, the methods described above are only preferred and not restrictive.

Claims

1. A target recognition method based on weak feature enhancement in complex environments, characterized in that, Includes the following steps: S1, based on the joint calibration of multiple sensors, uses different network structures to perform multi-scale feature extraction and defuzzification feature enhancement on the multi-sensor data respectively; S2, adaptively fuses the extracted multi-scale features and aligns them with heterogeneous scale features; S3, based on multi-scale feature fusion, adopts a multi-scale reinforcement learning fusion network based on cross-attention to enhance and fuse weak features at multiple scales; S4. Based on the enhanced fusion features, a deep learning-based target detection and recognition algorithm is used to achieve target recognition.

2. The method according to claim 1, characterized in that, The multiple sensors include various types of LiDAR, millimeter-wave radar, visible light cameras, and infrared cameras.

3. The method according to claim 1, characterized in that, In step S1, for the visible light image, an end-to-end residual aggregation network based on a convolutional neural network is constructed for image deblurring and enhancement processing. The network structure of the residual aggregation network includes four parts: low-quality image decomposition, low-dimensional feature extraction, residual aggregation module, and residual map generation. The specific processing steps include: set up These represent low-quality images, low-frequency features, and high-frequency features, respectively. The high-frequency features include one or more of the following: dust, rain, snow, fog, haze, and noise in the image. (1) First, median filtering is used to filter low-quality images to separate low-frequency information from them. Then, remove the low-quality images. Subtract low-frequency information Obtain high-frequency information : (2) Subsequently, a low-dimensional feature extractor is used to extract low-dimensional features from the high-frequency information: in, This represents a combined operation of convolution, batch normalization, and ReLU activation function, which takes high-frequency information as input. Output feature map ; (3) High-dimensional features are extracted by stacking multiple residual aggregation modules. The feature maps extracted by the residual aggregation modules are passed to the final residual transformation module through dense connections. More efficient high-dimensional features are extracted through feature compression and distillation. in, This indicates that the residual aggregation module is used to extract features. Indicates stacking Block residual aggregation module, This represents the high-dimensional feature output after stacking multiple residual aggregation modules, extracting and utilizing local hierarchical features; Each residual aggregation module includes: a residual feature extractor, a dense transform layer, and local feature fusion. The residual feature extractor extracts different feature maps using residual modules. The dense transform layer combines the feature maps extracted by the residual feature extractor and compresses and distills the features using a 1*1 convolution operation. The local feature fusion connects the input of each residual aggregation module to the input of the dense transform layer to allow the network's low-dimensional features to flow to high-dimensional features. (4) Use the residual plot generator to estimate the residual plot: in, This represents a combined operation of convolution and batch normalization, which takes high-dimensional features as input. Estimate the residual plot ; (5) From low-quality images Subtracting the estimated residual plot To obtain a high-quality image after sharpening: 。 4. The method according to claim 1, characterized in that, In step S2, the specific method for multi-scale feature adaptive fusion includes: Based on feature extraction, an adaptive feature fusion search method is adopted to fuse infrared and visible light two-dimensional features, and millimeter-wave radar point cloud features and lidar point cloud three-dimensional features. Three parameter-free fusion schemes are applied to each layer. , and Specifically: in, This refers to a specific feature from a specific layer of infrared features, millimeter-wave radar point cloud features, lidar point cloud features, and visible light features in the feature library. For Max_Pooling, max pooling operation, The sigmoid function is used, and feature fusion only occurs when features are of the same scale or in the same layer of the network. The core idea of ​​constructing an adaptive feature fusion search framework is to integrate features... The fusion process can be modeled as a weighted sum of all candidate fusion operations, expressed as: in, Representative from characteristics and Weighted fusion output; gather For all optional fusion operations, including , and Three parameter-free fusion schemes; to provide a continuous search space, a learnable weight is assigned to each operation. and through The function is normalized and then weighted and fused for output. The calculation method is as follows: ； in, and Both refer to sets The fusion operation in; and This is an operation. and The corresponding learnable weight parameters.

5. The method according to claim 1, characterized in that, In step S2, the specific method for heterogeneous feature alignment is as follows: Based on the camera imaging model, the point cloud data Three-dimensional space points Projecting onto the image plane yields the corresponding two-dimensional projection coordinates. ; For each projected coordinate point By using the bilinear interpolation method, the feature vector corresponding to the location is obtained by sampling from the neighboring image features of the image feature. ; Will Location feature vector With three-dimensional space points eigenvectors To stitch together the data and obtain three-dimensional spatial points fused feature vector ; Traversing point cloud data All three-dimensional spatial points are used to ultimately obtain the fusion features of the image and point cloud. .

6. The method according to claim 1, characterized in that, In step S3, a multi-scale reinforcement learning fusion network based on cross-attention is constructed. Its construction and workflow are as follows: This network uses a cross-attention mechanism as its core, aiming to integrate the correlation between features at multiple scales and adaptively compensate for the limitations of features at different scales, thereby enhancing weak features. In terms of network structure, a multi-head computation mechanism is adopted to improve efficiency; The specific integration steps include: There are two input sequences: a three-dimensional fusion feature sequence S1 and a two-dimensional fusion feature sequence S2; First, calculate the Key and Value values ​​of S1, and the Query value of S2 respectively. Then, calculate the attention matrix based on the Key and Query values. Finally, the Value is multiplied by the attention matrix to obtain the fused output sequence, which has the same length as S2. The calculation formula is as follows: in , , The weight matrices for calculating the Query, Key, and Value values ​​are implemented using a Multilayer Perceptron (MLP).

7. The method according to any one of claims 1-6, characterized in that, In step S4, a single-stage YOLO target detection algorithm is used for target recognition. The YOLO model first extracts features through a multi-layer convolutional network, and then completes regression prediction through a fully connected layer. Specifically, the convolutional network divides the input image into... There are 1 grid, and each grid is responsible for prediction. 1 candidate box, and output Probability value of each object category Each candidate box prediction includes Five parts, among which, The coordinates represent the offset of the object's center from the top-left corner of its corresponding grid point, while This predicts the width and height ratio of the object relative to the entire image. This represents the probability that an object exists within the candidate bounding box; all predicted values ​​are encoded into a single value. Tensors of dimension; The predicted probability of each candidate box for a certain object is represented by the following formula: ； The loss function used during training consists of the following parts: 。 8. The method according to claim 7, characterized in that, In step S4, the YOLO algorithm uses CSPDarknet53 as the backbone feature extraction network; to enhance the model's generalization ability, Dropblock is used to randomly discard local regions; to improve the recognition ability of small targets, the Mosaic algorithm is used for data augmentation, and the path aggregation network structure PAFPN is used to construct a feature pyramid; at the same time, the CIOU loss function is introduced as the bounding box regression loss to achieve more accurate localization and improve the accuracy and reliability of target detection in complex environments.

Images