Marine target detection method and system based on progressive light thunder information deep fusion

Abstract

The invention discloses a marine target detection method and system based on progressive light thunder information deep fusion, and the method is realized based on an X-band radar and a visible light camera, and the method comprises the steps: receiving a radar echo signal and a visible light visual image of a to-be-detected marine target, and forming a plurality of paired data; preprocessing the radar echo signal to obtain radar echo data after matched filtering, and obtaining radar point cloud data through a constant false alarm algorithm; carrying out nonlinear modal calibration on the preprocessed radar echo signal and radar point cloud data to obtain a radar pseudo image, and carrying out weighted addition on the radar pseudo image and a visible light visual image to carry out first-stage fusion; inputting the paired radar pseudo image, visible light visual image and weighted fusion image into a pre-trained target detection model to obtain the category and position information of the marine target contained in the to-be-detected area; the target detection model adopts a two-stage fusion strategy and combines a gating attention mechanism to realize deep fusion of light thunder information and improve the detection precision.

Description

Technical Field

[0001] The present invention relates to the fields of radar and visible light signal processing, multi-source signal fusion and deep learning technology, and in particular to a method and system for detecting marine targets based on progressive deep fusion of light-radar information. Background Art

[0002] Maritime target detection is crucial in maritime security monitoring, territorial sovereignty maintenance and smuggling detection. At present, advanced radar target detection methods and visual target detection methods have been developed to a very mature level. However, the single sensor method has limitations due to the single information obtained. For example, the visual system is greatly affected by ambient light, especially at night or in bad weather. Although the radar system can provide target location information, it is difficult to accurately identify the target type and shape at long distances, and it is easily interfered by sea clutter, resulting in false alarms. In order to improve the efficiency of maritime surveillance, combining the all-weather detection capability of the radar system with the high-resolution imaging capability of the visual system is an effective solution.

[0003] In recent years, radar and vision fusion technology has gradually been applied to the field of surface target detection. Existing models have improved detection performance by fusing radar and visual features. However, most fusion methods are based on millimeter-wave or lidar, and the point cloud of such radars is highly consistent with the target area in the image. Due to differences in the standards of X-band radars, the scale of target echoes is much larger than the scale of target images, which increases the complexity of fusion. Therefore, existing solutions based on millimeter-wave or lidar are difficult to meet the needs of X-band radar and visual image fusion. In summary, we face the following two key conditions when applying deep fusion technology of optical radar information to maritime target detection.

[0004] First, due to the different implementation principles of radar and optoelectronic signals, the characteristics of the acquired signals are quite different, and direct fusion is extremely difficult. In addition, in radar target detection, the radar beam diffusion causes the horizontal range of a single beam detection to be too large, the captured information is sparse, and the mapping scale in the image coordinate system is much larger than the scale of the target in the optoelectronic image. The resulting scale mismatch seriously affects the detection accuracy. Second, in complex sea conditions, non-target areas will introduce sea clutter due to phenomena such as sea spikes, driving the network to identify pseudo targets, or because the signals of small targets such as rafts and cursors are small, they are annihilated by clutter signals, resulting in the loss of radar target signals. Both of these situations will lead to inconsistency in bimodal feature information during alignment, resulting in feature mismatch problems during network learning, affecting the fusion effect. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the prior art and propose a multi-source target detection method and system based on progressive radar vision fusion.

[0006] In order to achieve the above object, the present invention proposes a method for detecting marine targets based on progressive deep fusion of optical mine information, which is implemented based on X-band radar and visible light camera. The method comprises:

[0007] Step 1) receiving radar echo signals and visible light visual images of targets to be detected at sea, forming multiple pairs of data;

[0008] Step 2) preprocessing the radar echo signal to obtain radar echo data after matched filtering, and obtaining radar point cloud data through a constant false alarm algorithm;

[0009] Step 3) The preprocessed radar echo signal and radar point cloud data are subjected to nonlinear modal calibration to obtain a radar pseudo image, which is then weightedly added with the visible light visual image for the first stage of fusion;

[0010] Step 4) inputting the paired radar pseudo images, visible light visual images and weighted fusion images into a pre-established and trained target detection model to obtain the category and location information of the marine targets contained in the area to be detected, thereby realizing target detection;

[0011] The target detection model adopts a two-stage fusion strategy combined with a gated attention mechanism to achieve deep fusion of light-ray information and improve detection accuracy.

[0012] Preferably, the detection horizontal range of the X-band radar is α degrees, and the visible light camera captures images of the sea at n angles with a step of β degrees, satisfying α>nβ;

[0013] In each pair of data, the angle of the radar echo signal is consistent with the angle of the horizontal field of view of the visible light camera;

[0014] The preprocessing of step 2) includes: performing matched filtering and normalization processing on the radar echo signal to obtain matched filtered radar echo data with regular amplitude, thereby realizing deep fusion of light-thunder information.

[0015] Preferably, obtaining radar point cloud data by using a constant false alarm algorithm includes:

[0016] The radar echo data after matched filtering is used to obtain radar point cloud data through the constant false alarm algorithm, and then the sea clutter suppression is achieved through the isolated point removal algorithm to obtain radar point cloud data.

[0017] Preferably, the step 3) comprises:

[0018] The preprocessed radar echo signal and radar point cloud data are horizontally mapped and distance mapped respectively to obtain the radar point cloud pseudo image I rp With radar echo pseudo image I re ;

[0019] The radar point cloud pseudo image I rp Compared with the original visible light image I v Weighted pixel-by-pixel addition to obtain shallow signal-level fusion image I rv , while I rp With radar echo pseudo image I re Superposition to obtain radar signal pseudo image I r , and then weighted addition with the visible light visual image to achieve the first stage of fusion.

[0020] Preferably, the target detection model includes a shallow fusion module, a detail basic feature decoupling module, a gated attention fusion module, a bottleneck network and a detection head connected in sequence, wherein:

[0021] The shallow fusion module uses point convolution for shallow fusion, weights the channel information to select relevant features, and obtains the shallow fusion feature F S , achieving the second stage of integration;

[0022] The detail basic feature decoupling module is used to decouple the shallow fusion feature F S Decouple local detail features and global basic features;

[0023] The gated attention fusion module is used to select detail features through gated convolution and cross attention design to achieve third-stage fusion;

[0024] The bottleneck network and the detection head are used to obtain multi-scale fusion features and realize end-to-end detection of target positions and categories.

[0025] Preferably, a visual backbone network is connected before the shallow fusion module, and the input of the visual backbone network is the shallow signal level fusion image I rv Compared with the original visible light image I v , extract the corresponding feature F rv With F v .

[0026] Preferably, the detail-basic feature decoupling module includes two sets of parallel detail feature extractors and basic feature extractors; the detail feature extractor uses stacked asymmetric convolutions to enhance sensitivity to directionality and local features, and the basic feature extractor is based on a Transformer network to learn global features;

[0027] Among them, the input of the first set of parallel detail feature extractors and basic feature extractors is the shallow fusion feature F s , the output is the shallow fusion detail feature F SD With the basic feature F SBThe input of the second set of parallel detail feature extractors and basic feature extractors is the radar feature F r , the output is the radar detail feature F RD With the basic feature F RB ;

[0028] The radar signature F r is the radar signal pseudo image I r The radar backbone network is obtained by inputting it, where the radar backbone network is designed as a visual backbone network with half the channels.

[0029] Preferably, the processing process of the gated attention fusion module includes:

[0030] First, point-wise convolution is used to fuse shallow fusion detail features F along the channel dimension SD With radar detail feature F RD , through gated convolution, use depth-wise separable convolution and nonlinear activation function to suppress irrelevant details and false alarm features to further refine the fusion features; then use the detail features selected by gated convolution to calculate the query vector Q of the cross attention mechanism, and at the same time calculate the key vector K and value vector V through the shallow detail features; get the fusion detail feature F D , achieving the third stage of integration.

[0031] Preferably, the bottleneck network adopts a convolution stacking method, and the input is the fused detail feature F D With the basic feature F B The fusion feature F is obtained by point-by-point addition. fuse , the basic feature F B By shallow fusion of basic features F SB Radar basic characteristics F RB The output of the bottleneck network is obtained by point-by-point addition and fusion, and is a multi-scale fusion feature.

[0032] On the other hand, the present invention provides a marine target detection system based on progressive deep fusion of optical mine information, which is implemented based on X-band radar and visible light camera, and the system includes:

[0033] The acquisition module is used to receive radar echo signals and visible light visual images of targets to be detected at sea, and form multiple pairs of data;

[0034] The preprocessing module is used to preprocess the radar echo signal to obtain the radar echo data after matched filtering, and obtain the radar point cloud data through the constant false alarm algorithm;

[0035] A nonlinear modal calibration module is used to perform nonlinear modal calibration on the preprocessed radar echo signal and radar point cloud data to obtain a radar pseudo image, which is then weightedly added with the visible light visual image for the first stage of fusion; and

[0036] The target detection module is used to input the paired radar pseudo images, visible light visual images and weighted fusion images into the pre-established and trained target detection model to obtain the category and location information of the marine targets contained in the area to be detected, thereby realizing target detection;

[0037] The target detection model adopts a two-stage fusion strategy combined with a gated attention mechanism to achieve deep fusion of light-ray information and improve detection accuracy.

[0038] Compared with the prior art, the advantages of the present invention are:

[0039] 1. The present invention fully utilizes the information of the two modes through the deep fusion of radar echo and visible light image, and enhances the robustness and accuracy of target detection. Compared with the detection method of a single mode, it can better cope with the complex marine environment.

[0040] 2. The detail-based feature decoupling module provided by the present invention formulates fusion strategies for features with different characteristics, enhances the alignment accuracy of feature fusion, and effectively reduces the fusion collapse phenomenon caused by inconsistent modal features.

[0041] 3. The gated attention fusion module provided by the present invention can dynamically select effective features, filter clutter in radar signals, and significantly reduce the detection probability of false targets, thereby improving the accuracy and reliability of detection.

[0042] 4. When testing benchmark datasets, the present invention achieves performance that exceeds existing methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flow chart of the method for detecting marine targets based on progressive deep fusion of light mine information of the present invention;

[0044] Figure 2 It is a schematic diagram of the framework of the three-stage progressive deep fusion of light mine information of the present invention. DETAILED DESCRIPTION

[0045] The method of the present invention comprises:

[0046] Preprocessing the radar echo signal to be detected;

[0047] The pre-processed radar echo and radar point cloud are mapped to the visual coordinate domain through coordinate calibration;

[0048] Inputting the radar pseudo image and the visible light visual image mapped to the visual coordinate domain into a pre-established and trained marine target detection model to obtain the category and location information of the marine targets contained in the area to be detected;

[0049] Based on the proposed maritime target detection algorithm, an intelligent multi-source fusion maritime target detection system is built;

[0050] The marine target detection model is implemented by decoupling detail-based features and fusion of gated attention.

[0051] As an improvement of the above method, the pre-processed radar echo and radar point cloud are mapped to the visual coordinate domain through coordinate calibration; specifically including:

[0052] For horizontal mapping, the center orientation and horizontal field of view angle of the visual image are first recorded, and then the azimuth angle θ corresponding to the radar signal is evenly distributed to the horizontal pixel coordinate x of the image.

[0053] For distance mapping, first record the vertical pixel coordinates y of the offshore buoy in the image b and the actual distance d measured by the radar b , and then fit y through a cubic polynomial b to d b The fitting process maps the distance measured by the radar to the vertical pixel coordinate y of the image.

[0054] As an improvement of the above method, the marine target detection model includes a shallow fusion module, a feature decoupling module, and a gated attention fusion module connected in series; wherein,

[0055] The shallow fusion module is used to fuse radar point clouds and visual images at the signal level, and realizes effective fusion of shallow features through point-by-point convolution. This module makes full use of the key areas of radar point clouds in early calibration images, while retaining image details and reducing the computational complexity of the fusion process.

[0056] The feature decoupling module is used to extract the detail features and basic features of the bimodal information, and formulate fusion strategies for different characteristic features to enhance the alignment accuracy of deep feature fusion.

[0057] The gated attention fusion module is used to select input detail features, filter clutter signals in radar signals, reduce the detection probability of false targets, and improve detection accuracy.

[0058] As an improvement of the above method, the shallow fusion module includes two stages, specifically including:

[0059] In the first stage, the signal-level fusion takes as input the radar point cloud pseudo image and the visual image mapped to the visual image domain, and outputs a fused image of signal-level weighted superposition;

[0060] The second stage of shallow feature fusion takes as input the output of the first stage and the corresponding features of the original visual image extracted through the visual backbone, and then performs shallow feature fusion through channel splicing and point-by-point convolution.

[0061] As an improvement of the above method, the feature decoupling module includes a detail feature extractor and a basic feature extractor, which are used to decouple local detail features and global basic features, specifically including:

[0062] The detail feature extractor uses stacked asymmetric convolutions to enhance sensitivity to directional and local features. It effectively captures horizontal radar data and extracts details such as edges and textures from shallow fusion features and radar features, which are critical for accurate object localization and classification, especially when there are significant scale differences between modalities.

[0063] The basic feature extractor is based on the Transformer network and focuses on learning global features. Its self-attention mechanism captures dependencies, extracts overall contours and contextual information from images, and solves the problem of consistency in global features of cross-modal images.

[0064] As an improvement of the above method, the gated attention fusion module is designed based on gated convolution and cross attention to enhance the selection ability of the attention mechanism, specifically including: firstly, point-by-point convolution is used to fuse radar and shallow features along the channel dimension, and then the semantic information of shallow detail features is introduced into the radar detail features to facilitate the semantic alignment of radar detail features. Subsequently, the gated convolution uses depthwise separable convolution and nonlinear activation function to further refine the fused features by suppressing irrelevant details and false alarm features. Then, the detail features selected by the gated convolution are used to calculate the query vector Q of the cross attention mechanism, and the key vector K and value vector V are calculated through the shallow detail features. Finally, the formula is used Calculate the final fused detail features.

[0065] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0066] Example 1

[0067] like Figure 1 As shown, embodiment 1 of the present invention proposes a method for detecting marine targets based on progressive deep fusion of light mine information, the method comprising:

[0068] Preprocessing the radar echo signal to be detected;

[0069] The pre-processed radar echo and radar point cloud are mapped to the visual coordinate domain through coordinate calibration;

[0070] Inputting the radar pseudo image and the visible light visual image mapped to the visual coordinate domain into a pre-established and trained marine target detection model to obtain the category and location information of the marine targets contained in the area to be detected;

[0071] Based on the proposed maritime target detection algorithm, an intelligent multi-source fusion maritime target detection system is built;

[0072] The marine target detection model is implemented by decoupling detail-based features and fusion of gated attention.

[0073] The marine target detection method based on progressive deep fusion of light mine information proposed in the present invention completes data processing by point-by-point convolution, detail basic feature decoupling module, gated attention fusion module and multi-scale detection module. Figure 2 The following is a schematic diagram of a three-stage progressive deep fusion framework for optical mine information. The specific process of this method includes:

[0074] Step 1) Data collection: deploy X-band shore-based radar and visible light camera on the coast to shoot the sea. The radar detection horizontal range is α degrees. The photoelectric camera shooting strategy is to shoot sea images at n angles with a step interval of β degrees, where α>nβ. The radar sample in each paired data selects the radar echo data of the corresponding angle according to the direction of the corresponding visible light camera to the sea, and the range is consistent with the horizontal field of view of the visible light camera.

[0075] Step 2) Data processing: Matched filtering and normalization are performed on the radar samples in each pair of data to obtain matched filtered radar echo data with regular amplitude.

[0076] Step 3) Based on the matched filtered echo data obtained in step 2), radar point cloud data is obtained by using a constant false alarm algorithm, and then sea clutter suppression is achieved by using an isolated point removal algorithm to obtain relatively clean radar point cloud data.

[0077] Step 4) Perform nonlinear modal calibration on the matched filtered echo data and radar point cloud data obtained in step 2) and step 3). The calibration process is mainly divided into two steps: horizontal mapping and distance mapping. Finally, the calibrated radar point cloud pseudo image I is obtained. rp With radar echo pseudo image I re .

[0078] The step 4) specifically includes:

[0079] Step 4-1) Perform horizontal mapping. First, record the center direction of the visual image and the horizontal field of view (FOV).

[0080] Step 4-2) Evenly distribute the azimuth angle θ corresponding to the radar signal to the horizontal pixel coordinate x of the image. The mapping relationship can be expressed as:

[0081]

[0082] Where W is the image width, FOV h is the horizontal field of view of the camera, θ center is the azimuth of the image center.

[0083] Step 4-3) Perform distance mapping using the vertical pixel coordinate y of the offshore buoy in the image b and its actual distance d measured by radar b , fitted by a cubic polynomial:

[0084] y(d)=a3d 3 +z2d 2 +a1d+a0 (2)

[0085] Where d is the distance measured by the radar, and a3, a2, a1, and a0 are fitting parameters. The fitting process maps the distance measured by the radar to the vertical pixel coordinate y of the image.

[0086] Step 5) The radar point cloud pseudo image I obtained in step 4) is rp With the original visual image I v The first stage shallow signal level fusion image I is obtained by weighted pixel-by-pixel addition. rv , while I rp With radar echo pseudo image I re Superposition to obtain radar signal pseudo image I r .

[0087] Step 6) The shallow fusion image I obtained in step 5) is rv With the original visual image I v The input parameter-sharing visual backbone network obtains the features F rv With F v , and then perform point-by-point convolution on the F concatenated along the channel rv With F v Perform the second stage of shallow fusion to obtain the shallow fusion feature F s .

[0088] The step 6) specifically includes:

[0089] Step 6-1) The shallow fusion image I obtained in step 5) is rv With the original visual image I v As the input of the visual backbone network. rv Extract feature F rv , from Iv Extract feature F v :

[0090]

[0091] Step 6-2) Use point convolution for shallow fusion, effectively weight the channel information to select relevant features, and generate shallow fusion features F S :

[0092] F S =Conv 1×1 (concat(F rv ,F v )) (4)

[0093] Step 7) The radar signal pseudo image I obtained in step 5) is r Input the radar backbone network to obtain the radar feature F r , where the radar backbone network is designed as a visual backbone network with half the channels.

[0094] Step 8) Set the radar feature F r With shallow fusion feature F s The detail features of each modality are obtained through the detail-basic feature decoupling module, including the shallow fusion detail feature F SD With the basic feature F SB And radar detail feature F RD With the basic feature F RB .like Figure 2 As shown, the decoupling module proposed in the present invention is composed of a detail feature extractor and a basic feature decoupler.

[0095] The step 8) specifically includes:

[0096] Step 8-1) The radar pseudo image I obtained in step 5) is r Input the radar backbone network to obtain the radar feature F r :

[0097]

[0098] Step 8-2) Set the radar feature F r With shallow fusion feature F S The input is a detail feature extractor composed of asymmetric convolution, standard convolution, and SiLU activation function, which enhances the algorithm's sensitivity to feature directionality and local features. S and F r Extract details such as edges and textures:

[0099] F SD =ε detail (FS ), F RD =ε detail (F r ) (5)

[0100] Step 8-3) Set the radar feature F r With shallow fusion feature F S The basic feature extractors composed of layer normalization, attention, and multi-layer perceptrons are input to extract the overall contour and context information from the image features, which solves the problem of inconsistent global features between modalities to a certain extent and ensures F S and F r The structural alignment between them improves the accuracy of cross-modal target detection. The basic features extracted from each stream are:

[0101] F SB =ε base (F S ), F RB =ε base (F r ) (6)

[0102] like Figure 2 As shown, the fusion module proposed in the present invention is composed of a gated convolution module.

[0103] Step 9) First, the shallow fusion detail feature F obtained in step 8) SD With radar detail feature F RD Obtain fused detail features F through deep fusion with gated attention D Then the shallow fusion basic feature F SB Radar basic characteristics F RB Obtain the fusion basic feature F by point-by-point addition fusion B Finally, the fused detail features F D With the basic feature F B The final fusion feature F is obtained by adding point by point fuse .

[0104] The step 9) specifically includes:

[0105] Step 9-1) First, point-wise convolution is used to fuse radar and shallow features along the channel dimension, introducing F SD semantic information, which is convenient for F RD Semantic alignment of ,enhanced feature representation:

[0106] F′ D =Conv 1×1 (concat(F SD ,F RD )) (7)

[0107] Step 9-2) The pre-enhanced detail feature F′ D Input is a gated convolution based on a depth-wise separable convolution kernel with a nonlinear activation function. To suppress irrelevant details and false alarm features, further refine and enhance features. Then the detail features F′ selected by gated convolution D To calculate the query vector Q of the cross-attention mechanism:

[0108]

[0109] Step 9-3) Using shallow detail features F SD Used to calculate the key vector K and value vector V:

[0110] K=F SD W K , V=F SD W V (9)

[0111] Step 9-4) Use the gated cross attention mechanism to integrate the detail features into:

[0112]

[0113] in is the scaling factor of the key dimension and softmax(·) represents the normalization function.

[0114] Step 9-5) Extract the base feature F from the shallow feature SB The base feature F extracted from radar features RB The fusion is performed by point-by-point addition:

[0115]

[0116] in Represents point-by-point addition.

[0117] Step 9-6) The fused detail features F D and the basic feature F B Add point by point to get the output of fusion stage three:

[0118]

[0119] Step 10) The obtained fusion feature F fuse The bottleneck network with stacked convolutions is input to obtain multi-scale fusion features, and then the target position and category are output end-to-end through the detection head.

[0120] Step 11) Using the gradient descent method to update the parameters in the above model; iterating repeatedly until the optimal parameter combination is trained;

[0121] The step 11) specifically includes:

[0122] Step 11-1) Calculate the difference between the center coordinates and size (width and height) of the detection head output and the true bounding box through the mean square error (MSE):

[0123]

[0124] in, Indicates whether there is an object in the i-th grid cell and the j-th bounding box.

[0125] Step 11-2) Use mean square error as classification loss to measure the difference between the predicted category and the true category:

[0126]

[0127] Among them, p i (c) and are the probabilities of the predicted and true categories, respectively.

[0128] Step 11-3) Measure the confidence error of whether the predicted box contains the target through confidence loss:

[0129]

[0130] Among them, C i is the prediction confidence, is the true confidence.

[0131] Step 11-4) Calculate the total loss function, including the weighted sum of position loss, classification loss and confidence loss:

[0132] L total =λ loc L loc +λ cls L cls +λ conf L conf (16)

[0133] Step 11-5) The parameters of the detection model are updated by the gradient descent method until the loss is minimized and the optimal model parameters are obtained.

[0134] Step 12) Implement marine target detection based on multi-source information fusion based on the optimal parameter model.

[0135] Step 12) specifically includes:

[0136] Step 12-1) Perform matched filtering, CFAR and clutter suppression on the echo signal to be detected to obtain a matched filtering signal and a radar point cloud signal.

[0137] Step 12-2) mapping the matched filter signal and the radar point cloud signal to an image coordinate system to generate a radar pseudo image;

[0138] Step 12-3) Input the radar pseudo image and the visual image into the optimal parameter model, and obtain the prediction result through forward transmission of the model.

[0139] Example 2

[0140] Embodiment 2 of the present invention proposes an intelligent marine monitoring system based on a marine target detection method based on progressive deep fusion of light mine information, the system comprising:

[0141] The acquisition module is used to receive radar echo signals and visible light visual images of targets to be detected at sea, and form multiple pairs of data;

[0142] The preprocessing module is used to preprocess the radar echo signal to obtain the radar echo data after matched filtering, and obtain the radar point cloud data through the constant false alarm algorithm;

[0143] The nonlinear modal calibration module is used to perform nonlinear modal calibration on the preprocessed radar echo signal and radar point cloud data to obtain a radar pseudo image, which is then weightedly added with the visible light visual image for the first stage of fusion;

[0144] The target detection module is used to input the paired radar pseudo images, visible light visual images and weighted fusion images into the pre-established and trained target detection model to obtain the category and location information of the marine targets contained in the area to be detected, thereby realizing target detection;

[0145] The target detection model adopts a two-stage fusion strategy, combined with a gated attention mechanism, to achieve deep fusion of light mine information and improve detection accuracy. The specific processing process is as follows:

[0146] Step 1) The acquisition module scans the target sea area through radar and visible light camera to obtain radar echo signals and visible light image data in real time.

[0147] Step 2) These data enter the data processing module. The system preprocesses the radar echo signal, including point cloud extraction, sea clutter suppression and other operations, and converts the preprocessed radar data into a radar pseudo image in the visible light image coordinate system through a nonlinear modal calibration module.

[0148] Step 3) In the target detection module, the system will perform deep information fusion on the fused images, combining the complementary information of radar and vision to achieve accurate detection and tracking of marine targets. The detection results will be fed back to the user interface in real time, providing decision makers with target category, location and dynamic information, ensuring efficient target monitoring and management in complex marine environments.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention is described in detail with reference to the embodiments, it should be understood by those skilled in the art that any modification or equivalent replacement of the technical solutions of the present invention does not depart from the spirit and scope of the technical solutions of the present invention and should be included in the scope of the claims of the present invention.

Claims

1. A method for detecting marine targets based on progressive deep fusion of optical mine information, implemented based on X-band radar and visible light camera, comprising: Step 1) receiving radar echo signals and visible light visual images of targets to be detected at sea, forming multiple pairs of data; Step 2) preprocessing the radar echo signal to obtain radar echo data after matched filtering, and obtaining radar point cloud data through a constant false alarm algorithm; Step 3) The preprocessed radar echo signal and radar point cloud data are subjected to nonlinear modal calibration to obtain a radar pseudo image, which is then weightedly added with the visible light visual image for the first stage of fusion; Step 4) inputting the paired radar pseudo images, visible light visual images and weighted fusion images into a pre-established and trained target detection model to obtain the category and location information of the marine targets contained in the area to be detected, thereby realizing target detection; The target detection model adopts a two-stage fusion strategy combined with a gated attention mechanism to achieve deep fusion of light-ray information and improve detection accuracy.

2. The method for detecting marine targets based on progressive deep fusion of optical mine information according to claim 1 is characterized in that: The detection horizontal range of the X-band radar is α degrees, and the visible light camera takes images of the sea at n angles with a step of β degrees, satisfying α>nβ; In each pair of data, the angle of the radar echo signal is consistent with the angle of the horizontal field of view of the visible light camera; The preprocessing of step 2) includes: performing matched filtering and normalization processing on the radar echo signal to obtain matched filtered radar echo data with regular amplitude, thereby realizing deep fusion of light-thunder information.

3. The method for detecting marine targets based on progressive deep fusion of optical mine information according to claim 2 is characterized in that: The radar point cloud data is obtained by using a constant false alarm algorithm, including: The radar echo data after matched filtering is used to obtain radar point cloud data through the constant false alarm algorithm, and then the sea clutter suppression is achieved through the isolated point removal algorithm to obtain radar point cloud data.

4. The method for detecting marine targets based on progressive deep fusion of optical mine information according to claim 1 is characterized in that: The step 3) comprises: The preprocessed radar echo signal and radar point cloud data are horizontally mapped and distance mapped respectively to obtain the radar point cloud pseudo image I rp With radar echo pseudo image I re ; The radar point cloud pseudo image I rp Compared with the original visible light image I v Weighted pixel-by-pixel addition to obtain shallow signal-level fusion image I rv , while I rp With radar echo pseudo image I re Superposition to obtain radar signal pseudo image I r , and then weighted addition with the visible light visual image to achieve the first stage of fusion.

5. The method for detecting marine targets based on progressive deep fusion of optical mine information according to claim 4 is characterized in that: The target detection model includes a shallow fusion module, a detail basic feature decoupling module, a gated attention fusion module, a bottleneck network and a detection head connected in sequence, wherein: The shallow fusion module uses point convolution for shallow fusion, weights the channel information to select relevant features, and obtains the shallow fusion feature F S , achieving the second stage of integration; The detail basic feature decoupling module is used to decouple the shallow fusion feature F S Decouple local detail features and global basic features; The gated attention fusion module is used to select detail features through gated convolution and cross attention design to achieve third-stage fusion; The bottleneck network and the detection head are used to obtain multi-scale fusion features and realize end-to-end detection of target positions and categories.

6. The method for detecting marine targets based on progressive deep fusion of optical mine information according to claim 5 is characterized in that: A visual backbone network is connected before the shallow fusion module, and the input of the visual backbone network is the shallow signal level fusion image I rv Compared with the original visible light image I v , extract the corresponding feature F rv With F v .

7. The method for detecting marine targets based on progressive deep fusion of optical mine information according to claim 5 is characterized in that: The detail-basic feature decoupling module includes two sets of parallel detail feature extractors and basic feature extractors; The detail feature extractor uses stacked asymmetric convolutions to enhance sensitivity to directional and local features, and the basic feature extractor is based on the Transformer network to learn global features; Among them, the input of the first set of parallel detail feature extractors and basic feature extractors is the shallow fusion feature F S , the output is the shallow fusion detail feature F SD With the basic feature F SB The input of the second set of parallel detail feature extractors and basic feature extractors is the radar feature F r , the output is the radar detail feature F RD With the basic feature F RB ; The radar signature F r is the radar signal pseudo image I r The radar backbone network is obtained by inputting it, where the radar backbone network is designed as a visual backbone network with half the channels.

8. The method for detecting marine targets based on progressive deep fusion of optical mine information according to claim 5 is characterized in that: The processing process of the gated attention fusion module includes: First, point-wise convolution is used to fuse shallow fusion detail features F along the channel dimension SD With radar detail feature F RD , through gated convolution, use depth-wise separable convolution and nonlinear activation function to suppress irrelevant details and false alarm features to further refine the fusion features; then use the detail features selected by gated convolution to calculate the query vector Q of the cross-attention mechanism, and at the same time calculate the key vector K and value vector V through the shallow detail features; get the fusion detail feature F D , achieving the third stage of integration.

9. The method for detecting marine targets based on progressive deep fusion of optical mine information according to claim 5 is characterized in that: The bottleneck network adopts a convolution stacking method, and the input is the fused detail feature F D With the basic feature F B The fusion feature F is obtained by point-by-point addition. fuse , the basic feature F B By shallow fusion of basic features F SB Radar basic characteristics F RB The output of the bottleneck network is obtained by point-by-point addition and fusion, and is a multi-scale fusion feature.

10. A marine target detection system based on progressive deep fusion of optical mine information, implemented based on X-band radar and visible light camera, characterized in that: The system comprises: The acquisition module is used to receive radar echo signals and visible light visual images of targets to be detected at sea, and form multiple pairs of data; The preprocessing module is used to preprocess the radar echo signal to obtain the radar echo data after matched filtering, and obtain the radar point cloud data through the constant false alarm algorithm; A nonlinear modal calibration module is used to perform nonlinear modal calibration on the preprocessed radar echo signal and radar point cloud data to obtain a radar pseudo image, which is then weightedly added with the visible light visual image for the first stage of fusion; and The target detection module is used to input the paired radar pseudo images, visible light visual images and weighted fusion images into the pre-established and trained target detection model to obtain the category and location information of the marine targets contained in the area to be detected, thereby realizing target detection; The target detection model adopts a two-stage fusion strategy combined with a gated attention mechanism to achieve deep fusion of light-ray information and improve detection accuracy.

Images