Ship target detection method based on feature-guided adaptive decision

Abstract

The invention provides a ship target detection method based on feature-guided adaptive decision, and the method comprises the steps: carrying out the target detection of a registered optical image and a registered SAR image, and obtaining two sets of initial detection results containing a bounding box, confidence, and target region features; secondly, carrying out space matching on two groups of results based on intersection and comparison to form a detection pair; then, constructing a lightweight reliability evaluation network, taking the bimodal target region features in the matching pairs as input, and dynamically outputting normalized fusion weights of optical and SAR modals; and finally, performing weighted fusion on the confidence coefficient of the detection pair and the bounding box based on the weight to generate a final ship detection result, and optimizing the whole system through end-to-end joint training. According to the method, a more reliable sensor mode can be adaptively trusted according to the context features of the target instance, and the detection precision and robustness of the ship target under the complex sea condition are remarkably improved.

Description

Technical Field

[0001] This invention relates to a feature-guided adaptive decision-making method for ship target detection in optical-SAR imagery, belonging to the interdisciplinary technical field of remote sensing image processing, artificial intelligence and information fusion. It is particularly suitable for high-precision and robust automatic detection of ship targets on the sea surface using multi-source remote sensing data from optical and synthetic aperture radar (SAR). Background Technology

[0002] Ship target detection is a core task in maritime surveillance, maritime traffic management, protection of maritime rights, and military reconnaissance. Optical remote sensing imagery offers high spatial resolution and intuitive visual characteristics, but it cannot acquire effective data under conditions such as clouds, fog, rain, and night. SAR imagery, as an active microwave remote sensing method, possesses all-weather, all-day imaging capabilities, but its images suffer from speckle noise and are susceptible to ocean clutter interference in complex sea conditions, leading to a high false alarm rate. A single sensor is insufficient to meet the demands of all-weather, high-precision ship monitoring.

[0003] Multi-source fusion is an effective way to improve detection performance. Existing fusion methods mainly include pixel-level, feature-level, and decision-level fusion. Pixel-level fusion requires extremely high registration accuracy and struggles to handle the fundamental differences between optical and SAR imaging mechanisms; feature-level fusion, while capable of extracting deep semantics, is susceptible to inconsistencies in feature distribution between modalities; decision-level fusion typically employs fixed weights, voting, or DS evidence theory, making it impossible to dynamically evaluate the reliability of each modality based on specific target instances and scenarios. Especially for small-scale ship targets, existing methods struggle to achieve a balance between missed detections and false detections. Summary of the Invention

[0004] Therefore, this invention proposes a ship target detection method that can dynamically and adaptively fuse optical and SAR decision information to fully utilize their complementary advantages and improve the overall performance of ship target detection.

[0005] This application provides a ship target detection method based on feature-guided adaptive decision-making, which utilizes the fusion of optical imagery and SAR imagery for ship target detection. The method includes:

[0006] S1: Input the registered optical image and SAR image into the first target detection network and the second target detection network respectively to obtain the first ship detection result set and the second ship detection result set respectively. Each ship detection result set contains the bounding box of the ship target, the detection confidence, and the region feature vector.

[0007] S2: Based on the intersection-union ratio, spatial matching is performed on the first ship detection result set and the second ship detection result set to form multiple matching pairs;

[0008] S3: Input the regional feature vectors in each matching pair into the reliability evaluation network, and then output the normalized optical mode weights and SAR mode weights from the reliability evaluation network.

[0009] S4: Based on optical mode weights and SAR mode weights, the confidence scores and bounding boxes of the matching pairs are weighted and fused to generate the final ship detection results.

[0010] According to one embodiment of the method of this application, in step S1,

[0011] The first object detection network and the second object detection network have the same structure, but they are trained independently of each other.

[0012] According to one embodiment of the method of this application, step S2 includes:

[0013] S21: Traverse all bounding boxes in the first ship detection result set D_"opt" and the second ship detection result set D_"SAR", and calculate the intersection-union ratio of the bounding boxes for each matching pair;

[0014] S22: Set a matching threshold. If the intersection-union ratio of the bounding box of the target detected by the optical image and the bounding box of the target detected by the SAR image are both greater than the matching threshold, then a matching pair will be considered to have been found.

[0015] According to one embodiment of the method of this application, step S3 includes:

[0016] S31: Concatenate the regional feature vectors of the matched pairs through channels to obtain fused features;

[0017] S32: Input the fused features into the reliability assessment network.

[0018] According to an embodiment of the method of this application, the reliability evaluation network in step S3 includes an input layer, a first fully connected layer, a ReLU activation function, a Dropout layer, a second fully connected layer, a ReLU activation function, and an output fully connected layer connected in sequence. The output fully connected layer is followed by a Softmax function, which is used to map the stitched bimodal region feature vector into normalized optical mode weights and SAR mode weights.

[0019] According to one embodiment of the method of this application, step S4 includes:

[0020] S41: The fusion confidence score is obtained by multiplying the optical confidence score in the optical image by the corresponding optical mode weight and adding the SAR confidence score in the SAR image by the corresponding SAR mode weight.

[0021] S42: Multiply the coordinates of the bounding box of the target in the optical image by the corresponding optical mode weight, and add the coordinates of the bounding box of the target in the SAR image by the corresponding SAR mode weight to obtain the coordinates of the fused bounding box.

[0022] S43: If the fusion confidence score is greater than the preset confidence score threshold, the fused bounding box will be output as the final ship detection result.

[0023] According to one embodiment of the method of this application, step S4 further includes:

[0024] S44: For unmatched SAR detection results, if their confidence level is greater than the preset single-mode confidence level threshold, they will be directly output as the final detection result; for unmatched optical detection results, if their confidence level is greater than the preset single-mode confidence level threshold, they will be directly output as the final detection result.

[0025] According to one embodiment of the method of this application, the method further includes:

[0026] S5: Jointly train the first target detection network, the second target detection network, and the reliability evaluation network.

[0027] According to one embodiment of the method of this application, the total loss function used in the joint training is a weighted sum of optical branch detection loss, SAR branch detection loss and fusion result detection loss.

[0028] According to one embodiment of the method of this application, in step S1, the region feature vector is the output feature of the fully connected layer before the classification head in the respective object detection network.

[0029] This application also provides a computer-readable storage medium having software instructions stored thereon, which, when executed, perform the above-described method.

[0030] This invention provides a novel optical-SAR method for ship target detection, effectively solving the problem of limited detection performance of single sensors in complex marine environments. By introducing deep feature information into the decision fusion process, "perception-driven decision-making" is achieved, which not only improves the detection accuracy and system robustness of small targets, but also provides a generalizable technical framework for intelligent interpretation of multi-source remote sensing, and can be widely used in fields such as maritime supervision, maritime security, and marine economic monitoring. Attached Figure Description

[0031] The following description, in conjunction with the accompanying drawings, will further illustrate the above-mentioned features, technical characteristics, advantages, and implementation methods of this application in a clear and understandable manner. The accompanying drawings are for illustrative and explanatory purposes only and do not limit the scope of this application. Wherein:

[0032] Figure 1 This is an overall flowchart of the method described in this invention;

[0033] Figure 2 This is a network structure diagram of the present invention;

[0034] Figure 3 This is a schematic diagram of the reliability assessment network. Detailed Implementation

[0035] To better understand the above technical solutions, the technical solutions of this application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of this application and the specific features in the embodiments are detailed descriptions of the technical solutions of this application, rather than limitations on the technical solutions of this application. In the absence of conflict, the embodiments of this application and the technical features in the embodiments can be combined with each other.

[0036] This invention provides a ship target detection method based on feature-guided adaptive decision-making, which utilizes the fusion of optical and SAR images for ship target detection, such as... Figure 1 and Figure 2 As shown, the method includes:

[0037] S1: Input the registered optical image and SAR image into the first target detection network and the second target detection network respectively to obtain the first ship detection result set and the second ship detection result set respectively. Each ship detection result set contains the bounding box of the ship target, the detection confidence, and the regional feature vector (detection region confidence vector).

[0038] This step specifically includes:

[0039] S11: Process optical images using a first object detection network based on deep learning (Faster R-CNN) to output the first ship detection result set. This includes the bounding box of the ship target, detection confidence, and region feature vector;

[0040] S12: Use a second target detection network (with the same structure as the first target detection network, but trained independently) to process SAR images and output a second set of ship detection results. This includes the bounding box of the ship target, the detection confidence, and the region feature vector.

[0041] Among them, the "region feature vector" refers to the output features of the fully connected layer before the classification head in each object detection network.

[0042] S2: Based on the intersection-union ratio, spatial matching is performed on the first ship detection result set and the second ship detection result set to form multiple matching pairs;

[0043] In this step, the intersection-over-union ratio (IoU) between the optical bounding box and the SAR bounding box is calculated. If the IoU is greater than a preset threshold τ and both categories are ships, then they are determined to be a matching pair. This step specifically includes:

[0044] S21: Traverse the first ship detection result set Second ship inspection result set For all bounding boxes in the dataset, calculate the intersection-over-union (IoU) ratio of the bounding boxes for each matching pair.

[0045] S22: Set matching threshold (For example If the intersection-union ratio (IU) of the bounding boxes of the target detected by the optical image and the bounding boxes of the target detected by the SAR image is greater than the threshold, then it will be considered that a matching pair of optical image and SAR image has been found.

[0046] S3: Input the region feature vector (detection region confidence vector) in each matching pair into the reliability evaluation network (REN), and then the reliability evaluation network outputs the normalized optical mode weights and SAR mode weights.

[0047] This step specifically includes:

[0048] S31: Match the pairs The regional feature vectors are concatenated through channels to obtain the fused features. ;

[0049] S32: Merge features The input is a reliability evaluation network (REN) consisting of a lightweight multilayer perceptron (MLP), such as Figure 3 As shown, the reliability evaluation network includes an input layer, a first fully connected layer, a ReLU activation function, a Dropout layer, a second fully connected layer, a ReLU activation function, and an output fully connected layer connected in sequence. The output fully connected layer is followed by a Softmax function, which is used to map the concatenated bimodal region feature vectors into normalized optical modality weights. With SAR mode weights And satisfy .

[0050] S4: Based on optical mode weights and SAR mode weights, the confidence and bounding boxes of the matching pairs are weighted and fused to generate the final ship detection results;

[0051] This step specifically includes:

[0052] S41: The fused confidence score is obtained by multiplying the optical confidence score in the optical image by the corresponding optical mode weight, and then adding the SAR confidence score in the SAR image by the corresponding SAR mode weight.

[0053] Fusion confidence score = Optical mode weight × Optical confidence score + SAR mode weight × SAR confidence score;

[0054] S42: Multiply the coordinates of the bounding box of the target in the optical image by the corresponding optical mode weight, and add the coordinates of the bounding box of the target in the SAR image by the corresponding SAR mode weight to obtain the coordinates of the fused bounding box, wherein the fused bounding box is represented by a four-dimensional vector of center coordinates and width and height.

[0055] The fused bounding box = optical mode weight × optical bounding box + SAR mode weight × SAR bounding box, where the bounding box is represented by a four-dimensional vector of center coordinates and width and height;

[0056] S43: If the fusion confidence is greater than the preset confidence threshold (e.g., 0.5), the fused bounding box will be output as the final ship detection result.

[0057] S44: For unmatched SAR detection results (i.e., optically undetected), if its confidence level s SAR Greater than the preset single-modal confidence threshold T single If the result is not a match, it will be directly output as the final detection result; similarly, for unmatched optical detection results (such as SAR not being detected due to low backscattering), if its confidence level s opt Greater than the preset single-modal confidence threshold T single , and will also be retained.

[0058] S5: Jointly train the first target detection network, the second target detection network, and the reliability evaluation network.

[0059] The total loss function used in the joint training is a weighted sum of the optical branch detection loss, the SAR branch detection loss, and the fusion result detection loss. This step specifically includes:

[0060] Using an end-to-end training approach, the total loss function... It consists of three parts:

[0061]

[0062] in, and These are target detection losses for optical and SAR branches (such as Focal Loss). λ represents the detection loss of the fusion result, and λ is the balance coefficient of the fusion loss, with a value range of [0.5, 2.0], preferably 1.0.

[0063] In summary, this invention proposes a feature-guided adaptive decision-making method for ship target detection using optical-SAR imagery. The method first performs target detection on registered optical and synthetic aperture radar (SAR) images separately, obtaining two initial detection results containing bounding boxes, confidence scores, and target region features. Second, it spatially matches the two sets of results based on the intersection-over-union (IoU) ratio, forming detection pairs. Then, it constructs a lightweight reliability evaluation network (REN), taking the dual-modal target region features from the matched pairs as input and dynamically outputting normalized fusion weights for the optical and SAR modes. Finally, it performs weighted fusion of the confidence scores and bounding boxes of the detection pairs based on these weights, generating the final ship detection result, and optimizes the entire system through end-to-end joint training. This invention can adaptively trust more reliable sensor modes based on the contextual features of target instances, significantly improving the detection accuracy and robustness of ship targets (especially small-scale ships) in complex sea conditions.

[0064] Compared with existing technologies, this invention has the following significant advantages: First, it breaks through the limitations of traditional decision-level fusion methods that use fixed weights or simple voting rules, and innovatively introduces a feature-guided mechanism. It utilizes the target area features output by dual-modal detectors to dynamically evaluate the reliability of each sensor under the current target instance, achieving true adaptive decision fusion. Second, this method can intelligently adjust the trust level of optical or SAR modes according to specific scenarios (such as clear daytime, cloud cover, high sea clutter, etc.). When optical modes are effective, its high-resolution advantage is emphasized; when SAR modes are more reliable, its all-weather capability is relied upon, thus significantly improving the robustness and environmental adaptability of detection. Finally, through end-to-end joint training, the entire system is collaboratively optimized, especially in the detection of small-scale ship targets, effectively reducing the false alarm rate and the missed detection rate. The overall detection accuracy is significantly better than existing single-modal methods and traditional fusion strategies.

[0065] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A ship target detection method based on feature-guided adaptive decision-making, utilizing the fusion of optical imagery and SAR imagery for ship target detection, the method comprising: S1: Input the registered optical image and SAR image into the first target detection network and the second target detection network respectively to obtain the first ship detection result set and the second ship detection result set respectively. Each ship detection result set contains the bounding box of the ship target, the detection confidence, and the region feature vector. S2: Based on the intersection-union ratio, spatial matching is performed on the first ship detection result set and the second ship detection result set to form multiple matching pairs; S3: Input the regional feature vectors in each matching pair into the reliability evaluation network, and then output the normalized optical mode weights and SAR mode weights from the reliability evaluation network. S4: Based on optical mode weights and SAR mode weights, the confidence scores and bounding boxes of the matching pairs are weighted and fused to generate the final ship detection results.

2. The method according to claim 1, wherein, In step S1 The first object detection network and the second object detection network have the same structure, but they are trained independently of each other.

3. The method according to claim 1, wherein, Step S2 includes: S21: Traverse the first ship detection result set Second ship inspection result set For all bounding boxes in the dataset, calculate the intersection-union ratio (IUU) of the bounding boxes for each matching pair; S22: Set a matching threshold. If the intersection-union ratio of the bounding box of the target detected by the optical image and the bounding box of the target detected by the SAR image are both greater than the matching threshold, then a matching pair will be considered to have been found.

4. The method according to claim 1, wherein, Step S3 includes: S31: Concatenate the regional feature vectors of the matched pairs through channels to obtain fused features; S32: Input the fused features into the reliability assessment network.

5. The method according to claim 1, wherein, The reliability evaluation network in step S3 includes an input layer, a first fully connected layer, a ReLU activation function, a Dropout layer, a second fully connected layer, a ReLU activation function, and an output fully connected layer connected in sequence. The output fully connected layer is followed by a Softmax function, which is used to map the stitched bimodal region feature vector into normalized optical mode weights and SAR mode weights.

6. The method according to claim 1, wherein, Step S4 includes: S41: The fusion confidence score is obtained by multiplying the optical confidence score in the optical image by the corresponding optical mode weight and adding the SAR confidence score in the SAR image by the corresponding SAR mode weight. S42: Multiply the coordinates of the bounding box of the target in the optical image by the corresponding optical mode weight, and add the coordinates of the bounding box of the target in the SAR image by the corresponding SAR mode weight to obtain the coordinates of the fused bounding box. S43: If the fusion confidence score is greater than the preset confidence score threshold, the fused bounding box will be output as the final ship detection result.

7. The method according to claim 6, wherein, Step S4 also includes: S44: For unmatched SAR detection results, if their confidence level is greater than the preset single-mode confidence level threshold, they will be directly output as the final detection result; for unmatched optical detection results, if their confidence level is greater than the preset single-mode confidence level threshold, they will be directly output as the final detection result.

8. The method according to claim 1, wherein, Also includes: S5: Jointly train the first target detection network, the second target detection network, and the reliability evaluation network. The total loss function used in the joint training is a weighted sum of the optical branch detection loss, the SAR branch detection loss, and the fusion result detection loss.

9. The method according to claim 1, wherein, In step S1, the region feature vector is the output feature of the fully connected layer before the classification head in each target detection network.

10. A computer-readable storage medium having software instructions stored thereon, which, when executed, perform the method according to any one of claims 1-9.

Images