A polar route ship target fusion detection method based on SAR and optical images

Abstract

The application provides a kind of polar route ship target fusion detection method based on SAR and optical image, belongs to remote sensing image processing and target detection field.Aiming at the problem that the moving target is difficult to associate in the different phase images of arctic ice area, the false alarm rate of sea ice is high, the technical scheme includes: respectively to SAR and optical image are initially detected, and the features are extracted to eliminate sea ice false alarm;Then the saliency detection is carried out to extract the channel area, and the target parameters such as the main shaft angle, position and intersection ratio are calculated;At the same time, the motion state is distinguished according to the difference of target parameters, and the maximum correlation area is determined by combining the maximum speed constraint;Finally, the target matching is completed in the area, and the D-S evidence theory is used for decision level fusion.The application realizes the accurate correlation and fusion detection of time shift ship targets in SAR and optical image, effectively overcomes the background interference by comprehensively utilizing the double modal information, and has important application value in the field of polar navigation safety monitoring.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing and target detection, and in particular to a method for fusion detection of ship targets in waterways based on the collaboration of Synthetic Aperture Radar (SAR) and optical remote sensing images, applicable to complex polar sea ice environments. Background Technology

[0002] With the gradual opening of Arctic shipping routes, polar shipping activities are increasing, and real-time and accurate detection of moving vessels in these routes is crucial for navigation safety, traffic management, and emergency response. SAR and optical remote sensing are currently the main methods for monitoring polar vessels: SAR has all-weather, day-and-night observation capabilities, unaffected by clouds, snow, or polar night; optical images offer high spatial resolution and rich spectral texture information, facilitating detailed vessel identification. However, the monitoring targets in polar shipping routes are mostly vessels in continuous motion. Accurately locating and associating moving targets on SAR and optical images from different time phases is challenging; although the background of the shipping routes is relatively open, floating ice floes, icebergs, and other false targets still pose a continuous obstacle to detection. It is worth noting that the unique operating environment of the Arctic shipping routes also provides new opportunities for detection. Constrained by complex sea ice distribution and harsh weather conditions, vessels typically strictly follow predetermined routes, exhibiting significant spatial clustering; coupled with the overall sparse shipping activity in polar regions, this prior knowledge provides key constraints for effectively screening and associating real vessel targets. Furthermore, in waterway scenarios, acquiring simultaneous, highly registered SAR and optical images is extremely difficult, and heterogeneous images of moving vessels often exhibit significant temporal and geometric differences. Therefore, directly applying the underlying feature fusion strategy faces enormous challenges.

[0003] Current research largely focuses on ship detection in static scenarios such as ports or on single-modal detection methods, with limited research on multimodal collaborative detection of moving vessels in polar waterways. Existing methods have significant shortcomings in moving target association, sea ice false alarm suppression, and heterogeneous data decision fusion, failing to meet the urgent needs of dynamic monitoring in polar waterways. Therefore, there is a pressing need to develop a fusion detection method that can effectively address time shifts, overcome background interference, and fully utilize complementary information from both modes. Summary of the Invention

[0004] (a) Technical problems to be solved

[0005] This invention aims to provide a fusion detection method for ship targets in polar waterways based on SAR and optical imagery, which solves the problems of difficulty in matching the displacement of moving ships, high false alarm rate in sea ice background, and insufficient reliability of single-mode detection methods caused by the asynchronous imaging time of SAR and optical images in the complex sea ice and meteorological environment of the polar region, thereby realizing the automated detection of moving ships in polar waterways.

[0006] (II) Technical Solution

[0007] This invention includes the following steps:

[0008] Step 1: Detection and false alarm removal of SAR and optical images respectively.

[0009] Initial ship target detection was performed on SAR and optical images using deep learning models respectively. For false alarms caused by sea ice in SAR images, the texture, geometric and polarization features of the targets were extracted, and a "ship-iceberg" binary classification was performed by a classifier to remove false alarms caused by sea ice.

[0010] Step 2: Channel detection and target parameter extraction.

[0011] First, saliency detection based on anisotropic Gaussian filtering is performed on the SAR image to extract channel structure information and determine the channel centerline and its dominant direction. Then, for each detected target in the SAR and optical images obtained in step one, its position information and the principal axis angle representing the target orientation are extracted.

[0012] Step 3: Motion state determination and associated region construction.

[0013] Calculate the intersection-union ratio (IoU) of the detection boxes in SAR and optical images; based on the IoU matching results and the difference between the target principal axis angle and the flight path direction, set a threshold to determine the target's dynamic or static state. For dynamic candidate targets, using their position in the SAR image as a benchmark, combined with the flight path direction and maximum speed constraints, construct their maximum predictable correlation region in the different temporal phase optical images.

[0014] Step 4: Integration of target matching association with decision-making based on DS evidence theory.

[0015] Cross-modal target matching is performed within the region of maximum predicted association. Using the DS evidence theory, evidence is constructed based on the detection status and confidence of the target in different modalities and then fused at the decision level to output the fused confidence.

[0016] (III) Beneficial Effects

[0017] The advantages of this invention are as follows:

[0018] By introducing the correlation region construction of principal axis angle consistency discrimination and channel constraints, the matching problem of moving targets in different time phase images is effectively solved. Combined with targeted sea ice false alarm removal and decision fusion based on DS evidence theory, the detection accuracy and anti-interference ability in complex ice background are significantly improved, and reliable detection of moving vessels in polar channels is realized. Attached Figure Description

[0019] Figure 1 The flowchart of a method for fusion detection of ship targets in polar waterways based on SAR and optical imagery provided by the present invention is shown.

[0020] Figure 2 The present invention provides target detection results for SAR and optical images based on Faster R-CNN.

[0021] Figure 3 The results of image saliency detection provided by this invention.

[0022] Figure 4 The results provided by this invention are for generating a dynamic target search zone based on the channel centerline.

[0023] Figure 5 The final result of the maximum correlation region of the moving ship target in different time phases provided by the present invention. Detailed Implementation

[0024] To make the objectives, contents, and advantages of the present invention clearer, the following description is provided in conjunction with the appendix. Figure 1 The specific embodiments of the present invention will be described in further detail below:

[0025] This invention provides a method for ship target fusion detection in polar shipping lanes based on SAR and optical imagery, the process of which is as follows: Figure 1 As shown, the main steps include SAR and optical image detection and false alarm removal, channel detection and target parameter extraction, motion state discrimination and associated region construction, and target matching association and decision fusion based on DS evidence theory. The specific implementation is as follows:

[0026] Step 1: Detection and false alarm removal of SAR and optical images respectively.

[0027] First, preprocessing and spatial synchronization were performed on both the Sentinel-1 SAR imagery and the Sentinel-2 optical imagery to ensure consistent geographic coordinates. Then, a Faster R-CNN deep learning detection model was used to perform preliminary independent ship target detection on the preprocessed SAR and optical images. The model output included detection results containing target bounding box locations and confidence levels. The results are shown below. Figure 2 As shown.

[0028] To address the false alarm problem caused by sea ice interference in SAR imagery, texture, geometric, and polarization features of the SAR target region are extracted, resulting in a total of 52-dimensional feature vectors. An independent samples t-test is used to evaluate the class discrimination ability of each feature dimension. For the i-th feature, its sample mean μ0⁽ is calculated for both the ship (C0) and iceberg (C1) classes. 1 ⁾、μ1⁽ 1 ⁾ and variance σ0⁽1 ⁾²、σ1⁽ 1 The formula for calculating the t-value is:

[0029]

[0030] Where n0 and n1 are the number of samples in the two classes, respectively. i The larger the value, the stronger the feature's ability to distinguish between the two types of targets. All 52-dimensional features are sorted in descending order, and the top 6 features are selected to form the optimal feature subset. Then, the LightGBM classifier is trained on a publicly available iceberg-ship dataset to achieve binary classification of "ship-iceberg," thereby eliminating false alarms caused by sea ice.

[0031] Step 2: Channel detection and target parameter extraction.

[0032] Based on the SAR image detection results obtained in step one, saliency detection is first performed to extract the waterway: an anisotropic Gaussian filter is used to enhance the linear structure, and its response function is:

[0033]

[0034] in: For pixel coordinates, For scale parameters, The principal direction angle of the Gaussian filter kernel. Here is the anisotropic proportionality constant. For rotating coordinates, by and The calculation yields a binary region of the waterway after threshold segmentation and morphological processing, from which the centerline and main direction of the waterway are extracted; for example... Figure 3 As shown, an improved linear saliency detection method is used to extract the waterway from SAR imagery, obtaining the waterway centerline. .

[0035] Subsequently, the principal axis angle and bounding box coordinates of each detected target in the SAR and optical images were extracted separately. The principal axis angle was calculated using a combination of principal component analysis and ellipse fitting: the principal axis direction angle of the principal component analysis is:

[0036]

[0037] in, It is the central moment of the target; the principal axis direction angle of the ellipse fitting is:

[0038]

[0039] in, These are the coefficients of the quadratic form that constitute the equation of the ellipse.

[0040] Step 3: Motion state determination and associated region construction.

[0041] Calculate the intersection-over-union ratio (IoU) of the detection boxes in the SAR and optical images, and calculate the principal axis angle difference for target pairs with successful IoU matching. If the target location matches and Less than the set threshold If it is true, it is determined to be a static target; otherwise, it is considered a dynamic candidate target.

[0042] For any dynamic candidate target, such as Figure 4 As shown, the channel centerline is used as a reference, and a fixed distance is extended to both sides. Pixels form a strip-shaped channel area , serving as the geometric constraint range; the target center location is Calculate its distance to the nearest channel centerline vertical projection point Combining kinematic constraints, namely the ship's maximum speed... Time difference and image resolution Calculate the target in Maximum reachable radius within The calculation formula is as follows:

[0043]

[0044] like Figure 5 As shown, with projection points Centered on the waterway, along the direction of the waterway and its opposite direction Construct radii of respectively Zhang Jiao The fan-shaped motion can reach the area and Ultimately, the maximum predicted correlation region of this dynamic target in the intertemporal imagery. Determined by the intersection of the fan-shaped movement region and the strip-shaped channel region, it is defined as:

[0045] .

[0046] Step 4: Integration of target matching association with decision-making based on DS evidence theory.

[0047] Within the maximum predicted correlation region determined in step three, cross-modal and temporal matching is performed on the detection results of SAR and optical imagery: For example... Figure 5 As shown, if a SAR-detected target and an optically detected target are both located within the same maximum prediction correlation region, they are determined to be different temporal observations of the same moving target. Based on the matching results, each target has three detection states: (Detected optically only) (Detected only by SAR) (Both were detected). For each state, a framework for identifying evidence based on the DS theory was constructed. ,in This indicates that a target has been detected. This indicates "No target detected".

[0048] For the two Arctic ship detection scenarios described above, a Basic Probability Assignment (BPA) needs to be assigned to the detection results of the two types of sensors. The detection output of the optical sensor is considered as evidence E1, with a basic probability assignment of m1; the detection output of the SAR sensor is considered as evidence E2, with a basic probability assignment of m2. Each basic probability assignment contains three parameters: m1(A) and m2(A) represent the confidence levels of the optical and SAR sensors in supporting the existence of the target, respectively; m1(¬A) and m2(¬A) represent the confidence levels in supporting the non-existence of the target, respectively; and m1(Θ) and m2(Θ) are uncertainty probabilities, reflecting the degree of unknown about the detection result by the sensor, and satisfying m1(A) + m1(¬A) + m1(Θ) = 1 and m2(A) + m2(¬A) + m2(Θ) = 1. In optical-SAR imagery matching scenarios, if both optical and SAR images clearly show a ship target at the same location, m1(A) and m2(A) can be set to higher values, while uncertainty probabilities m1(Θ) and m2(Θ) can be set to lower values. In optical-SAR imagery single-target displacement scenarios, if the detected positions of the two images are different but both point to the same target, m1(A) and m2(A) can be appropriately reduced to increase the uncertainty probabilities and reflect the impact of positional differences.

[0049] Based on the aforementioned basic probability allocation, the synthesis rules of DS evidence theory are used to fuse the two pieces of evidence. For the identification framework... Two independent pieces of evidence within and The basic probability distribution after fusion It can be calculated using the following formula:

[0050] ,

[0051] In the formula, A and B are arbitrary combinations of propositions in the identification frame Θ, respectively from the focal element sets of evidence bodies E1 and E2, i.e., A and B can each be {A}, {¬A}, or {Θ}; A∩B=C represents the summation operation traversing all cases that satisfy "the intersection of proposition A and proposition B equals proposition C"; K is the conflict coefficient, used to measure the degree of conflict between the two evidence bodies, and is calculated using the following formula:

[0052] ,

[0053] when When the above synthesis rules are valid, the fusion result is... This is the overall confidence level of "target detected"; if This indicates that the two pieces of evidence completely conflict, and the evidence needs to be revised based on the actual context before being integrated. After integration, As the final target fusion confidence level, it is compared with a preset threshold. Comparison: If If it is a valid target, it will be output as a valid target, and at the same time... As a credibility assessment indicator; if If the result is not found, it is determined that the target was not successfully fused.

[0054] Through this process, DS evidence theory can effectively handle the detection uncertainties of optical and SAR sensors in different Arctic scenarios. It can strengthen the confidence of double-confirmed targets in "matching scenarios" and weaken the interference caused by position deviations through probability synthesis in "target displacement scenarios", ultimately achieving complementary advantages of the two types of sensor detection information.

Claims

1. A method for ship target fusion detection in polar shipping lanes based on SAR and optical imagery, characterized in that, Includes the following steps: (1) Detection and false alarm removal of SAR and optical images respectively: The deep learning detection model is used to perform preliminary ship target detection on polar SAR and optical images that are spatially synchronized but have time intervals. The detection results include target position and confidence level. In order to address the false alarm problem caused by sea ice in SAR images, the texture, geometric and polarization features of the target are extracted, and the detection results are classified into "ship-iceberg" binary classification by a classifier to remove sea ice false alarms and obtain the target detection results of SAR and optical images respectively. (2) Channel detection and target parameter extraction: Based on the detection results obtained in step (1), the image is first subjected to saliency detection to extract channel information and obtain the channel centerline and its direction; then, based on the target detection results of the SAR and optical images obtained in step (1), the principal axis angle and position information of each detected target are extracted respectively. (3) Motion state discrimination and associated region construction: Calculate the intersection-union ratio of the detection boxes in the SAR and optical images; based on the IoU matching results and the difference between the principal axis angle and the channel direction of the detection target extracted in step (2), set a threshold to determine whether the target is in a stationary state or a moving state; if the target's position matches in the SAR and optical images and the difference between the principal axis angle and the channel direction is less than the set threshold, it is determined to be a static target; otherwise, it is considered a dynamic candidate target; for dynamic candidate targets, based on their position in the SAR image, combined with the channel direction extracted in step (2) and the target's own principal axis direction, under the preset maximum speed constraint, calculate and construct the maximum predicted associated region of the dynamic candidate target in the different phase optical images; The motion state discrimination and associated region construction include: ① Target motion state determination: Calculate the intersection over union (IOU) of the detection boxes in the SAR and optical images; for target pairs with successful IoU matching, calculate the difference in their principal axis angles in the SAR and optical images. And set an angle difference threshold. If the target location matches and If it is, it is determined to be a static target; otherwise, it is considered a dynamic candidate target. ② Channel geometric modeling and dynamic region calculation: Based on the channel information extracted in step (2), the channel region is geometrically modeled, and its principal axis direction angle is calculated using principal component analysis. Based on this direction, the boundary points are grouped, and the upper and lower boundary lines are fitted using the least squares method. and Then by distance The pixels expand to both sides, forming an extended strip-shaped channel area. Its centerline is Based on kinematic constraints, the maximum speed of the ship... Time difference and image resolution The maximum reachable radius is calculated as follows: ③ For any SAR-detected target, locate the center point of its bounding box. Calculate the distance from this point to the nearest channel centerline. vertical projection point ; ④ Generation of the maximum associated region: based on the projection points Centered on a central region, a complex relational region is constructed, which consists of two intersecting parts: one is two sector regions defined by kinematic constraints. and Their radii are all Zhang Jiao The directions of the central axis are respectively the directions of the waterway. and its opposite direction ; secondly, the extended strip-shaped waterway area obtained from step (1) Ultimately, the target's dynamic maximum correlation region. Determined by the intersection of the two: ； (4) Target matching association and decision fusion based on DS evidence theory: Based on the maximum association area obtained in step (3), target matching is performed in the area, that is, targets in SAR and optical images that are simultaneously located in the area are identified as the same target; for the detection results, based on their detection status and confidence, decision-level fusion is performed using DS evidence theory, and the fused target confidence is output.

2. The method for ship target fusion detection in polar waterways based on SAR and optical imagery according to claim 1, characterized in that, The SAR and optical image detection and false alarm removal described in step (1) include: (1) The detection adopts a deep learning model based on Faster R-CNN, with ResNet-FPN as the backbone network and a feature pyramid network for multi-scale feature extraction. (2) In the false alarm elimination of SAR images, texture features, geometric features and polarization statistical features are extracted for the detected candidate targets. The top six features with the highest discrimination are selected. The independent samples t-test is used to evaluate the class discrimination ability of each feature. For the i-th feature, the sample mean μ0⁽ in the ship class (C0) and iceberg class (C1) is calculated respectively. 1 ⁾、μ1⁽ 1 ⁾ and variance σ0⁽ 1 ⁾²、σ1⁽ 1 The formula for calculating the t-value is: Where n0 and n1 are the number of samples in the two classes, respectively, |t i The larger the value, the stronger the ability of the feature to distinguish between the two types of targets. All 52-dimensional features are sorted in descending order, and the top 6 features are selected to form the optimal feature subset. Then, the LightGBM classifier is trained on the public iceberg-ship dataset to achieve binary classification of "ship-iceberg", thereby eliminating false alarm targets caused by sea ice.

3. The method for ship target fusion detection in polar waterways based on SAR and optical imagery according to claim 1, characterized in that, The waterway detection and target parameter extraction described in step (2) include: (1) Channel detection: Saliency detection is performed on SAR images to extract channels. Specifically, anisotropic Gaussian filtering is used to enhance linear structures, and its response function is: in: These are the original two-dimensional coordinates of the pixels in the SAR image. This is the scaling parameter of the Gaussian filter, which controls the effective range of the filter. The principal orientation angle of the Gaussian filter kernel determines the direction of the linear characteristic enhanced by the filter. The anisotropy scaling constant controls the Gaussian kernel in the vertical direction ( Direction) relative to the main direction ( The decay rate (in the direction) is used to selectively enhance slender linear structures. Original coordinates Rotate around the filter center point The new coordinates after the angle are determined by and Calculated; (2) Target parameter extraction: After obtaining the channel centerline and its principal direction, the position information and principal axis angle of each detected target in the SAR and optical images are extracted respectively; the principal axis angle extraction adopts the method of principal component analysis combined with ellipse fitting, and the principal axis direction angle of the principal component analysis is: in, It is the central moment of the target; eigenvalue decomposition and ellipse approximation are performed on the binary region of the target, and the principal axis direction angle of the ellipse fitting is: in, These are the coefficients of the quadratic form that constitute the equation of the ellipse.

4. The method for ship target fusion detection in polar waterways based on SAR and optical imagery according to claim 1, characterized in that, The target matching association and decision fusion based on DS evidence theory described in step (4) include: (1) Target matching and association are carried out within the constructed association area. The cross-modal association of dynamic targets is completed by combining the consistency of target position, direction of motion and maximum velocity constraints. That is, targets in SAR and optical images that are simultaneously located in the area are identified as the same target. (2) In the DS evidence theory, the identification frame Θ={A,¬A} is defined, where A represents "real vessel detected" and ¬A represents "real vessel not detected". Based on the detection status of the target in SAR and optical images (M1, M2, M... 12 The basic probabilities are assigned to them, and the fusion calculation uses the DS synthesis rule, the formula of which is: In the formula, A and B represent propositions in evidence bodies E1 (optical) and E2 (SAR), respectively, and can take values ​​{A}, {¬A}, or {Θ}; C is the proposition to be calculated after fusion; m1(A) and m2(B) are the basic probability assignments of the corresponding propositions; K is the conflict coefficient, which measures the degree of conflict between the two pieces of evidence and is defined as: Set fusion decision threshold If the proposition is merged reliability If the result is positive, it is determined to be the final detected target, and its fusion confidence score is output.

Images