Ship contour extraction method from SAR images based on the combined method of Faster R-CNN and CV model

Abstract

The present invention discloses a method for extracting ship contours from SAR images based on a combined method of Faster R-CNN and a CV model, and relates to the technical field of satellite remote sensing image processing and application. The purpose of the present invention is to solve the problems of existing SAR image ship contour methods, such as limited applicable scenarios, poor extraction accuracy, and severe detail loss. The present invention can extract ship contours from large-scene SAR images by introducing a deep learning method. First, a Faster R-CNN network is used to perform transfer learning and target detection on the SSDD dataset to obtain the positioning coordinates of the ship target, and the small-scene ship area is sliced; then, the ship slice is subjected to fast non-local mean (FNLM) filtering processing, and finally, an improved CV model is used to iteratively generate the final ship contour, and the contour information is fused and displayed in the original image. The method for extracting ship contours from SAR images based on a combined method of Faster R-CNN and a CV model proposed by the present invention can achieve rapid and accurate extraction of ship target contours in SAR images of different scenes.

Description

Technical Field

[0001] The invention belongs to the technical field of satellite remote sensing image processing and application. Background Art

[0002] The target contour extraction method is an important branch of SAR image target detection. With the continuous advancement of SAR radar imaging technology, the resolution of SAR images has been greatly improved, making high-precision sea surface ship target contour extraction and deep fusion information application gradually become the research focus of sea detection. Traditional contour extraction methods are mostly targeted at specific scenes or optical images, mainly for modeling and simulating the sea surface environment and target features. These methods are generally only applicable to specific scenes such as simple sea surface scenes or single targets. For the task of multi-target contour extraction in complex scene SAR images, there are mainly two categories: extraction methods based on image segmentation and extraction methods based on target detection. Representative studies are as follows:

[0003] Segmentation extraction methods directly segment objects using a segmentation algorithm and define the resulting instance edges as the object contour. Traditional segmentation methods are mainly divided into those based on region information and those based on models. Region-based SAR image segmentation methods can effectively extract effective image information. Superpixels are a commonly used region segmentation method. They introduce the concept of seed points and the idea of ​​dynamic growth. One or more preselected pixels are used as seeds. Through iteration, pixels adjacent to the seed points with a certain degree of similarity are merged into connected regions, ultimately achieving a partitioning of similar regions (see TurboPixels: Fast superpixels using geometric flows). Superpixel segmentation utilizes image statistics to segment an image into consistent pixel blocks within a local region, transforming the image segmentation problem from the pixel level to the region level. This method is simple and easy to use, reducing the complexity of subsequent processing. However, this method often requires the combination of image preprocessing methods and has high requirements for seed point selection, merging criteria, and growth termination conditions. Although simple and easy to use and achieves good segmentation results, the formation of holes during the growth process is a significant issue. Model-based segmentation methods have been a research hotspot for contour extraction in recent years. Their basic idea is to construct an energy functional of an active curve on the image to be detected, and in the process of finding the optimal solution of the energy functional, the active curve is continuously approximated to the true image boundary. Active contour models combine image features and structural information, and are more consistent with human visual understanding of target contours. In 1987, Kass et al. proposed the Snake model. During image processing, this model first sets an initial contour and uses elastic energy and bending energy to construct internal constraints to maintain the smoothness of the contour. External constraints, which are related to image information, are used to drive the contour curve toward the target boundary (see Snakes: Active Contour Models). Caselles et al. proposed the Geodesic Active Contours (GAC) model, which uses image gradient information to construct an energy functional and obtains the target contour by finding the minimum weighted arc length of the curve edge (see Geodesic active contours). Mumford and Shah et al. use region information to drive level set contours, constructing energy functionals using piecewise smooth functions to continuously approximate the true visual edge. Since the Mumford-Shah model uses the gradient information of the image to average the grayscale in the region, it is not applicable to images with uneven grayscale and is also sensitive to noise (see Optimal approximations by piecewise smooth functions and associated variational problems).Chan and Vese simplified the Mumford-Shah model, replacing piecewise smooth functions with piecewise constants for the target and background regions, respectively, and proposed the classic Chan-Vese model (see Active Contours without Edges for Vector-Valued Images). The CV model detects target contours based on global image information, and is effective for detecting contours with weak edges while reducing sensitivity to noise. However, the model is not applicable to images with uneven grayscale, and its convergence speed is affected by the initial contour. Traditional segmentation algorithms are unable to distinguish between target and background contours, resulting in limited application scenarios. In recent years, they have been gradually replaced by image segmentation methods based on deep learning. This method is extremely robust and scalable, avoids the uncertainty caused by noise and human intervention, and can achieve target instance segmentation without interference from the image background. The Mask R-CNN algorithm proposed by He et al. in 2017 is a typical representative of the "two-stage" instance segmentation algorithm. It is developed from target detection algorithms such as Fast R-CNN and Faster R-CNN (see Mask R-CNN). It proposed the RoIAlign layer to solve the problem of misalignment between the feature map and the region of interest on the original image. At the same time, it added a Mask prediction branch to achieve high-precision automated pixel-level instance segmentation tasks. Although Mask R-CNN and its various improved algorithms have good results in instance segmentation and detection accuracy, its detection speed is its biggest flaw and it is difficult to meet industrial needs. Secondly, the fixed-size Mask prediction branch makes the generated mask rough, and its practical application value is not strong. In response to the inefficiency of the two-stage instance segmentation algorithm, Daniel et al. proposed the highly real-time single-stage instance segmentation algorithm YOLACT in 2019, which decomposes instance segmentation into two parallel tasks, namely generating a mask combination of the entire input image and predicting the mask coefficient of each instance and combining the corresponding predicted coefficient with the entire image. Although YOLACT has solved the problem of slow detection speed of Mask R-CNN to a certain extent, it also has problems such as inability to accurately locate the target when the scene is complex and mask overlap between two instances that are far apart (see YOLACT: Real-time Instance Segmentation).

[0004] The detection and extraction method divides contour extraction into two stages. The first stage uses semi-automatic methods such as image preprocessing or manual interpretation to locate the target area in complex scenes. The second stage uses image segmentation algorithms to extract the contour. Compared with the segmentation method, the detection and extraction method reduces the target positioning area during the contour extraction stage, achieving higher contour accuracy. However, the accuracy of the target positioning is entirely dependent on the target detection stage. Among traditional SAR image ship detection algorithms, the Constant False Alarm Rate (CFAR) algorithm is the most widely used. In SAR image target detection, target signals are often easily submerged in noise and clutter. In any given area, the energy intensity of clutter and noise is uncertain, making it unsuitable to use a constant threshold to distinguish targets. The CFAR algorithm is a detection method that dynamically estimates the threshold using background cells around the detection cell (see Analysis of CFARprocessors in nonhomogeneous background). Due to its simple algorithm, constant false alarm rate, and adaptive threshold, it can quickly detect targets from complex backgrounds. However, since CFAR uses a pixel-by-pixel sliding window for target detection, the algorithm has a very high time complexity and is not suitable for wide-band SAR image detection and high-speed detection scenarios. In addition, the CFAR algorithm only uses pixels in a local area for statistical modeling of clutter distribution, ignoring the global information of the SAR image, making it unsuitable for ship detection in complex environments. In actual SAR images, the detection background environment is very complex. It includes not only pure ocean environments, but also islands and land. Especially in the ocean areas close to the land, there are also artificial port terminals, coastal dams, reefs, marine aquaculture, etc. These not only have high backscatter coefficients but also complex geometric shapes. Therefore, the detection accuracy of the CFAR algorithm is not ideal.In sharp contrast, deep learning has achieved remarkable results in the field of target detection. In 2015, Girshick et al. proposed the R-CNN algorithm (see Rich feature hierarchies for accurate object detection and semantic segmentation) and applied it to the field of target detection, which made a leap forward compared with traditional target detection algorithms. However, the detection efficiency was greatly reduced because the detection process of the model was divided into multiple stages. Therefore, He and Girshick et al. proposed the Spatial Pyramid Pooling Net (SPPNet) based on R-CNN (see Spatial PyramidPooling in Deep Convolutional Networks for Visual Recognition) and the Fast R-CNN algorithm (see Fast R-CNN). These two methods only need to send the image into the deep network once, and then map all candidate windows on a certain layer in the network, which improves the detection speed. However, in the Fast R-CNN algorithm, a large part of the time is sometimes spent on selective search. To address this problem, Ren et al. proposed the Faster R-CNN algorithm in 2018 based on Fast R-CNN (see Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks). They added a Region Proposal Network (RPN) and extracted candidate frames by setting anchor frames of different scales, replacing traditional candidate frame generation methods such as selective search. This achieved end-to-end training of the network, improved the network calculation speed, and is widely popular in engineering applications due to its good stability.

[0005] High-resolution SAR images can effectively highlight the morphological features of ship targets and provide reliable data for accurately extracting ship contours. The Faster R-CNN network based on deep learning methods has advantages in SAR image target detection. Using it as a detector can reduce background noise and interference from the background environment. The extractor based on the improved CV model can obtain detailed and accurate ship contours with a small amount of calculation. Different from traditional contour extraction methods and pure deep learning methods, this joint method utilizes the high target detection accuracy of Faster R-CNN and the high contour extraction accuracy of the improved CV model to extract the contours of ship targets in high-resolution SAR images. This method can simultaneously meet the requirements of extraction accuracy and speed, which is conducive to the accurate extraction of ship target contours. Summary of the Invention

[0006] There are many methods for ship detection in Synthetic Aperture Radar (SAR) images, but there is relatively little research on ship contour extraction from SAR images. In order to solve the problems of limited application scenarios, poor extraction accuracy, and severe detail loss of existing SAR image ship contour extraction methods, this paper provides a SAR image ship contour extraction method based on the combined method of Faster R-CNN and CV model. The specific steps of the technical solution adopted by the present invention are as follows:

[0007] Step 1: Select the SAR image ship target detection dataset (SSDD) as the detection network training data of this method, and relabel the ship outline;

[0008] Step 2: Build the Faster R-CNN target detection network and feed the dataset in step 1 into the network model for training.

[0009] Step 3: Send the test data into the trained network, output the ship coordinate points and slice the ship area image.

[0010] Step 4: After grayscale processing, the ship slice is filtered using the Fast Non-local Means (FNLM) filter.

[0011] Step 5: The data processed by the fast non-local mean filter is fed into the improved CV model to iteratively calculate the target contour, and the contour information is fused and displayed in the original image.

[0012] The implementation process of step 2 is as follows:

[0013] In step 2, to further improve the detection accuracy of the Faster R-CNN network, ResNet50 was used as the backbone network. The ResNet50 backbone network contains one input segment, four subsequent segments, and one output layer.

[0014] (a) The input segment consists of a convolutional layer (Conv) and a maximum pooling layer (Max pooling).

[0015] The parameters of the convolution layer are: kernel size kernel_size:, strides: 2; number of channels channels: 64; the parameters of the maximum pooling layer are: kernel size kernel_size:, strides: 2;

[0016] (b) The four subsequent segments contain 3, 4, 6, and 3 residual blocks respectively. Each residual block contains two branches, A and B. Branch A consists of three convolutional layers. The parameters of the first convolutional layer are: kernel size kernel_size:; the parameters of the second convolutional layer are: kernel size kernel_size:, strides: 2; the parameters of the third convolutional layer are: kernel size kernel_size:; branch B consists of one convolutional layer, and the parameters of the convolutional layer are: kernel size kernel_size:, strides: 2.

[0017] (c) The output layer consists of an average pooling layer and a fully connected layer.

[0018] At the same time, in order to improve the detection speed of the network, the RPN structure is used to extract candidate frames by reducing the number of anchor frames of different scales, replacing traditional candidate frame generation methods such as selective search, and realizing end-to-end training of the network.

[0019] The RPN architecture consists of two branches: classification and regression. The RPN performs a 3×3 convolution on the feature map output by the backbone network and then feeds it into the classification and regression branches. The classification branch consists of 18 1×1 convolutions, while the regression branch consists of 36 1×1 convolutions. Both branches simultaneously feed into the proposal layer, and through error calculation and repeated training, the approximate location of the proposal is determined.

[0020] In step 4 of the present invention, the FNLM filtering algorithm is used to enhance target details and suppress background noise within the slice. The method of using the FNLM filter for filtering is as follows:

[0021] Assume that a discrete noise image is g(i) = g{g(i)|i∈Ω}, whose size is N×N, Ω represents the image neighborhood, and i is the pixel index. After the noise image is processed by NLM filtering, it can be normalized and expressed as:

[0022]

[0023] Ω i represents a search window with a center pixel i and a size of q×q; ω(i, j) represents the weight assigned to the noise image g(i), where h represents the control exponential decay filter coefficient, also known as the smoothing factor, whose size will affect the smoothness of the image. ω(i, j) can be expressed as:

[0024]

[0025] d(i,j) represents the Gaussian weighted Euclidean distance between similar windows N(i) and N(j) with standard deviation α, i and j as center points, and window size p:

[0026]

[0027] Since the Gaussian weighted calculation of the 2-normal form in formula (3) takes a long time, the present invention uses an exponential kernel for weight calculation. When the square distance of the similar window is less than 2α 2 When , it is set to 1, while larger distances are rapidly reduced according to the exponential kernel, as shown in formula (4).

[0028]

[0029] In this method, q=21, p=7, and h=25.

[0030] In step 5, the improved CV model is used to extract the ship's contours from the slices processed by FNLM. The CV model is a curve evolution model based on active contours. The CV model uses the image's pixel grayscale information as energy to construct an energy functional, thereby evolving the curve to the target area. This gives it a natural advantage in SAR image contour extraction. The specific steps are as follows:

[0031] For an image (x, y)∈Ω, use I(x, y) to represent the pixel value of the point (x, y) on the image, define the closed curve C as the boundary of a closed area ω on the image, and inside(C) represents the closed area inside the curve Outside (C) represents the area outside the curve C1 and C2 represent the average values ​​of pixels inside and outside the curve on the image, respectively, and define the following energy terms:

[0032] E1(C)+E2(C)=∫ inside(C)|I0(x,y)-C1| 2 dxdy+∫ outside(C) |I0(x,y)-C2| 2 dxdy (5)

[0033] When the curve C is at the target boundary in the target image, the above energy term takes a minimum value. After adding the regularization term, the following energy functional can be obtained:

[0034] E(C)=vlength(C)+μarea(C)+E1(C)+E2(C) (6)

[0035] The level set method expresses the contour of a curve on a two-dimensional plane as a contour curve on a three-dimensional surface through an implicit equation, the level set function. This contour line is called the zero level of the level set function. In contour extraction methods based on the geometric active contour model, the detected image contour is the curve where the zero level set is located when the energy functional reaches its minimum value. First, the level set function φ(x,y) is defined as follows and used to adjust the energy function:

[0036]

[0037] Then, define the unit step function H(z) as follows:

[0038]

[0039] Thus we can get:

[0040]

[0041] The total energy function can be written as:

[0042]

[0043] in:

[0044]

[0045] Since the unit step function H(z) in the CV model does not exist in reality, (12) is usually used instead:

[0046]

[0047] The original step function requires multiple iterations before the curve approaches the target boundary. To increase the curve evolution speed and reduce the number of iterations, the present invention improves equation (12) to a hyperbolic sine function. Finally, c1 and c2 are kept fixed and φ is formally minimized. The gradient descent method is used to parameterize the descent direction by artificial time to obtain the partial differential equation (13) of the Euler-Lagrange equation with respect to φ:

[0048]

[0049] After calculating the gradient descent and reinitializing φ, the final curve can be obtained when the curve gradually evolves to the zero plane.

[0050] Beneficial effects of the present invention:

[0051] The present invention uses high-resolution SAR image data, with backgrounds covering complex scenes such as open sea areas and nearshore coastal areas, and the scenes contain ship targets of various types and sizes. Through transfer learning and large-scale data training, the Faster R-CNN network can achieve higher detection accuracy as a ship detector, while maintaining a faster detection speed. The present invention uses the FNLM filter as a slice data denoising algorithm, which can significantly enhance the foreground target features while homogenizing the background. The present invention uses an improved CV model as a contour extractor. Since the scale of the slice data input is small, multiple iterative operations can be completed quickly, so that the joint model can realize SAR image ship contour extraction at a faster speed, and has practical applicability for batch ship contour extraction of SAR images. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is the SSDD dataset used in the present invention;

[0053] Figure 2 It is the overall flow chart of the present invention;

[0054] Figure 3 : Comparison diagram before and after contour re-annotation; (a) is the original image, (b) is the original annotation, and (c) is the re-annotation using the method of the present invention;

[0055] Figure 4 Schematic diagram of the Faster R-CNN network structure used in the present invention;

[0056] Figure 5 Figure 1 is the ship detection result of the Faster R-CNN network used in this paper; (a) is the detection result of the SAR image nearshore scene, and (b) is the detection result of the SAR image offshore scene;

[0057] Figure 6 : This is the result of ship slice processing using the FNLM filter used in the present invention; (a) is before filtering, and (b) is after filtering;

[0058] Figure 7 The following are the results of the ship contour extraction from SAR images of the present invention; (a) is the result of ship extraction from a nearshore scene in a SAR image; (b) is the result of ship extraction from a far-sea scene in a SAR image;

[0059] Figure 8: This is a binary image of the ship contour extraction result of the SAR image of the present invention; (a) is the ship extraction result of the SAR image nearshore scene; (b) is the ship extraction result of the SAR image offshore scene;

[0060] Figure 9 The figures are a comparison between the SAR image ship outline extraction result and the real outline; (a) is the comparison result of the SAR image nearshore scene detection, and (b) is the comparison result of the SAR image offshore scene detection. DETAILED DESCRIPTION

[0061] The technical solution of the present invention is further explained and illustrated in the following in the form of specific embodiments.

[0062] Example 1:

[0063] The SAR Ship Detection Dataset (SSDD) is the first public SAR image dataset widely used in the field of ship detection. It clearly defines the annotation information and establishes strict usage standards, including the division of training and test data, the reasonable definition of ship size, etc., and supports both target detection tasks and instance segmentation tasks. The data mainly comes from RadarSat-2, TerraSAR-X and Sentinel-1 satellite radar sensors, with resolutions ranging from 1 to 15 meters. A total of 1,160 image slices cover a variety of background environments from offshore to nearshore, including 2,456 ship targets of various sizes and types, such as Figure 1 shown.

[0064] Table 1 lists some basic information of the SSDD dataset. The dataset provides annotations in two formats: MS COCO and PASCAL VOC. Taking the PASCAL VOC format as an example, in the target detection annotation, each ship target is annotated with a boundingbox (BBox), expressed in the form of (xmin, ymin, xmax, ymax), where (xmin, ymin) represents the upper left corner vertex of the annotation box, and (xmax, ymax) is the lower right corner vertex of the annotation box. The average size of the target is approximately pixels. In the instance segmentation task, each ship target is marked with a segm, and the target contour corners are annotated in the form of point-numbers. From the test set of the SSDD dataset, 4 SAR images of nearshore scenes and 4 SAR images of offshore scenes were selected as experimental data, covering ship targets of different sizes, types, and numbers. The data after re-annotating the contours was used as the true contour verification data to verify the accuracy of the SAR image ship contour extraction method of the present invention. Refer to the overall flow chart of this example ( Figure 2 ).

[0065] Table 1

[0066]

[0067] Step 1: SSDD dataset annotation

[0068] Use the Labelme tool to re-label the segm data of the ship outline of the test set of the SSDD dataset and save it in PASCAL VOC format. Figure 3 Comparison images before and after contour relabeling. The SSDD dataset's training set was used as training data for the Faster R-CNN network. After removing eight experimental data from the test set, the SSDD test set data was used as validation data for the network. The network performed object detection, using only the BBox annotations from the SSDD dataset during training.

[0069] Step 2: Faster R-CNN network training:

[0070] This paper uses the Faster R-CNN network as the target detection end of the joint method. The network diagram is shown in Figure 4 . The ResNet50 network is used as the backbone network, the training set of the SSDD dataset is used as the training data of the network, the test set of the SSDD dataset is used as the verification data of the network, the batch_size of the network training is set to 2, the initial learning rate is set to 0.0025, and after 8 epochs of training, the learning rate decays to 0.00025, and a total of 30 epochs of training are performed. During the training process, the foreground and background are distinguished by the Intersect over Union (IoU) between the candidate box and the true value. The foreground is considered to be the ship target to be classified and located, and the background is the area that does not contain the ship target or only contains a small part of the ship target. The present invention uses the candidate box with an IoU threshold greater than 0.7 as a positive sample, and the one less than 0.3 as a negative sample. After training, the final average accuracy can reach more than 80% of the ship detection accuracy, and the detection results are as follows Figure 5 shown.

[0071] Step 3: Feed the test data into the trained network and output the coordinates of the ship:

[0072] Four SAR images of nearshore scenes and four SAR images of offshore scenes were sequentially fed into the trained Faster R-CNN network. The network confidence was set to 0.7, and the coordinate information of the ship area was obtained from the network output. The SAR image of each ship target area was sliced ​​and saved.

[0073] Step 4: grayscale the image slice of the ship area obtained in step 3, and then filter it using the FNLM filter;

[0074] After grayscale processing, the ship slices are sent to the FNLM filter. The search window size of the FNLM filter is set to 21×21, the similarity window size is set to 7×7, and the smoothing factor is set to 25. The filtered ship slices are saved. The results are as follows: Figure 6 .

[0075] Step 5: The data processed by the FNLM filter is fed into the improved CV model to iteratively calculate the target contour. The improved CV model of the present invention sets the initial curve to the 8×8 area in the center of the slice. Iterate the calculation 30 times. The ship slice processed by the FNLM is fed into the improved CV model. After iterative calculation, the ship contour area in the slice is obtained, and all ship contours are fused and displayed in the original SAR image ( Figure 7 ), and set the area surrounded by the ship outline in the slice to 1 and the background area to 0 ( Figure 8 ), which can facilitate the comparison of the accuracy of ship contour extraction using the combined method ( Figure 9 ).

[0076] This method refers to the target detection evaluation method and uses the recall rate R to evaluate the extraction accuracy of the combined method, which is defined as formula (14).

[0077]

[0078] The pixel value in the area contained by the true contour is recorded as 1, and recorded as Ω GT , the pixel values ​​in other areas are recorded as 0, and the detected ship area is recorded as Ω. The ideal value of R is 0. The closer the R value is to 0, the closer it is to Ω GT When R>0, it means that the detected contour is in an expanded state relative to the true contour, and when R<0, it means that the detected contour is in a contracted state relative to the true contour. At the same time, the following four invalid situations are artificially defined ( Figure 6 ):(1) and At this time, R<-0.96; (2) and At this time, R>0.46; (3)Ω∩Ω GT ,and At this time, R<-0.5; (4) At this time, R<-0.5. To facilitate accuracy evaluation, the effective range of R is adjusted to [-0.5, 0.5]. In the nearshore experimental data, the average number of true pixels of ship targets was 677, 1496, 2176, and 1115, respectively, and the average number of detected pixels was 766, 1728, 2741, and 1292. In the offshore experimental data, the average number of true pixels of ship targets was 3253, 5505, 1479, and 2201, respectively, and the average number of detected pixels was 3369, 5512, 1521, and 2688. The experimental results are shown in Table 2.

[0079] Table 2

[0080]

[0081] The results of ship contour extraction from SAR images using the combined method are shown in Figure 7 、 Figure 8 、 Figure 9 , Figure 7 The red curve in the middle is the extracted contour. Figure 9 In the figure, the light-colored area is the extraction result of the combined method, and the white area is the actual target area. The combined use of recall R and manual visual evaluation shows that this method has a good effect in extracting ship contours in SAR images.

[0082] The Faster R-CNN network in step 2 of the present invention is implemented by using the PaddlePaddle platform, and the FNLM filter and the improved CV model use the Findpeaks, numpy, and openCV libraries in the python language. The present invention uses a high-resolution SAR image dataset, and the background covers complex scenes such as open sea areas and nearshore coastal areas, and the scenes contain ship targets of various types and sizes. By training the Faster R-CNN target detection network with transfer learning and large-scale data, a higher ship detection accuracy is obtained, and a faster detection speed is maintained. The present invention uses the FNLM filtering algorithm as a slice data denoising algorithm, which can significantly enhance the foreground target features while homogenizing the background, making it easier for subsequent models to process the data. The present invention uses an improved CV model as a contour extractor, which can effectively reduce the amount of model iteration calculation, accelerate the convergence speed of the ship contour in the target slice, and enable the joint model to achieve rapid extraction of ship contours in SAR images. The present invention uses SSDD data after fine annotation of target contours to verify the ship contour extraction results obtained by the present invention. The combined method of the present invention can be used to quickly detect and extract contours of ship targets in SAR images, and the average ship contour extraction accuracy index R is -0.008.

[0083] Experimental results: In order to illustrate the effectiveness of the SAR image ship contour extraction method using the combined method of Faster R-CNN and the CV model in extracting buildings, Mask R-CNN based on image segmentation extraction was selected for comparison with the method of the present invention. The experimental results show that although the Mask R-CNN method can obtain relatively good results for SAR image ship contour extraction, it still has many problems such as poor contour accuracy, large loss of details, and being greatly affected by data annotation accuracy. The SAR image ship contour extraction method based on the combined method of Faster R-CNN and the CV model of the present invention has high contour extraction accuracy, high detail retention, and is not affected by data annotation. It is conducive to improving the accuracy of sea detection and can provide more information support for maritime sovereignty and security and maritime economic and trade activities.

Claims

1. A ship contour extraction method from SAR images based on the combined method of Faster R-CNN and CV model, characterized by: The steps of this method are as follows: Step 1: Select the SAR image ship target detection dataset as the detection network training data of this method, and relabel the ship outline; Step 2: Build the Faster R-CNN target detection network and feed the data set selected in step 1 into the network model for training to obtain the trained Faster R-CNN. Step 3: Feed the test data into the trained Faster R-CNN network, output the ship coordinate information and slice the ship area image; Step 4: grayscale the ship area image slice obtained in step 3, and then filter it using a fast non-local means filter; Step 5: The data processed by the fast non-local mean filter is fed into the improved CV model to iteratively calculate the target contour, and the contour information is fused and displayed in the original image; In step 5, the improved CV model is used to extract the ship contour from the slices processed by fast non-local mean filtering. The improved CV model is as follows: For an image (x, y)∈Ω, use I(x, y) to represent the pixel value of the point (x, y) on the image, define the closed curve C as the boundary of a closed area ω on the image, and inside(C) represents the closed area inside the curve Outside (C) represents the area outside the curve C1 and C2 represent the average values ​​of pixels inside and outside the curve on the image, respectively, and define the following energy terms: E1(C)+E2(C)=∫ inside(C) |I0(x,y)-C1| 2 dxdy+∫ outside(C) |I0(x,y)-C2| 2 dxdy (5) When the curve C is at the target boundary in the target image, the above energy term takes a minimum value. After adding the regularization term, the following energy functional can be obtained: E(C)=vlength(C)+μarea(C)+E1(C)+E2(C) (6) The level set method expresses the contour of a curve on a two-dimensional plane as a contour curve of a three-dimensional surface through an implicit equation, namely the level set function. This contour line is called the zero level of the level set function. In the contour extraction method based on the geometric active contour model, the detected image contour is the curve where the zero level set is located when the energy functional reaches its minimum value. First, the level set function φ(x,y) is defined as follows and used to adjust the energy function: Then, define the unit step function H(z) as follows: Thus we can get: The total energy function can be written as: in: Since the unit step function H(z) in the CV model does not exist in reality, (12) is usually used instead: Before the curve approaches the target boundary, the original step function needs multiple iterations. In order to increase the evolution speed of the curve and reduce the number of iterations, Equation (12) is improved to a hyperbolic sine function. Finally, c1 and c2 are kept fixed and φ is formally minimized. The gradient descent method is used to parameterize the descent direction by artificial time to obtain the partial differential equation (13) of the Euler-Lagrange equation with respect to φ: After calculating the gradient descent and reinitializing φ, the final curve can be obtained when the curve gradually evolves to the zero plane.

2. The method for extracting ship contours from SAR images based on the Faster R-CNN and CV model combination method according to claim 1, characterized in that: In step 2, the Faster R-CNN network uses ResNet50 as the backbone network. The ResNet50 backbone network contains 1 input segment, 4 subsequent segments, and 1 output layer. (a) The input segment consists of a convolutional layer and a maximum pooling layer. The parameters of the convolutional layer are: kernel size: 7×7, strides: 2; channels: 64; the parameters of the maximum pooling layer are: kernel size: 3×3, strides: 2; (b) The four subsequent segments contain 3, 4, 6, and 3 residual blocks respectively. Each residual block contains two branches, A and B. Branch A consists of three convolutional layers. The parameters of the first convolutional layer are: convolution kernel size kernel_size: 1×1; the parameters of the second convolutional layer are: convolution kernel size kernel_size: 3×3, strides: 2; the parameters of the third convolutional layer are: convolution kernel size kernel_size: 1×1; branch B consists of one convolutional layer, and the parameters of the convolutional layer are: convolution kernel size kernel_size: 1×1, strides:

2. (c) The output layer consists of an average pooling layer and a fully connected layer.

3. The method for extracting ship contours from SAR images based on the combined method of Faster R-CNN and CV model according to claim 2, characterized in that: In step 2, the Faster R-CNN network uses an RPN structure, which includes two branches: classification and regression. First, the RPN network performs a 3×3 convolution on the feature map output by the backbone network and then feeds it into the classification branch and the regression branch. The classification branch consists of 18 1×1 convolutions; the regression branch consists of 36 1×1 convolutions. Both branches simultaneously input the target detection layer proposal, and finally output the ship coordinate points and slice the ship area image.

4. The method for extracting ship contours from SAR images based on the combined method of Faster R-CNN and CV model according to claim 1, characterized in that: The method of using the fast non-local means filter for filtering in step 4 is as follows: Assume that a discrete noise image is g(i) = g{g(i)|i∈Ω}, whose size is N×N, Ω represents the image neighborhood, and i is the pixel index. After the noise image is processed by NLM filtering, it can be normalized and expressed as: Ω i represents a search window with a center pixel i and a size of q×q; ω(i, j) represents the weight assigned to the noise image g(i), where h represents the control exponential decay filter coefficient, also known as the smoothing factor, whose size will affect the smoothness of the image. ω(i, j) can be expressed as: d(i,j) represents the Gaussian weighted Euclidean distance between similar windows N(i) and N(j) with a standard deviation of α, i and j as center points, and a window size of p, as shown in formula (3): Since the Gaussian weighted calculation of the 2-norm in formula (3) takes a long time, the exponential kernel is used for weight calculation. When the square distance of the similar window is less than 2α 2 When , it is set to 1, while larger distances are rapidly reduced according to the exponential kernel, as shown in formula (4), In this method, q=21, p=7, and h=25.

Images