A method and device for detecting ship targets in SAR images and building a model thereof

The pre-selected boxes with good adaptability were determined through K-Means++ clustering and genetic algorithm, and the improved VGG16 network optimized the SAR image ship object detection model, solving the problems of low detection accuracy and susceptibility to background interference in the prior art, realizing the High precision and adaptability of ship target detection.

CN117523390BActive Publication Date: 2024-11-19PLA PEOPLES LIBERATION ARMY OF CHINA STRATEGIC SUPPORT FORCE AEROSPACE ENG UNIV
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202311471109.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-11-07
Publication Date
2024-11-19
Estimated Expiration
2043-11-07

AI Technical Summary

Technical Problem

The existing SAR image ship target detection method based on deep learning rarely takes into account the electromagnetic scattering characteristics and dimension shape information of ship targets, resulting in low detection accuracy, susceptibility to background interference, and poor inspection frame wrapping.

Method used

The real boxes of the target ship are clustered and slightly varied by K-Means++ clustering algorithm and genetic algorithm to obtain pre-selected boxes with good adaptability, and feature extraction and object detection models are performed for Faster R-CNN based on the improved VGG16 network. optimization.

Benefits of technology

The detection accuracy and adaptability of the target detection model are improved, missed detection and false alarms in the detection results are reduced, and the accuracy and adaptability of the detection results are improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117523390B_ABST
    Figure CN117523390B_ABST
Patent Text Reader

Abstract

The disclosed embodiment proposes a method and device for constructing a SAR image ship target detection model, the method comprising: obtaining a SAR image training data set, clustering the real frame of the target ship in the annotation information using the K-Means++ clustering algorithm to obtain K cluster pre-selected frames of different sizes, and using a genetic algorithm to slightly mutate the K cluster pre-selected frames of different sizes to obtain K pre-selected frames of different sizes; according to the pre-selected frames, optimizing the target detection model to obtain the final target detection model, the target detection model is obtained based on the Faster R-CNN network training, and the feature extraction network of the Faster R-CNN network adopts the improved VGG16 network. The disclosed method determines the pre-selected frame through an adaptive pre-selected frame mechanism, uses the improved VGG16 network as the feature extraction network, optimizes the network structure, and improves the accuracy and adaptability of the detection result.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The disclosed embodiments relate to the field of image processing technology, and in particular to the construction of a SAR image ship target detection model, and a SAR image ship target detection method and device. Background Art

[0002] Real-time monitoring and identification of marine ship targets is an important part of marine target surveillance and is of great significance to economic and national defense security. Synthetic Aperture Radar (SAR) uses active microwave imaging and has the unique advantage of all-day and all-weather. It can reduce the impact of harsh environments such as night, clouds, rain, and fog to a certain extent, and is an important means to achieve marine ship target surveillance.

[0003] In recent decades, SAR data acquisition equipment and imaging technology have achieved rapid development. The improvement of image resolution and the acquisition of a large amount of remote sensing data have provided data guarantee conditions for SAR ship target detection. However, at the same time, with the emergence of a large amount of high-quality SAR image data and the differences between SAR images and traditional optical images in imaging mechanism, radiation characteristics, geometric characteristics, etc., traditional feature-based ship target detection methods are stretched.

[0004] The wide application of deep learning technology in target detection and recognition has provided new ideas for SAR ship target detection. Many researchers have proposed a variety of detection algorithms based on deep learning for SAR ship target detection. The core idea is to use mainstream networks for feature extraction and target detection, but rarely consider the electromagnetic scattering characteristics and size and shape of ship targets in SAR images. The detection results have problems such as poor detection frame wrapping, low detection accuracy, and susceptibility to background interference. Summary of the invention

[0005] The purpose of the embodiments of the present disclosure is to provide a method and device for constructing a SAR image ship target detection model, a SAR image ship target detection method, device, electronic device and computer-readable medium, so as to solve the above-mentioned problems existing in the prior art.

[0006] In order to achieve the above objectives, the technical solutions adopted by the embodiments of the present disclosure are as follows:

[0007] The present disclosure provides a method for constructing a SAR image ship target detection model in a first aspect. The method includes:

[0008] Acquire a SAR image training data set, wherein the SAR image training data set includes annotation information of a target ship in each image, and the annotation information includes a position and a size of the target ship, that is, a real frame of the target ship;

[0009] A K-Means++ clustering algorithm is used to cluster the real frames of the target ship in the annotation information to obtain K cluster pre-selected frames of different sizes, and a genetic algorithm is used to slightly mutate the K cluster pre-selected frames of different sizes within a range of 1 to obtain K pre-selected frames of different sizes;

[0010] According to the pre-selected box, the target detection model is optimized to obtain a final target detection model, wherein the target detection model is obtained based on Faster R-CNN network training, and the feature extraction network of the Faster R-CNN network adopts an improved VGG16 network.

[0011] Exemplarily, the method of using a genetic algorithm to perform a small variation on K cluster pre-selection boxes of different sizes within a range of 1 to screen out K target pre-selection boxes of different sizes includes:

[0012] Overlapping the center point of the real frame of the target ship with the center point of the cluster pre-selected frame to which it belongs, and calculating the intersection-over-union ratio of the real frame of the target ship and the cluster pre-selected frame;

[0013] Traverse all the real boxes of the target ships and the pre-selected boxes of the clusters to which they belong, and calculate the average value of the intersection-over-union ratio Avg IOU of the two. The calculation formula is as follows:

[0014]

[0015] Among them, N represents the number of real boxes of the target ship, K represents the number of cluster pre-selected boxes, and x i represents the true box of the i-th target ship, c i Represents the i-th cluster pre-selection box;

[0016] The genetic algorithm is used to mutate K cluster pre-selected boxes of different sizes in the range of 1. The mutation results are used as new cluster pre-selected boxes. The average intersection-over-union (IOU) of the real boxes of all target ships and their new cluster pre-selected boxes is recalculated. If Avg IOU increases, the cluster pre-selected box is updated to the mutation result corresponding to the increase in Avg IOU. After several iterations, the mutation result corresponding to the increase in Avg IOU is obtained as the final pre-selected box.

[0017] Exemplarily, the method further includes: determining the K value according to a curve showing the change of Avg IOU with K.

[0018] Exemplarily, the improved VGG16 network includes: 7 convolutional layers, each convolutional layer is followed by a BN layer, 2 pooling layers and 4 dilated convolutional layers.

[0019] Exemplarily, after obtaining the SAR image training data set, the method further includes:

[0020] Traversing all the annotated files in the SAR image training data set, and reading the size information of the images in the annotated files;

[0021] Resize images to preset pixel dimensions;

[0022] The ground-truth boxes of target ships whose pixel values ​​are less than the preset threshold are removed.

[0023] A second aspect of the embodiments of the present disclosure provides a device for constructing a SAR image ship target detection model, the device comprising:

[0024] An acquisition module is used to acquire a SAR image training data set, wherein the SAR image training data set includes annotation information of a target ship in each image, and the annotation information includes a position and a size of the target ship, that is, a real frame of the target ship;

[0025] A pre-selection box determination module is used to cluster the real boxes of the target ship in the annotation information using a K-Means++ clustering algorithm to obtain K cluster pre-selection boxes of different sizes, and to perform a small variation on the K cluster pre-selection boxes of different sizes within a range of 1 using a genetic algorithm to obtain K pre-selection boxes of different sizes;

[0026] The optimization module is used to optimize the target detection model according to the pre-selected box to obtain a final target detection model.

[0027] A third aspect of the embodiments of the present disclosure provides a SAR image ship target detection method, the method comprising:

[0028] Acquire a SAR image, and preprocess the SAR image;

[0029] The preprocessed SAR image is input into the target detection model obtained according to the method described above to obtain the ship in the SAR image.

[0030] A fourth aspect of the embodiments of the present disclosure provides a SAR image ship target detection device, the device comprising:

[0031] An acquisition module, used for acquiring SAR images and preprocessing the SAR images;

[0032] The detection module is used to input the preprocessed SAR image into the target detection model obtained according to the method of claim 1 to obtain the ship in the SAR image.

[0033] A fifth aspect of the embodiments of the present disclosure provides an electronic device, characterized by comprising:

[0034] at least one processor; and,

[0035] a memory communicatively connected to the at least one processor; wherein,

[0036] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method described in the first invention and the third aspect of the present disclosure.

[0037] A sixth aspect of an embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the method described in the first invention and the third aspect of the present disclosure is implemented.

[0038] The beneficial effects of the embodiments of the present disclosure are:

[0039] The disclosed embodiment firstly targets a given SAR image ship target detection training data set image, and determines a pre-selection box suitable for the ship target by designing an adaptive pre-selection box mechanism, thereby improving the detection accuracy of the target detection model; and uses a feature extraction network based on an improved VGG16 network to make corresponding adjustments to the network structure and optimize the network structure, so that the detection result has the advantages of high detection accuracy and good adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a flowchart of a method for constructing a SAR image ship target detection model according to an embodiment of the present disclosure;

[0041] Figure 2 It is a structural schematic diagram of a device for constructing a SAR image ship target detection model according to an embodiment of the present disclosure;

[0042] Figure 3 is a flowchart of a SAR image ship target detection method according to some embodiments of the present disclosure;

[0043] Figure 4 is a structural schematic diagram of a SAR image ship target detection device according to some embodiments of the present disclosure;

[0044] Figure 5 is a curve diagram showing the change of Avg IOU with K value in the present disclosure;

[0045] Figure 6 Schematic diagram of the VGG16 network structure used in the SAR image ship target detection model of the present disclosure;

[0046] Figure 7 is a comparison diagram of the original pre-selected box and the pre-selected box adjusted by the construction method of the SAR image ship target detection model disclosed in the present invention;

[0047] Figure 8 It is a schematic diagram comparing the detection results of small ship targets near islands using the disclosed SAR image ship target detection model and other detection models; Figure 8 (a) The four leaves are the original annotation diagram of the ship target in the SAR image. Figure 8 (b) is the detection effect diagram using the VGG network. The larger box represents the missed target. Figure 8 (c) shows the detection effect of ResNet network. The larger box represents the missed target. Figure 8 (d) is the detection effect of the VGG network used in this disclosure. The ellipse represents the false alarm target.

[0048] Fig. 9 It is a schematic diagram comparing the detection results of nearshore and large ship targets using the disclosed SAR image ship target detection model and other detection models; Fig. 9 (a) is a schematic diagram of the original annotation of ship targets in SAR images. Fig. 9 (b) is a schematic diagram of the detection effect using the VGG network. Fig. 9 (c) is a schematic diagram of the detection effect using the ResNet network. Fig. 9 (d) is a schematic diagram of the detection effect of the VGG network using the present disclosure, wherein the thicker box represents the missed detection target and the oval box represents the false alarm target;

[0049] Fig.10 It is a schematic diagram comparing the detection results of small ships far from the coast using the disclosed SAR image ship target detection model and other detection models; Fig.10 (a) is a schematic diagram of the original annotation of ship targets in SAR images. Fig.10 (b) is a schematic diagram of the detection effect using the VGG network. Fig.10 (c) is a schematic diagram of the detection effect using the ResNet network. Fig.10 (d) is a schematic diagram of the detection effect of the VGG network using the present disclosure, wherein the larger box represents the missed detection target and the elliptical box represents the false alarm target;

[0050] Figure 11 is a comparison diagram of the PR curves of FasterR-CNN using the VGG16 network and the original VGG16 network in the present disclosure; Figure 11(a) is a schematic diagram of the PR curve of FasterR-CNN using the improved original VGG16 network, and Figure 11(b) is a schematic diagram of the PR curve of FasterR-CNN using the original VGG16 network in the present disclosure. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solution and advantages of the embodiment of the present disclosure more clear, the embodiment of the present disclosure is further described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation methods described here are only used to explain the embodiment of the present disclosure and are not used to limit the embodiment of the present disclosure.

[0052] like Figure 1 As shown, a method for constructing a SAR image ship target detection model is provided in one aspect of an embodiment of the present disclosure, and the method includes:

[0053] Step 1: Obtain a SAR image training data set, wherein the SAR image training data set includes the position and gray value information of each pixel in each image and the annotation information of the target ship, etc. The annotation information includes the position and size of the target ship, that is, the real frame of the target ship.

[0054] In the disclosed embodiment, the original SAR image training data set is obtained from the SAR ship detection data set SSDD (SARShip Detection Dataset, SSDD). The SAR image ship target detection training data set includes the position and gray value information of each pixel point of each image and the annotation information of the target ship, such as the gray value of the pixel point x, y of the i-th image is f(x, y, i) and the annotation information annotation(i) of each target ship in the i-th image, wherein: the i-th image f(x, y, i), x, y represents the pixel coordinates in the image, and f(x, y, i) represents the pixel gray value of the i-th image at the coordinates x, y. The annotation file of the i-th image includes the file name of the image, the length and width pixel size of the image, the number of channels, and the annotation information annotation(i) of each target ship in the image. The annotation information annotation(i) includes: the name of the target ship, the position coordinates of the target ship annotation box, the length height and width width information of the target ship, and the annotation box is the real box. The size of the target ship refers to the length height and width width information of the target ship. Each image may include multiple target ship annotation information, that is, there are multiple target ship real frames.

[0055] Exemplarily, after obtaining the SAR image training data set, the method further includes:

[0056] All the annotation files in the SAR image training data set are traversed, and the size information of the images in the annotation files is read.

[0057] Resize the image to a preset pixel size;

[0058] The ground-truth boxes of target ships whose pixel values ​​are less than the preset threshold are removed.

[0059] Here, the reading of the image size information in the annotation file means reading the length and width of each image, and adjusting the image size to a preset pixel size; the original image has various sizes, and in order to unify the image size of the input target detection model, the original image is uniformly scaled to a preset pixel size of 600×600, and the corresponding target ship real frame in the image is scaled at the same ratio. The real frame with a pixel value less than a preset threshold is removed. A pixel value that is too small indicates that the target is small and is not a target of interest, so it can be removed. The preset threshold can be pre-set by the user based on empirical data, or it can be a preset threshold obtained by adjusting the set threshold according to actual needs.

[0060] Step 2: Use the K-Means++ clustering algorithm to cluster the real boxes of the target ship in the annotation information to obtain K cluster pre-selected boxes of different sizes, and use the genetic algorithm to slightly mutate the K cluster pre-selected boxes of different sizes within the range of 1 to obtain K target pre-selected boxes of different sizes.

[0061] The K-Means++ clustering algorithm used in the disclosed embodiment can also cluster the length and width of the target ship in the remaining annotation information after the real frame of the target ship with too small pixel value is removed after the size is adjusted. According to the characteristics of the training data set, the geometric shape and other information of the target ship are fully utilized. By setting an adaptive pre-selection box mechanism, a pre-selection box suitable for the ship target in the data set is determined, thereby improving the impact of the manually designed preset anchor point box on the detection result due to the mismatch between the target and the preset anchor point box.

[0062] As an example of screening target pre-selection boxes, the genetic algorithm is used to perform a small variation on K cluster pre-selection boxes of different sizes within a range of 1, and K target pre-selection boxes of different sizes are screened, including:

[0063] Step 21: Place the center point of the real frame of the target ship and the center point of the cluster pre-selected frame to which it belongs at the same point, and calculate the intersection-over-union ratio of the real frame of the target ship and the cluster pre-selected frame.

[0064]

[0065] Among them, c iis the pre-selected box for the i-th cluster; x i is the real box of the i target ship. Generally, the real box and the pre-selected box are rectangular boxes.

[0066] Step 22: traverse all the real boxes of the target ship and the cluster pre-selected boxes to which they belong, and calculate the average value of the intersection-over-union ratio Avg IOU. Assuming that there are N real boxes of the target ship and K cluster pre-selected boxes, use x i represents the true box of the i-th target ship, c i Represents the i-th cluster pre-selection box, and the calculation formula is as follows:

[0067]

[0068] Step 23, use the genetic algorithm to perform a small mutation on K cluster pre-selected boxes of different sizes within the range of 1, use the mutation result as the new cluster pre-selected box, recalculate the average value of the intersection and union of the real boxes of all target ships and the new cluster pre-selected boxes to which they belong, Avg IOU. If Avg IOU increases, the cluster pre-selected box is updated to the mutation result corresponding to the increase in Avg IOU. After several mutation iterations, the mutation result corresponding to the increase in Avg IOU is obtained as the final pre-selected box. If AvgIOU decreases, that is, the effect becomes worse, the cluster pre-selected box is not updated.

[0069] Here, the increase of Avg IOU means that the Avg IOU calculated by taking the mutation result as the new cluster pre-selection box is larger than the Avg IOU calculated by taking the cluster pre-selection box before the mutation or the cluster pre-selection box obtained by the last mutation. For each mutation, if Avg IOU becomes smaller, that is, the effect becomes worse, the box is not updated. If Avg IOU becomes larger, that is, the effect becomes better, the mutated box is kept for update, and then the next mutation is performed. Each iteration Avg IOU will not decrease, but always grow positively. When the iteration stops after 1000 times, the mutation result corresponding to the increase of Avg IOU is still selected as the final pre-selection box. Each mutation result corresponds to the scale of the cluster pre-selection box. The overall indicator is reflected in the increase of Avg IOU and the better the wrapping effect of the pre-selection box. The larger the value of Avg IOU, the greater the overlap of the cluster pre-selection box with the real box of all target ships, and the better the clustering effect. It can be iterated 1000 times here.

[0070] In the genetic algorithm, assume that the population size is M, and a single individual in the population is the length and width of the pre-selected box. The fitness function is F(·), and the individual c i The probability of being selected is:

[0071]

[0072] The fitness function F(·) is a function of AvgIoU (average intersection over union). The fitness function F() represents the length and width of a pre-selected box (corresponding to an individual c i ) is suitable, and the probability is calculated by the fitness according to the formula P. The fitness function is a function in the genetic algorithm that measures the fitness of individuals to the environment, so as to facilitate the survival of the fittest in the population. The purpose of the genetic algorithm is to find individuals with higher scores obtained using the fitness function.

[0073] It should be noted that the number of cluster pre-selection boxes cannot be increased indefinitely. Assuming that the value range of K is 1 to 30, K is clustered one by one from 1 to 30 to obtain the corresponding number of cluster pre-selection boxes, and the size of the cluster pre-selection box, that is, the length and width, is mutated by the above method, and the maximum value of the Avg IOU increase of each number of cluster pre-selection boxes is compared, and the optimal result is selected after comprehensive consideration. Specifically, each mutation of the K cluster pre-selection boxes will produce K mutation results with small changes. If the Avg IOU increases, the cluster pre-selection box after the mutation is used. If the Avg IOU decreases, the cluster pre-selection box before the mutation is maintained. In this way, each iteration will gradually increase the Avg IOU. After multiple iterations, the K cluster pre-selection boxes corresponding to the mutation results are used as the final result of the size of the cluster pre-selection box.

[0074] Exemplarily, the method further includes: determining the K value according to a curve showing that Avg IOU varies with the K value.

[0075] In the embodiment of the present disclosure, K values ​​are calculated one by one from 1 to 30, and the curve of Avg IOU changing with K value is analyzed, as shown in FIG. Figure 5 As shown, the larger the K value is, the larger the AvgIoU is, the greater the computational burden is, and the curve of Avg IOU changing with K value rises more gently after K≥12. Considering the computer performance burden and the overall detection speed, the number of pre-selected boxes cannot be increased indefinitely, so K≤12 is selected, and K=12 is preferred.

[0076] like Figure 7 As shown, in the embodiment of the present disclosure, a genetic algorithm is used to change the length and width of the cluster pre-selection box in a small range, with the purpose of making the pre-selection box more accurate and having a certain degree of generalization and diversity.

[0077] The disclosed embodiment uses the average intersection-and-union ratio of the real frames of all target ships and the clustered pre-selected frames to which they belong to measure the improvement performance of the K-Means++ algorithm on the pre-selected frames. According to the clustering effect, taking into account the computer performance burden, the overall detection speed, the target detection model training speed and other issues, the number of pre-selected frames cannot be increased indefinitely, and K-Means++ and genetic algorithms are used for clustering, and K is determined to be 12. Thus, the number of pre-selected frames that adapt to the data set and the aspect ratio of the pre-selected frames are obtained through the clustering method. The disclosed embodiment uses the K-Means++ clustering algorithm and the genetic algorithm to obtain the number of pre-selected frames that adapt to the data set and the aspect ratio of the pre-selected frames. The height of the pre-selected frame is height1 i and width1 i .

[0078] Step 3: According to the pre-selected box, the target detection model is optimized to obtain a final target detection model, wherein the target detection model is trained based on the Faster R-CNN network, and the feature extraction network of the Faster R-CNN network adopts an improved VGG16 network.

[0079] The pre-selected box in the disclosed embodiment is used for the number, size, aspect ratio, etc. of anchor boxes in the target detection model, thereby improving the influence of the mismatch between the manually designed preset anchor box and the target on the detection result; according to the electromagnetic scattering characteristics of ship targets in SAR images, a feature extraction network based on the improved VGG16 network is used to optimize the network structure; so that the detection results have the advantages of high detection accuracy and good adaptability.

[0080] Here, the improved VGG16 network includes: 7 convolutional layers, BN is performed after each convolutional layer, 2 pooling layers and 4 void convolutional layers.

[0081] The original VGG16 network contains 13 convolutional layers, 5 pooling layers and 3 fully connected layers. The convolution kernel size is 3×3. For the input image, two convolution operations with a convolution kernel size of 64 are first performed, followed by a maximum pooling, and then two convolution operations with a convolution kernel size of 128 followed by maximum pooling; after repeating the three-layer convolution and pooling operations three times, it enters the fully connected layer and finally performs classification.

[0082] In the Faster R-CNN algorithm framework, VGG16 is used as the basic feature extraction network. According to the characteristics of SAR ship targets, the embodiment of the present disclosure adjusts the original VGG16 network structure: the last three pooling layers of the original VGG16 network are deleted to prevent the loss of information in the pooling process; in order to solve the problem of reduced receptive field caused by reducing the pooling layer, four dilated_conv are added at the end of the network; in order to alleviate the overfitting that may occur during the training process, the BN (BatchNormalization) method is added. The specific network structure of the improved VGG16 is shown in Figure 6 ,The scale information of each layer of the improved VGG16 network is shown in Table 1.

[0083] Table 1 Scale information of each layer of the improved VGG16 network

[0084]

[0085]

[0086] The improved VGG16 network is used as the basic feature extraction network of the Faster R-CNN algorithm, and the training data set is used to learn and train Faster R-CNN. Finally, the trained Faster R-CNN network is used to detect ship targets. The results show that compared with the Faster R-CNN algorithm using the original VGG16 network, the overall average accuracy of the COCO evaluation index in the disclosed embodiment is improved by 1.96%. Regardless of large, medium or small ship targets, the improved method has better detection effects and has the advantage of high detection accuracy.

[0087] like Figure 2 As shown, another aspect of the embodiment of the present disclosure provides a device for constructing a SAR image ship target detection model, the device comprising:

[0088] The acquisition module 110 is used to acquire a SAR image training data set, wherein the SAR image training data set includes annotation information of a target ship in each image, and the annotation information includes the position and size of the target ship, that is, a real frame of the target ship.

[0089] The pre-selection box determination module 120 is used to cluster the real boxes of the target ship in the annotation information using the K-Means++ clustering algorithm to obtain K cluster pre-selection boxes of different sizes, and to perform a small variation on the K cluster pre-selection boxes of different sizes within a range of 1 using a genetic algorithm to obtain K pre-selection boxes of different sizes. The number of pre-selection boxes and the aspect ratio, i.e., height and width, are determined by the pre-selection box determination target detection model.

[0090] The optimization module 130 is used to optimize the target detection model according to the pre-selected box to obtain a final target detection model.

[0091] like Figure 3 As shown, another aspect of the embodiment of the present disclosure provides a SAR image ship target detection method, the method comprising:

[0092] Acquire a SAR image, and preprocess the SAR image;

[0093] The preprocessed SAR image is input into the target detection model obtained according to the method described above to obtain the ship in the SAR image.

[0094] When detecting ship targets in SAR images in the disclosed embodiment, the SAR image is preprocessed, and the SAR image size is adjusted to a preset pixel size of 600×600, and then input into the target detection model; the improved VGG16 performs feature extraction on the image to be detected, and the feature map is extracted and sent to RPN (Region Proposal Network), and after a 512-dimensional 3×3 convolution, the pre-selected box target background judgment and coordinate prediction are performed respectively. For the judgment of the target background, the pre-trained region proposal network RPN is used. After the region proposal network RPN is trained, further detection is performed. During the detection process, the pre-selected box and the groundtruth intersection over Union (IoU) greater than 0.7 are set as targets, less than 0.3 as backgrounds, and IoU between 0.3-0.7 are not involved in training. The loss function for judgment is as follows:

[0095]

[0096] Among them, L is the total loss function; L cls represents the classification loss function, that is, the loss of the target or non-target; p i represents the probability that the i-th bounding box is the foreground; p i * When the i-th border is a positive sample, the value is 1, and when it is a negative sample, the value is 0; N cls is the normalized weight of the classification loss; λ is the weight adjustment coefficient; N reg is the normalized weight of regression loss; L reg is the regression loss function; t i The coordinates of the prediction box; t * i The coordinates of the real box. This part belongs to the network training process and will not be described here.

[0097] Finally, the pre-selected box target background classification results and the coordinate predicted transformation are sent to the Proposal layer to correct the pre-selected box position.

[0098] The pre-selected box is input into RoIPooling and scaled to the classic size of 7*7. Classification and bounding box regression are completed in the classification layer. I will not go into details here.

[0099] By adopting the above technical solution disclosed in the embodiment of the present disclosure, the following beneficial effects are obtained: according to the characteristics of the training data set, the geometric shape of the ship and other information are fully utilized. Through the designed adaptive pre-selection box mechanism, the pre-selection box adapted to the data set is determined, and the impact of the mismatch between the manually designed preset anchor point box and the target on the detection result is improved. According to the electromagnetic scattering characteristics of the ship target in the SAR image, the feature extraction network based on the VGG16 network is adjusted accordingly by adding batch normalization, introducing void convolution, and making corresponding adjustments to the network structure. The detection result has the advantages of high detection accuracy and good adaptability.

[0100] like Figure 4 As shown, in some embodiments of the present disclosure, a SAR image ship target detection device includes:

[0101] An acquisition module, used for acquiring SAR images and preprocessing the SAR images;

[0102] The detection module is used to input the preprocessed SAR image into the target detection model obtained according to the above method to obtain the ship in the SAR image.

[0103] An embodiment of the present disclosure provides an electronic device, including:

[0104] at least one processor; and,

[0105] a memory communicatively connected to the at least one processor; wherein,

[0106] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for constructing a SAR image ship target detection model or the SAR image ship target detection method as described above.

[0107] One embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for constructing a SAR image ship target detection model or the method for detecting a SAR image ship target is implemented as described above.

[0108] The disclosed embodiment adopts an improved VGG16 network as the basic feature extraction network of the Faster R-CNN algorithm. Compared with the original VGG16 network and the ResNet network, the improved VGG16 network has the advantages of better SAR image ship target detection effect and higher detection accuracy.

[0109] like Figure 8 As shown in the figure, the detection effect of small ship targets near the island is demonstrated. There are 24 ship targets in the original image. For some very small ships, both the VGG16 network and the ResNet network have a large number of missed detections. The ResNet network missed 8 ships, with a missed detection rate of 33.3%, and the VGG16 network missed 45.8% of the ships. Although the VGG16 network improved by the present disclosure has a false alarm, the detection rate is as high as 95.8%. Some extremely small ships can also be detected.

[0110] like Fig. 9 As shown in the figure, in the case of nearshore, the three network models all have different degrees of false alarms. ResNet has the lowest false alarm rate, with only 1 false alarm. The envelopment of the pre-selection box is not as good as that of the other two networks. The sizes of the two ships are obviously beyond the range of the pre-selection box. The original VGG16 network has 2 false alarms, and the improved VGG16 network of the present invention has 3 false alarms. The performance is poor when there are more background clutter near the shore.

[0111] like Fig.10 As shown in FIG. 1 , a large number of small ship target detection situations are shown in FIG. 1 . By comparison, it is not difficult to see that in the case of far-shore conditions and less background clutter interference, the detection effect of the improved VGG16 network disclosed in the present invention is significantly better than the previous two.

[0112] In general, using the improved VGG16 as the feature extraction network has better detection effects in situations such as far from the shore and when the target size is small, except for slightly worse performance near the port.

[0113] As shown in Figure 11, through the comparison of the PR curve, it is found that although the improved VGG16 network has a lower accuracy when the IoU value is low, the accuracy does not drop sharply when the IoU requirement is high, indicating that the use of the improved VGG16 network as the feature extraction network of the Faster R-CNN algorithm has higher pre-selection box positioning accuracy and stronger envelopment.

[0114] In general, although the improved VGG16 network still has some shortcomings in near-shore conditions, it has better performance in other measurement indicators, and the overall average accuracy has also improved significantly. Using the improved VGG16 network as the basic network of the Faster R-CNN algorithm is more suitable for SAR image ship target detection.

[0115] The above is only a preferred implementation of the embodiment of the present disclosure. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the embodiment of the present disclosure. These improvements and modifications should also be considered as the protection scope of the embodiment of the present disclosure.

Claims

1. A method for constructing a ship target detection model for SAR images. It is characterized in that The method comprises: Acquire a SAR image training data set, wherein the SAR image training data set includes annotation information of a target ship in each image, and the annotation information includes a position and a size of the target ship, that is, a real frame of the target ship; A K-Means++ clustering algorithm is used to cluster the real frames of the target ship in the annotation information to obtain K cluster pre-selected frames of different sizes, and a genetic algorithm is used to slightly mutate the K cluster pre-selected frames of different sizes within a range of 1 to obtain K pre-selected frames of different sizes; The genetic algorithm is used to perform a small variation on K cluster pre-selection boxes of different sizes within a range of 1, and K target pre-selection boxes of different sizes are screened, including: Overlapping the center point of the real frame of the target ship with the center point of the cluster pre-selected frame to which it belongs, and calculating the intersection-over-union ratio of the real frame of the target ship and the cluster pre-selected frame; Traverse all the real boxes of the target ships and the pre-selected boxes of the clusters to which they belong, and calculate the average value of the intersection-over-union ratio Avg IOU of the two. The calculation formula is as follows: Among them, N represents the number of real boxes of the target ship, K represents the number of cluster pre-selected boxes, and x i represents the true box of the i-th target ship, c j represents the jth cluster pre-selection box; The genetic algorithm is used to perform small mutations on K cluster pre-selected boxes of different sizes within the range of 1, and the mutation results are used as new cluster pre-selected boxes. The average value of the intersection and union (Avg IOU) of the real boxes of all target ships and their new cluster pre-selected boxes is recalculated. If Avg IOU increases, the cluster pre-selected box is updated to the mutation result corresponding to the increase in Avg IOU. After several iterations, the mutation result corresponding to the increase in Avg IOU is obtained as the final pre-selected box. According to the curve of Avg IOU changing with K, determine the K value, K value ≤ 12; According to the pre-selected box, the target detection model is optimized to obtain a final target detection model, wherein the target detection model is trained based on the Faster R-CNN network, and the feature extraction network of the Faster R-CNN network adopts an improved VGG16 network; the improved VGG16 network includes: retaining the first 7 convolutional layers and the first 2 pooling layers of the original VGG16 network, adding a BN layer after each convolutional layer, deleting the last 6 convolutional layers of the original VGG16 network, and replacing the last three pooling layers of the original VGG16 network with 4 hole convolutional layers; the number of holes in the 4 hole convolutional layers increases successively by 2, 3, 4 and 5 respectively.

2. The method according to claim 1, It is characterized in that After obtaining the SAR image training data set, the method further includes: Traversing all the annotated files in the SAR image training data set, and reading the size information of the images in the annotated files; Resize images to preset pixel dimensions; The ground-truth boxes of target ships whose pixel values ​​are less than the preset threshold are removed.

3. A device for constructing a SAR image ship target detection model, It is characterized in that The device comprises: An acquisition module is used to acquire a SAR image training data set, wherein the SAR image training data set includes annotation information of a target ship in each image, and the annotation information includes a position and a size of the target ship, that is, a real frame of the target ship; A pre-selection box determination module is used to cluster the real boxes of the target ship in the annotation information using a K-Means++ clustering algorithm to obtain K cluster pre-selection boxes of different sizes, and to perform a small variation on the K cluster pre-selection boxes of different sizes within a range of 1 using a genetic algorithm to obtain K pre-selection boxes of different sizes; The genetic algorithm is used to perform a small variation on K cluster pre-selection boxes of different sizes within a range of 1, and K target pre-selection boxes of different sizes are screened, including: Overlapping the center point of the real frame of the target ship with the center point of the cluster pre-selected frame to which it belongs, and calculating the intersection-over-union ratio of the real frame of the target ship and the cluster pre-selected frame; Traverse all the real boxes of the target ships and the pre-selected boxes of the clusters to which they belong, and calculate the average value of the intersection-over-union ratio Avg IOU of the two. The calculation formula is as follows: Among them, N represents the number of real boxes of the target ship, K represents the number of cluster pre-selected boxes, and x i represents the true box of the i-th target ship, c j represents the jth cluster pre-selection box; The genetic algorithm is used to perform small mutations on K cluster pre-selected boxes of different sizes within the range of 1, and the mutation results are used as new cluster pre-selected boxes. The average value of the intersection and union (Avg IOU) of the real boxes of all target ships and their new cluster pre-selected boxes is recalculated. If Avg IOU increases, the cluster pre-selected box is updated to the mutation result corresponding to the increase in Avg IOU. After several iterations, the mutation result corresponding to the increase in Avg IOU is obtained as the final pre-selected box. According to the curve of Avg IOU changing with K, determine the K value, K value ≤ 12; An optimization module, used to optimize the target detection model according to the pre-selected box to obtain a final target detection model; The target detection model is trained based on the Faster R-CNN network, and the feature extraction network of the Faster R-CNN network adopts an improved VGG16 network; the improved VGG16 network includes: retaining the first 7 convolutional layers and the first 2 pooling layers of the original VGG16 network, adding a BN layer after each convolutional layer, deleting the last 6 convolutional layers of the original VGG16 network, and replacing the last three pooling layers of the original VGG16 network with 4 hole convolutional layers; the number of holes in the 4 hole convolutional layers increases successively by 2, 3, 4 and 5 respectively.

4. A method for detecting ship targets using SAR images. It is characterized in that The method comprises: Acquire a SAR image, and preprocess the SAR image; The preprocessed SAR image is input into the target detection model obtained according to the method of claim 1 to obtain the ship in the SAR image.

5. A SAR image ship target detection device, It is characterized in that The device comprises: An acquisition module, used for acquiring SAR images and preprocessing the SAR images; The detection module is used to input the preprocessed SAR image into the target detection model obtained according to the method of claim 1 to obtain the ship in the SAR image.

6. An electronic device, It is characterized in that include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of claim 1.

7. A computer readable medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the method according to claim 1 is implemented.

Citation Information

Patent Citations

  • Ship detection method based on RetinaNet algorithm

    CN113505699A

  • SAR image ship detection method and device, electronic equipment and storage medium

    CN113780272A

  • Armsuit personnel equipment detection method and system based on dynamic neural network

    CN115019096A

  • Armed personnel identification method and system fusing equipment and behavior characteristics

    CN115035599A

  • SAR image ship identification method based on improved Faster RCNN

    CN115294468A