A target detection method and device based on SAR images

Abstract

A method and device for target detection of SAR images, which relate to the technical field of target detection and can improve the detection accuracy of target detection. This method is implemented by proposing a new target detection model based on the YOLOv4 model. The target detection model includes a benchmark network, an intermediate network, and a detection network. The intermediate network of the target detection model includes a first feature fusion unit, a second feature fusion unit, and a third feature fusion unit. The third feature fusion unit includes a first CBL module, a DSPP module, a second CBL module, a first merging module, and a third CBL module connected in sequence. The method includes: using the new target detection model to detect each sub-image of the SAR image to be detected, obtaining the prediction box information corresponding to the sub-image, which is used to determine the detection target in the sub-image, and then integrating the prediction box information corresponding to all sub-images of the SAR image to be detected to determine the detection target in the SAR image to be detected.

Description

Technical Field

[0001] This application relates to the technical field of object detection, and particularly to an object detection method and device based on SAR images. Background Art

[0002] Satellite remote sensing is an effective monitoring means and has been widely applied in the fields of national defense, natural resources, transportation, meteorology, ocean, environmental protection, emergency, etc. Compared with optical remote sensing, synthetic aperture radar (SAR) is a sensor using active microwave remote sensing technology, and the image data obtained by SAR can be called SAR images.

[0003] Currently, the recognition of ships in the deep sea can also be achieved based on SAR images. By analyzing and processing SAR images, ships can be detected. However, the detection accuracy of existing ship detection methods based on SAR images is relatively low, and the detection performance needs to be improved. Summary of the Invention

[0004] The embodiments of this application provide an object detection method and device based on SAR images, which can improve the accuracy of object detection.

[0005] To achieve the above object, the embodiments of this application adopt the following technical solutions:

[0006] In a first aspect, the present application provides a target detection method based on synthetic aperture radar (SAR) images. The method is implemented by proposing a new target detection model based on the YOLOv4 model. The target detection model includes a benchmark network, an intermediate network, and a detection network connected in sequence. The intermediate network includes a first feature fusion unit, a second feature fusion unit, and a third feature fusion unit. The first input ends of the first feature fusion unit, the second feature fusion unit, and the third feature fusion unit are respectively coupled to three input ends of the intermediate network. The second input end of the first feature fusion unit is connected to the second output end of the second feature fusion unit. The first output end of the first feature fusion unit is coupled to the first output end of the intermediate network. The second output end of the first feature fusion unit is connected to the second input end of the second feature fusion unit. The first output end of the second feature fusion unit is coupled to the second output end of the intermediate network. The third input end of the second feature fusion unit is connected to the second output end of the third feature fusion unit. The third output end of the second feature fusion unit is connected to the second input end of the third feature fusion unit. The first output end of the third feature fusion unit is coupled to the third output end of the intermediate network. Among them, the third feature fusion unit includes a first CBL module, an atrous spatial pyramid pooling (DSPP) module with dilated convolution, a second CBL module, a first merging module, and a third CBL module connected in sequence. The third feature fusion unit further includes a first downsampling module. One end of the first downsampling module is coupled to the second input end of the third feature fusion unit, and the other end of the first downsampling module is connected to the merging module. The CBL module is formed by connecting a convolutional layer, a batch normalization layer, and a LeakyRelu activation function layer in series.

[0007] Based on the above object detection model, the object detection method based on SAR images provided in the embodiments of the present application includes: using the reference network to extract features from each sub-image of the SAR image to be detected, obtaining a first predicted feature map, a second predicted feature map, and a third predicted feature map of the sub-image; wherein, the sizes of the first predicted feature map, the second predicted feature map, and the third predicted feature map decrease in sequence; the SAR image to be detected includes multiple sub-images, and the multiple sub-images are obtained by splitting the SAR image to be detected; and performing feature fusion on the first predicted feature map, the second predicted feature map, and the third predicted feature map through the intermediate network to obtain a first fused feature map, a second fused feature map, and a third fused feature map; wherein, the first fused feature map is processed by the first feature fusion unit, the second predicted feature map is processed by the second feature fusion unit, and the third fused feature map is processed by the third feature fusion unit; and based on the detection network to process the first fused feature map, the second fused feature map, and the third fused feature map, determining the predicted box information corresponding to the sub-image, and the predicted box information is used to determine the detection target in the sub-image; finally, integrating the predicted box information corresponding to all sub-images of the SAR image to be detected to determine the detection target in the SAR image to be detected.

[0008] In one implementation manner of the first aspect, the second feature fusion unit includes a fourth CBL module, a second merging module, a fifth CBL module, a third merging module, and a sixth CBL module connected in sequence. The second feature fusion unit further includes a seventh CBL module, a first upsampling module, and a second downsampling module; one end of the first upsampling module is connected to the second merging module, the other end of the first upsampling module is connected to one end of the seventh CBL module, the other end of the seventh CBL module is coupled to the third input end of the second feature fusion unit, one end of the second downsampling module is connected to the third merging module, and the other end of the second downsampling module is coupled to the second input end of the second feature fusion unit.

[0009] In one implementation manner of the first aspect, the first feature fusion unit includes an eighth CBL module, a fourth merging module, and a ninth CBL module connected in sequence. The first feature fusion unit further includes a tenth CBL module and a second upsampling module. One end of the second upsampling module is connected to the fourth merging module, and the other end is connected to one end of the tenth CBL module. The other end of the tenth CBL module is coupled to the second input end of the first feature fusion unit.

[0010] In an implementation of the first aspect, the DSPP module includes an eleventh CBL module, a first branch module, a second branch module, a third branch module, a fifth merging module, a twelfth CBL module, and a summing module; one end of the eleventh CBL module is coupled to the input end of the DSPP module, one ends of the first branch module, the second branch module, and the third branch module are respectively connected to the other end of the eleventh CBL module, the other ends of the first branch module, the second branch module, and the third branch module are respectively connected to one end of the fifth merging module, the other end of the fifth merging module is connected to one end of the twelfth CBL module, the other end of the twelfth CBL module is connected to one end of the summing module, and the other end of the summing module is coupled to the output end of the DSPP module;

[0011] Wherein, the first branch module includes a thirteenth CBL module, a first DBL module, and a first max pooling layer connected in sequence; the second branch module includes a fourteenth CBL module, a second DBL module, and a second max pooling layer connected in sequence; the third branch module includes a fifteenth CBL module, a sixteenth CBL module, a third DBL module, and a third max pooling layer connected in sequence; the DBL module is formed by connecting a dilated convolutional layer, a batch normalization layer, and a LeakyRelu activation function layer in series.

[0012] In an implementation of the first aspect, the predicted bounding box information includes the coordinates of the center point of the predicted bounding box, the height of the predicted bounding box, and the width of the predicted bounding box.

[0013] In an implementation of the first aspect, the object detection method based on SAR images provided in the embodiments of the present application further includes: using an artificial intelligence algorithm to train the object detection model based on an SAR image dataset, where the SAR image dataset includes a plurality of SAR images and the predicted bounding box information corresponding to the plurality of SAR images respectively.

[0014] In an implementation of the first aspect, integrating the predicted bounding box information corresponding to all sub-images of the SAR image to be detected to determine the detection target in the SAR image to be detected includes: converting the coordinates of the center points of the predicted bounding boxes corresponding to all sub-images in the sub-images to the coordinates in the SAR image to be detected to determine the predicted bounding box information corresponding to the SAR image to be detected, and the predicted bounding box information corresponding to the SAR image to be detected is used to determine the detection target in the SAR image to be detected.

[0015] In an implementation of the first aspect, after converting the coordinates of the center points of the prediction boxes corresponding to all sub-images in the coordinates of the sub-images to the coordinates in the SAR image to be detected, the method further includes: using the non-maximum suppression algorithm to perform duplicate removal processing on the prediction boxes corresponding to the SAR image to be detected.

[0016] In an implementation of the first aspect, the object detection method provided by the embodiments of the present application further includes: based on the SAR image dataset, using the K-means clustering algorithm to determine the input parameters of the detection network, and the input parameters include the initial width and height of the prediction box.

[0017] In a second aspect, the present application provides a detection device, in which a new object detection model proposed based on the YOLOv4 model is integrated. The object detection model includes a reference network, an intermediate network, and a detection network connected in sequence. The intermediate network includes a first feature fusion unit, a second feature fusion unit, and a third feature fusion unit. The first input ends of the first feature fusion unit, the second feature fusion unit, and the third feature fusion unit are respectively coupled to three input ends of the intermediate network. The second input end of the first feature fusion unit is connected to the second output end of the second feature fusion unit. The first output end of the first feature fusion unit is coupled to the first output end of the intermediate network. The second output end of the first feature fusion unit is connected to the second input end of the second feature fusion unit; the first output end of the second feature fusion unit is coupled to the second output end of the intermediate network. The third input end of the second feature fusion unit is connected to the second output end of the third feature fusion unit. The third output end of the second feature fusion unit is connected to the second input end of the third feature fusion unit; the first output end of the third feature fusion unit is coupled to the third output end of the intermediate network; wherein, the third feature fusion unit includes a first CBL module, an atrous spatial pyramid pooling DSPP module, a second CBL module, a first merging module, and a third CBL module connected in sequence. The third feature fusion unit further includes a first downsampling module. One end of the first downsampling module is coupled to the second input end of the third feature fusion unit, and the other end of the first downsampling module is connected to the merging module; the CBL module is composed of a convolutional layer, a batch normalization layer, and a LeakyRelu activation function layer connected in series.

[0018] The detection device includes a feature extraction module, a feature fusion module, and a determination module. Among them, the feature extraction module is used to extract features from each sub-image of the SAR image to be detected by using the reference network, and obtain a first predicted feature map, a second predicted feature map, and a third predicted feature map of the sub-image; among them, the sizes of the first predicted feature map, the second predicted feature map, and the third predicted feature map decrease in sequence; the SAR image to be detected includes a plurality of sub-images, and the plurality of sub-images are obtained by slicing the SAR image to be detected; the feature fusion module is used to perform feature fusion on the first predicted feature map, the second predicted feature map, and the third predicted feature map through the intermediate network, and obtain a first fused feature map, a second fused feature map, and a third fused feature map; among them, the first fused feature map is processed by the first feature fusion unit, the second predicted feature map is processed by the second feature fusion unit, and the third fused feature map is processed by the third feature fusion unit; the determination module is used to process the first fused feature map, the second fused feature map, and the third fused feature map based on the detection network, determine the predicted box information corresponding to the sub-image, and the predicted box information is used to determine the detection target in the sub-image; and integrate the predicted box information corresponding to all sub-images of the SAR image to be detected to determine the detection target in the SAR image to be detected.

[0019] In an implementation manner of the second aspect, the detection device provided in the embodiment of the present application further includes a processing module, and the processing module is used to train the target detection model based on the SAR image data set by using an artificial intelligence algorithm. The SAR image data set includes a plurality of SAR images and the predicted box information corresponding to the plurality of SAR images respectively.

[0020] In an implementation manner of the second aspect, the determination module is specifically used to convert the coordinates of the center points of the predicted boxes corresponding to all sub-images in the sub-images to the coordinates in the SAR image to be detected, so as to determine the predicted box information corresponding to the SAR image to be detected, and the predicted box information corresponding to the SAR image to be detected is used to determine the detection target in the SAR image to be detected.

[0021] In an implementation manner of the second aspect, the processing module is further used to perform duplicate removal processing on the predicted boxes corresponding to the SAR image to be detected by using a non-maximum suppression algorithm.

[0022] In an implementation manner of the second aspect, the determination module is further used to determine the input parameters of the detection network based on the SAR image data set by using a K-means clustering algorithm, and the input parameters include the initial width and height of the predicted box.

[0023] In a third aspect, an embodiment of the present application provides an electronic device, including: a processor and a memory coupled to the processor; the memory is used to store computer instructions, and when the electronic device runs, the processor executes the computer instructions stored in the memory, so that the electronic device executes the method described in the above first aspect or any one of its implementation manners.

[0024] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which includes a computer program. When the computer program runs on a computer, it is used to execute the method described in the above first aspect or any one of its implementation manners.

[0025] In the target detection method based on SAR images provided by the embodiments of the present application, due to the introduction of the idea of dilated convolution in the YOLOv4 model and the setting of the DSPP module with strong feature extraction ability, a new target detection model is obtained. The receptive field of this target detection model is relatively large. Then, this target detection model is used to detect targets in SAR images, which improves the detection rate of small targets to a certain extent. In this way, the detection accuracy of target detection can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is one of the schematic diagrams of the architecture of a YOLOv4 model provided by an embodiment of the present application;

[0027] Figure 2 It is the overall flowchart of the target detection method based on SAR images provided by an embodiment of the present application;

[0028] Figure 3 It is another schematic diagram of the architecture of a target detection model provided by an embodiment of the present application;

[0029] Figure 4 It is still another schematic diagram of the architecture of a target detection model provided by an embodiment of the present application;

[0030] Figure 5 It is the schematic diagram of the structure of a DSPP module provided by an embodiment of the present application;

[0031] Figure 6 It is one of the schematic diagrams of the target detection method based on SAR images provided by an embodiment of the present application;

[0032] Figure 7 It is another schematic diagram of the target detection method based on SAR images provided by an embodiment of the present application;

[0033] Figure 8 It is the schematic diagram of the structure of a detection device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0034] The terms "first", "second", etc. in the description and claims of this application are used to distinguish different objects, rather than to describe a specific order of the objects.

[0035] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0036] In the description of this application, unless otherwise specified, "a plurality of" means two or more. For example, a plurality of sub-images means two or more sub-images.

[0037] Next, some concepts related to the embodiments of this application will be described and explained.

[0038] Dilated convolution: Also known as atrous convolution or dilated convolution, it is to inject holes into a standard convolution kernel (the standard convolution kernel can be considered as a basic convolution kernel) to increase the size of the convolution kernel, thereby increasing the reception field. It should be understood that injecting holes means adding 0 values at specified positions. Compared with the standard convolution, one parameter of the dilated convolution is the dilation rate, which refers to the number of intervals between the points of the convolution kernel.

[0039] Reception field: In a convolutional neural network, the reception field refers to the area of the input image that a certain point on the feature map can correspond to (or map to). That is to say, the points on the feature map are calculated based on the area of the reception field size in the input image. The larger the value of the reception field, the larger the range of the original image it can map to, which also means that it may contain more global and higher-level semantic features; on the contrary, the smaller the value of the reception field, the more local and detailed the features it contains.

[0040] YOLO algorithm: It is a regression algorithm based on deep learning and is currently widely used in object detection. The idea of the YOLO algorithm is to predict the information of an image, learn the image features, and then perform object detection based on the learned image features. With the development of technology, a series of improved models have been derived from the YOLO algorithm, such as the YOLOv1 model, the YOLOv2 model, the YOLOv3 model, and the YOLOv4 model. It should be understood that currently the YOLO algorithm is mostly used for object detection based on optical images.

[0041] It should be noted that the embodiments of the present application involve improving the traditional YOLOv4 model to obtain a new object detection model, and applying the object detection model to object detection (such as ship detection). First, the architecture of the traditional YOLOv4 model will be introduced in detail below.

[0042] Reference Figure 1 , the YOLOv4 model includes a backbone network (abbreviated as backbone) 101, an intermediate network (abbreviated as neck) 102, and a detection network (abbreviated as head) 103 connected in sequence. For the specific structures of the backbone network 101, the intermediate network 102, and the detection network 103, please refer to Figure 1 and other existing materials of YOLOv4.

[0043] Among them, CBM represents the series connection of a convolutional layer, a batch normalization layer, and a Mish activation function layer, and CBL represents the series connection of a convolutional layer, a batch normalization layer, and a LeakyRelu activation function layer. The backbone network 101 uses the CSPDarknet53 feature extraction network to extract features from images. CPSDarknet53 is an improved structure obtained by combining the cross-stage network (CSPNet) algorithm on the basis of Darknet53 in YOLOv3. Among them, the cross-stage (cross stage partial, CSP) module divides the input feature map into two branches and then merges them through cross-stage hierarchies. Combining Figure 1 it can be seen that after the backbone network 101 extracts features from the input image, three feature maps of the input image are obtained. The sizes of the three feature maps are different. Exemplarily, the sizes of the three feature maps can also be 76*76, 38*38, and 19*19 respectively.

[0044] The above backbone network 101 can enhance the learning ability of the convolutional neural networks (CNN) network and lightweight the model while maintaining the model accuracy. In addition, the Mish activation function is used in CSPDarknet53. The Mish activation function is improved from the LeakyRelu function. Compared with the LeakyRelu function, the Mish function is smoother and can further improve the model accuracy.

[0045] The SPP in the intermediate network 102 is a spatial pyramid pooling (SPP) module. For the detailed structure of the SPP module, reference can be made to the relevant descriptions in the prior art. The merging module in the intermediate network 102 corresponds to the concat operation. Combining Figure 1It can be seen that through the intermediate network 102, feature pyramids of three scales can be obtained. Furthermore, the feature pyramids of the three scales are input into the detection network 103. The detection heads in the detection network 103 process the feature maps of different scales, and then the output results of different detection heads are merged and the prediction box information of the detection target is output. According to the prediction box information, the detection target in the input image can be determined.

[0046] In the prior art, when performing target detection based on SAR images, due to the limitation of the receptive field of the detection model, some small targets in the image cannot be detected (for example, small ships may not be detected when detecting SAR images of the deep sea and far sea), resulting in low detection accuracy. To solve the problem of low detection accuracy in the prior art, the embodiments of the present application provide a target detection method and device based on SAR images, improve the traditional YOLOv4 model, and use the improved YOLOv4 model (the embodiments of the present application collectively refer to the improved YOLOv4 model as the target detection model) to implement target detection based on SAR images, which can improve the detection accuracy.

[0047] The target detection method and device based on SAR images provided by the embodiments of the present application will be described in detail below.

[0048] Figure 2 It is the overall flowchart of the target detection method based on SAR images provided by the embodiments of the present application. It should be noted that when applying the technical solution provided by the embodiments of the present application for target detection, the detection target is not limited. In practical applications, the detection target can be any target, such as vehicles, ships, etc. In the embodiments of the present application, the detection target is a ship as an example for illustration.

[0049] As Figure 2 shown, the target detection method based on SAR images provided by the embodiments of the present application includes a model training stage and a target detection stage. The initialized target detection model (the target detection model is an improved YOLOv4 model) is trained with a large number of SAR images (ship data) to obtain the target detection model. Then, the target detection model is used to detect the SAR image to be detected (i.e., the measured SAR image), and the prediction box information is obtained. Then, according to the prediction box information, the position of the target in the measured SAR image is determined, that is, the detection target in the measured SAR image is determined.

[0050] Combined with Figure 2 , the SAR image ship data set includes a large number of SAR images and the prediction box information corresponding to each SAR image. It should be understood that according to the prediction box information, the ships in the SAR image can be determined, and the prediction box information can be understood as the detection result of target detection.

[0051] During the model training phase, the SAR image ship dataset is divided into three mutually exclusive sets: the training set, the validation set, and the test set. The training set is used to learn and adjust the parameters of the detection model in each iteration. The validation set is used to verify the model accuracy in each iteration. After a stage of learning, the test set is used to evaluate the trained model, and parameters such as the average precision are calculated through the test set.

[0052] During the object detection phase, the SAR image obtained after preprocessing the measured SAR image is called the SAR image to be detected. The SAR image to be detected is used as the input of the object detection model to output the detection result of the SAR image to be detected.

[0053] Optionally, in the embodiments of the present application, the preprocessing of the measured SAR image includes, but is not limited to, image denoising of the measured SAR image (such as adaptive threshold segmentation, morphological filtering, and geometric clustering processing. A clean and smooth SAR image can be obtained through denoising), image enhancement, etc.

[0054] In summary, the object detection method based on SAR images provided in the embodiments of the present application is implemented by a new object detection model proposed based on the YOLOv4 model. This object detection model is an improved model of the traditional YOLOv4 model. As Figure 3 shown, the object detection model provided in the embodiments of the present application includes a backbone network 301, an intermediate network 302, and a detection network 303 connected in sequence. Among them, the structures of the backbone network 301 and the detection network 303 are the same as those of the backbone network 101 and the detection network 103 in the Figure 1 YOLOv4 model shown, and specific references can be made to Figure 1 .

[0055] The embodiments of the present application have improved the intermediate network in the YOLOv4 model. Referring to Figure 3 , the intermediate network 302 of the object detection model provided in the embodiments of the present application includes a first feature fusion unit 3021, a second feature fusion unit 3022, and a third feature fusion unit 3023.

[0056] Among them, the first input end 211 of the first feature fusion unit 3021, the first input end 221 of the second feature fusion unit 3022, and the first input end 231 of the third feature fusion unit 3023 are respectively coupled to the three input ends of the intermediate network 302, that is, the input end 21, the input end 22, and the input end 23. The second input end 212 of the first feature fusion unit 3021 is connected to the second output end 222 of the second feature fusion unit 3022. The first output end 213 of the first feature fusion unit 3021 is coupled to the first output end 24 of the intermediate network 302. The second output end 214 of the first feature fusion unit 3021 is connected to the second input end 223 of the second feature fusion unit 3022; the first output end 224 of the second feature fusion unit 3022 is coupled to the second output end 25 of the intermediate network 302. The third input end 225 of the second feature fusion unit 3022 is connected to the second output end 232 of the third feature fusion unit 3023. The third output end 226 of the second feature fusion unit 3022 is connected to the second input end 233 of the third feature fusion unit 3023; the first output end 234 of the third feature fusion unit 3023 is coupled to the third output end 26 of the intermediate network 302.

[0057] Continue to refer to Figure 3 , in the embodiment of the present application, the third feature fusion unit 3023 includes a first CBL module, an atrous spatial pyramid pooling DSPP module, a second CBL module, a first merging module, and a third CBL module connected in sequence. The third feature fusion unit 3023 further includes a first downsampling module. One end of the first downsampling module is coupled to the second input end 233 of the third feature fusion unit 3023, and the other end of the first downsampling module is connected to the merging module.

[0058] It should be noted that the CBL module in the embodiment of the present application is composed of a convolutional layer, a batch normalization layer, and a LeakyRelu activation function layer connected in series.

[0059] Optionally, in combination with Figure 3 , as Figure 4 shown, the second feature fusion unit 3022 includes a fourth CBL module, a second merging module, a fifth CBL module, a third merging module, and a sixth CBL module connected in sequence. The second feature fusion unit 3022 further includes a seventh CBL module, a first upsampling module, and a second downsampling module. Among them, one end of the first upsampling module is connected to the second merging module, the other end of the first upsampling module is connected to one end of the seventh CBL module, the other end of the seventh CBL module is coupled to the third input end 225 of the second feature fusion unit 3022, one end of the second downsampling module is connected to the third merging module, and the other end of the second downsampling module is coupled to the second input end 223 of the second feature fusion unit 3022.

[0060] Furthermore, as Figure 4 shown, the first feature fusion unit 3021 includes an eighth CBL module, a fourth merging module, and a ninth CBL module connected in sequence. The first feature fusion unit 3021 further includes a tenth CBL module and a second upsampling module. One end of the second upsampling module is connected to the fourth merging module, and the other end is connected to one end of the tenth CBL module. The other end of the tenth CBL module is coupled to the second input end 212 of the first feature fusion unit 3021.

[0061] Optionally, the DSPP module in the above object detection model is a new feature extraction module proposed in the embodiments of the present application. The specific structure of the DSPP module is as Figure 5 shown. The DSPP module includes an eleventh CBL module, a first branch module, a second branch module, a third branch module, a fifth merging module, a twelfth CBL module, and a summation module. Among them, one end of the eleventh CBL module is coupled to the input end of the DSPP module. One ends of the first branch module, the second branch module, and the third branch module are respectively connected to the other end of the eleventh CBL module. The other ends of the first branch module, the second branch module, and the third branch module are respectively connected to one end of the fifth merging module. The other end of the fifth merging module is connected to one end of the twelfth CBL module. The other end of the twelfth CBL module is connected to one end of the summation module. The other end of the summation module is coupled to the output end of the DSPP module.

[0062] The above first branch module includes a thirteenth CBL module, a first DBL module, and a first max pooling layer connected in sequence; the second branch module includes a fourteenth CBL module, a second DBL module, and a second max pooling layer connected in sequence; the third branch module includes a fifteenth CBL module, a sixteenth CBL module, a third DBL module, and a third max pooling layer connected in sequence.

[0063] It should be noted that the DBL module in the embodiments of the present application is formed by connecting a dilated convolutional layer, a batch normalization layer, and a LeakyRelu activation function layer in series.

[0064] Exemplarily, the size of the convolution kernel in the convolutional layer of the above eleventh CBL module is 1×1.

[0065] In the above first branch module, the size of the convolution kernel in the convolutional layer of the thirteenth CBL module is 1×1, the size of the convolution kernel in the dilated convolutional layer of the first DBL module is 1×1, and the size of the kernel of the first max pooling layer is 5.

[0066] In the above second branch module, the size of the convolution kernel in the convolution layer of the fourteenth CBL module is 3×3, the size of the convolution kernel in the dilated convolution layer of the second DBL module is 3×3, and the size of the kernel of the second max pooling layer is 9.

[0067] In the above third branch module, the size of the convolution kernel in the convolution layer of the fifteenth CBL module is 3×3, the size of the convolution kernel in the convolution layer of the sixteenth CBL module is 3×3, the size of the convolution kernel in the dilated convolution layer of the third DBL module is 5×5, and the size of the kernel of the first max pooling layer is 13.

[0068] Combined with Figure 6 It can be seen that in the DSPP module, each DBL module is correspondingly connected to a CBL module. For example, the first DBL module is connected to the thirteenth CBL module, the second DBL module is connected to the fourteenth CBL module, and the third DBL module is connected to the fifteenth CBL module and the sixteenth CBL module. Moreover, in the first branch module and the second branch module, the size of the convolution kernel in the DBL module is the same as that in the CBL module connected to this DBL module. For example, the size of the convolution kernel in the first DBL module is 1×1, and the size of the convolution kernel in the thirteenth CBL module is also 1×1; the size of the convolution kernel in the second DBL module is 3×3, and the size of the convolution kernel in the fourteenth CBL module is also 1×1.

[0069] Particularly, in the third branch module, after two CBL modules are connected, they are then connected to the third DBL module. The size of the convolution kernel in the third DBL module is 5×5, and the size of the convolution kernel in both of these two CBL modules is 3×3. That is to say, two CBL modules with 3×3-sized convolution kernels are used instead of a CBL module with a 5×5-sized convolution kernel. This can reduce the parameters of the model and also deepen the non-linear layer of the model, making the target detection faster.

[0070] Furthermore, referring to Figure 5 , the results obtained through the first branch module, the second branch module, and the third branch module can be regarded as residuals. Then, the sum module sums the residuals and the input of the DSPP module. Such a residual structure can enhance gradient propagation and make the model easier to optimize.

[0071] In the embodiments of the present application, the above target detection model can be used to perform target detection based on SAR images. Before performing target detection, it is necessary to train the initial target detection model using the SAR image dataset corresponding to this application scenario to obtain a model suitable for this application scenario, and then use this target detection model for target detection, such as ship detection. Specifically, the embodiments of the present application adopt artificial intelligence algorithms to train a target detection model based on the SAR image dataset. Among them, the SAR image dataset includes multiple SAR images and the prediction box information corresponding to each of the multiple SAR images. Optionally, the artificial intelligence algorithms include, but are not limited to, machine learning, deep learning and other algorithms.

[0072] Based on the trained target detection model, as Figure 6 shown, the embodiments of the present application provide a target detection method based on SAR images, and this method includes S601 - S604.

[0073] S601. Use a benchmark network to extract features from each sub - image of the SAR image to be detected, and obtain the first predicted feature map, the second predicted feature map, and the third predicted feature map of the sub - image.

[0074] Among them, the sizes of the first predicted feature map, the second predicted feature map, and the third predicted feature map decrease in sequence. The SAR image to be detected includes multiple sub - images, and the multiple sub - images are obtained by splitting the SAR image to be detected.

[0075] In the embodiments of the present application, the SAR image to be detected can be a pre - processed SAR image. Since the size of the SAR image is very large, while the size of the input image of the target detection model is limited, when performing target detection, the SAR image to be detected is divided into multiple sub - images, that is, the SAR image is sliced into multiple image blocks, and the sub - images are used as the input of the target detection model.

[0076] It should be noted that when splitting the SAR image to be detected, the step size of the split is smaller than the size of the sub - image, that is, there is an overlap between the sub - images. The overlapping split method can prevent the detection target from being missed when the detection target straddles two sub - images, or can prevent the detection target from being recognized as two different targets when the detection target straddles two sub - images.

[0077] S602. Through an intermediate network, perform feature fusion on the first predicted feature map, the second predicted feature map, and the third predicted feature map to obtain the first fused feature map, the second fused feature map, and the third fused feature map.

[0078] Among them, the first fused feature map is obtained by processing of the first feature fusion unit, the second predicted feature map is obtained by processing of the second feature fusion unit, and the third fused feature map is obtained by processing of the third feature fusion unit. The detailed process of obtaining the first fused feature map, the second fused feature map, and the third fused feature map by processing the first predicted feature map, the second predicted feature map, and the third predicted feature map through the intermediate network can be combined with Figure 5 the structure of the object detection model shown in the understanding, which will not be elaborated here.

[0079] In the embodiment of the present application, dilated convolution is used in the DSPP module. Since dilated convolution can increase the receptive field of the model, and the larger the receptive field, the stronger the ability of the model to capture features. That is, the object detection model provided by the embodiment of the present application can effectively extract the features of the SAR image to be detected. Thus, the effect of feature extraction of the model for small targets in space is improved, which is beneficial to the detection of small targets, and thus the detection accuracy can be improved.

[0080] When this method is applied to ship detection, it effectively solves the problem that small ships cannot be detected, and improves the ship detection accuracy. Further, compared with the existing CFAR ship detection method, ship detection based on the improved YOLOv4 model (i.e., the object detection model) provided by the embodiment of the present application can improve the speed of ship detection.

[0081] S603. Process the first fused feature map, the second fused feature map, and the third fused feature map based on the detection network to determine the prediction box information corresponding to the sub-image.

[0082] In the embodiment of the present application, the prediction box information is used to determine the detection target in the sub-image. The prediction box information includes the coordinates of the center point of the prediction box, the height of the prediction box, and the width of the prediction box. It can be understood that the rough position of the detection target in the sub-image can be determined according to the coordinates of the center point of the prediction box, and then the precise position of the target can be determined according to the size information of the prediction box (including the height and width of the prediction box).

[0083] Optionally, the above prediction box information may further include other information, such as the index (or number) of the sub-image.

[0084] In the embodiment of the present application, the process of processing the first fused feature map, the second fused feature map, and the third fused feature map through the detection network to determine the prediction box information corresponding to the sub-image is essentially a process of continuously updating the preset prediction box according to the first fused feature map, the second fused feature map, and the third fused feature map, so that the position of the prediction box in the sub-image continuously approaches the position of the target to be detected, and the size of the prediction box approaches the size of the target to be detected. It can be understood that the size of the prediction box is an input parameter of the detection network.

[0085] The size of the above preset prediction box can be set according to the characteristics of the target to be detected in a certain application scenario. In the embodiments of the present application, based on the SAR image dataset (corresponding to the SAR image dataset containing ships), the K-means clustering algorithm can be used to determine the input parameters of the detection network, and the input parameters include the initial width and height of the prediction box. The specific process is as follows:

[0086] Step 1: Extract the coordinates of the 4 vertices of the rectangular box corresponding to the detection target (such as a ship) in the SAR image dataset (this rectangular box is used to frame the detection target, and this rectangular box is the prediction box), and determine the width and height of the rectangular box according to the coordinate data. Multiple rectangular boxes can be extracted from the SAR image dataset.

[0087] Step 2: Initialize K prior boxes and use these K prior boxes as the prior box centers (i.e., clustering centers).

[0088] Among them, K is a positive integer multiple of 3. For example, K is 3, 9, 12, etc.

[0089] Reference Figure 4 It can be known that the detection network includes three channels, and these three channels respectively correspond to the three output values of the intermediate network. Therefore, the three channels of the detection network respectively correspond to a group of prediction boxes. Exemplarily, if it is set that there is one target in the sub-image and each target corresponds to a prediction box, then one channel of the detection network corresponds to one prediction box, and the three channels of the detection network correspond to a total of 3 prediction boxes; if it is set that there are three targets in the sub-image and each target corresponds to a prediction box, then one channel of the detection network corresponds to three prediction boxes, and the three channels of the detection network correspond to a total of 9 prediction boxes.

[0090] Optionally, K rectangular boxes can be randomly selected from the multiple rectangular boxes in the above step 1 as the initialized K prior boxes, that is, the sizes of the randomly selected K rectangular boxes are used as the initial values of the sizes of the K prior boxes, that is, the initial clustering centers.

[0091] Step 3: Calculate the overlap degree (intersection over union, IOU) between each rectangular box (i.e., each sample) and the centers of the K prior boxes.

[0092] In the embodiments of the present application, IOU is an index for measuring the overlap degree (or intersection-to-union ratio) between rectangular boxes. The more similar two rectangular boxes are, the larger their IOU values are. The IOU calculation formula is:

[0093]

[0094] Among them, box represents a rectangular box, anchor represents the prior box center, wa represents the width of the prior box, h a is the height of the prior box, w b is the width of the rectangular box, h b is the height of the rectangular box. The value of IOU is between 0 and 1.

[0095] Based on the calculation formula of IOU, the distance formula between two rectangular boxes is defined as the following formula:

[0096] d(box,anchor)=1 - IOU(box,anchor)

[0097] As is well known, the traditional clustering method uses the Euclidean distance to measure the difference between the clustering center and each sample. If the traditional K-means clustering algorithm is used, the width and height of the prior box can be directly clustered to obtain K combinations of width and height. However, this method treats the height and width as two independent variables and does not consider the correlation between the height and width of the prior box. In the embodiments of the present application, IOU is introduced to measure the distance metric, fully considering the correlation between the height and width of the prior box.

[0098] Step 4: Assign each rectangular box to the category corresponding to the center of the prior box with the closest distance, then multiple rectangular boxes can be divided into K clusters (i.e., K classes), and calculate the average values of the width and height of the rectangular boxes in each cluster, and use these average values (including the average width and average height) to update the center of the prior box. And repeat Step 3 and Step 4 until the center of the prior box no longer changes, and then use the size of the finally determined center of the prior box as the input parameter of the intermediate network.

[0099] S604: Integrate the prediction box information corresponding to all sub-images of the SAR image to be detected to determine the detection target in the SAR image to be detected.

[0100] Optionally, in combination with Figure 6 , as Figure 7 shown, the above S604 can be specifically implemented through S6041.

[0101] S6041: Convert the coordinates of the center points of the prediction boxes corresponding to all sub-images in the sub-images to the coordinates in the SAR image to be detected to determine the prediction box information corresponding to the SAR image to be detected.

[0102] The above prediction box information corresponding to the SAR image to be detected is used to determine the detection target in the SAR image to be detected.

[0103] In the embodiments of the present application, the target detection model (i.e., the improved YOLOv4 model) outputs the prediction box information corresponding to each sub-image. The coordinates of the center point of the prediction box are the coordinates in the coordinate system established with the sub-image as the reference. For the entire SAR image, the coordinates of the center point of the prediction box corresponding to the SAR image should be the coordinates in the coordinate system established with the SAR image as the reference. Therefore, it is necessary to convert the coordinates of the center point of the prediction box corresponding to the sub-image in the sub-image to the coordinates in the SAR image to be detected.

[0104] It should be understood that since there is an overlap between sub-images when the SAR image is sliced, the overlapping sub-images may contain the same target. In this way, the prediction boxes of two sub-images correspond to the same target. Optionally, in the embodiments of the present application, after converting the coordinates of the center points of the prediction boxes corresponding to all sub-images in the sub-images to the coordinates in the SAR image to be detected, the non-maximum suppression algorithm is used to remove duplicates from the prediction boxes corresponding to the SAR image to be detected, so as to eliminate the targets that are repeatedly detected.

[0105] In summary, in the target detection method based on SAR images provided by the embodiments of the present application, due to the introduction of the idea of dilated convolution in the YOLOv4 model and the setting of the DSPP module with strong feature extraction ability, a new target detection model is obtained. The target detection model has a larger receptive field. Then, the target detection model is used to detect targets in the SAR image, which improves the detection rate of small targets to a certain extent. In this way, the detection accuracy of target detection can be improved.

[0106] Moreover, compared with the existing CFAR ship detection method, ship detection based on the improved YOLOv4 model (i.e., the target detection model) provided by the embodiments of the present application can improve the speed of ship detection.

[0107] Correspondingly, an embodiment of the present application provides a detection device, in which a new object detection model proposed based on the YOLOv4 model is integrated. The object detection model includes a reference network, an intermediate network, and a detection network connected in sequence. The intermediate network includes a first feature fusion unit, a second feature fusion unit, and a third feature fusion unit. The first input ends of the first feature fusion unit, the second feature fusion unit, and the third feature fusion unit are respectively coupled to three input ends of the intermediate network. The second input end of the first feature fusion unit is connected to the second output end of the second feature fusion unit. The first output end of the first feature fusion unit is coupled to the first output end of the intermediate network. The second output end of the first feature fusion unit is connected to the second input end of the second feature fusion unit. The first output end of the second feature fusion unit is coupled to the second output end of the intermediate network. The third input end of the second feature fusion unit is connected to the second output end of the third feature fusion unit. The third output end of the second feature fusion unit is connected to the second input end of the third feature fusion unit. The first output end of the third feature fusion unit is coupled to the third output end of the intermediate network. Wherein, the third feature fusion unit includes a first CBL module, an atrous spatial pyramid pooling DSPP module, a second CBL module, a first merging module, and a third CBL module connected in sequence. The third feature fusion unit further includes a first downsampling module. One end of the first downsampling module is coupled to the second input end of the third feature fusion unit, and the other end of the first downsampling module is connected to the merging module. The CBL module is formed by connecting a convolutional layer, a batch normalization layer, and a LeakyRelu activation function layer in series.

[0108] The detection device is used to execute the object detection method based on SAR images described in the above embodiment. As Figure 8As shown in the figure, the detection device includes a feature extraction module 801, a feature fusion module, a determination module 802, and a determination module 803. Among them, the feature extraction module 801 is used to extract features from each sub-image of the SAR image to be detected by using a reference network, and obtain a first predicted feature map, a second predicted feature map, and a third predicted feature map of the sub-image; among them, the sizes of the first predicted feature map, the second predicted feature map, and the third predicted feature map decrease in sequence; the SAR image to be detected includes multiple sub-images, and the multiple sub-images are obtained by slicing the SAR image to be detected, for example, performing S601 in the above method embodiment. The feature fusion module 802 is used to perform feature fusion on the first predicted feature map, the second predicted feature map, and the third predicted feature map through an intermediate network, and obtain a first fused feature map, a second fused feature map, and a third fused feature map; among them, the first fused feature map is processed by a first feature fusion unit, the second predicted feature map is processed by a second feature fusion unit, and the third fused feature map is processed by a third feature fusion unit, for example, performing S602 in the above method embodiment. The determination module 803 is used to process the first fused feature map, the second fused feature map, and the third fused feature map based on a detection network, determine the prediction box information corresponding to the sub-image, and the prediction box information is used to determine the detection target in the sub-image; and integrate the prediction box information corresponding to all sub-images of the SAR image to be detected to determine the detection target in the SAR image to be detected, for example, performing S603 and S604 in the above method embodiment.

[0109] Optionally, the detection device provided in the embodiment of the present application further includes a processing module 804, and the processing module 804 is used to train a target detection model based on an SAR image data set by using an artificial intelligence algorithm. The SAR image data set includes multiple SAR images and the prediction box information corresponding to the multiple SAR images respectively.

[0110] Optionally, the above determination module 803 is specifically used to convert the coordinates of the center points of the prediction boxes corresponding to all sub-images in the sub-images to the coordinates in the SAR image to be detected, so as to determine the prediction box information corresponding to the SAR image to be detected, and the prediction box information corresponding to the SAR image to be detected is used to determine the detection target in the SAR image to be detected, for example, performing S6041 in the above method embodiment.

[0111] Optionally, the processing module 804 is further used to perform duplicate removal processing on the prediction boxes corresponding to the SAR image to be detected by using a non-maximum suppression algorithm.

[0112] Each module of the above detection device can also be used to execute other steps in the above method embodiment. All relevant contents of each step involved in the above method embodiment can be cited in the function description of the corresponding functional module, and will not be repeated here.

[0113] An embodiment of this application further provides an electronic device, including: a processor and a memory coupled to the processor; the memory is used to store computer instructions, and when the electronic device runs, the processor executes the computer instructions stored in the memory, so that the electronic device executes the method described in the foregoing embodiment.

[0114] An embodiment of this application further provides a computer-readable storage medium, which includes a computer program that, when running on a computer, executes the method described in the foregoing embodiment.

[0115] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between each embodiment can be referred to each other. The key point of each embodiment is to illustrate the differences from other embodiments.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A target detection method based on synthetic aperture radar (SAR) images, characterized in that The method is implemented by proposing a new object detection model based on the YOLOv4 model. The object detection model includes a benchmark network, an intermediate network, and a detection network connected in sequence; The intermediate network includes a first feature fusion unit, a second feature fusion unit, and a third feature fusion unit. The first input ends of the first feature fusion unit, the second feature fusion unit, and the third feature fusion unit are respectively coupled to three input ends of the intermediate network. The second input end of the first feature fusion unit is connected to the second output end of the second feature fusion unit. The first output end of the first feature fusion unit is coupled to the first output end of the intermediate network. The second output end of the first feature fusion unit is connected to the second input end of the second feature fusion unit; the first output end of the second feature fusion unit is coupled to the second output end of the intermediate network. The third input end of the second feature fusion unit is connected to the second output end of the third feature fusion unit. The third output end of the second feature fusion unit is connected to the second input end of the third feature fusion unit; the first output end of the third feature fusion unit is coupled to the third output end of the intermediate network; wherein, the third feature fusion unit includes a first CBL module, an atrous spatial pyramid pooling (ASPP) module with dilated convolutions, a second CBL module, a first merging module, and a third CBL module connected in sequence. The third feature fusion unit further includes a first downsampling module. One end of the first downsampling module is coupled to the second input end of the third feature fusion unit, and the other end of the first downsampling module is connected to the merging module; The CBL module is formed by cascading a convolutional layer, a batch normalization layer, and a LeakyRelu activation function layer; The method includes: Using the benchmark network to extract features from each sub-image of the SAR image to be detected, obtaining a first predicted feature map, a second predicted feature map, and a third predicted feature map of the sub-image; wherein, the sizes of the first predicted feature map, the second predicted feature map, and the third predicted feature map decrease in sequence; the SAR image to be detected includes multiple sub-images, and the multiple sub-images are obtained by splitting the SAR image to be detected; Performing feature fusion on the first predicted feature map, the second predicted feature map, and the third predicted feature map through the intermediate network to obtain a first fused feature map, a second fused feature map, and a third fused feature map; wherein, the first fused feature map is processed by the first feature fusion unit, the second predicted feature map is processed by the second feature fusion unit, and the third fused feature map is processed by the third feature fusion unit; Based on the detection network, processing the first fused feature map, the second fused feature map, and the third fused feature map to determine the predicted bounding box information corresponding to the sub-image, and the predicted bounding box information is used to determine the detection target in the sub-image; Integrate the prediction box information corresponding to all sub-images of the SAR image to be detected to determine the detection targets in the SAR image to be detected.

2. The method according to claim 1, wherein The second feature fusion unit includes a fourth CBL module, a second merging module, a fifth CBL module, a third merging module, and a sixth CBL module connected in sequence. The second feature fusion unit further includes a seventh CBL module, a first upsampling module, and a second downsampling module; one end of the first upsampling module is connected to the second merging module, the other end of the first upsampling module is connected to one end of the seventh CBL module, the other end of the seventh CBL module is coupled to the third input end of the second feature fusion unit, one end of the second downsampling module is connected to the third merging module, and the other end of the second downsampling module is coupled to the second input end of the second feature fusion unit; The first feature fusion unit includes an eighth CBL module, a fourth merging module, and a ninth CBL module connected in sequence. The first feature fusion unit further includes a tenth CBL module and a second upsampling module. One end of the second upsampling module is connected to the fourth merging module, and the other end is connected to one end of the tenth CBL module. The other end of the tenth CBL module is coupled to the second input end of the first feature fusion unit.

3. The method according to claim 2, wherein The DSPP module includes an eleventh CBL module, a first branch module, a second branch module, a third branch module, a fifth merging module, a twelfth CBL module, and a summing module; one end of the eleventh CBL module is coupled to the input end of the DSPP module. One end of the first branch module, the second branch module, and the third branch module are respectively connected to the other end of the eleventh CBL module. The other ends of the first branch module, the second branch module, and the third branch module are respectively connected to one end of the fifth merging module. The other end of the fifth merging module is connected to one end of the twelfth CBL module. The other end of the twelfth CBL module is connected to one end of the summing module. The other end of the summing module is coupled to the output end of the DSPP module; wherein, the first branch module includes a thirteenth CBL module, a first DBL module, and a first max pooling layer connected in sequence; the second branch module includes a fourteenth CBL module, a second DBL module, and a second max pooling layer connected in sequence; the third branch module includes a fifteenth CBL module, a sixteenth CBL module, a third DBL module, and a third max pooling layer connected in sequence; the DBL module is formed by connecting an atrous convolutional layer, a batch normalization layer, and a LeakyRelu activation function layer in series.

4. The method according to claim 1, wherein The prediction box information includes the coordinates of the center point of the prediction box, the height of the prediction box, and the width of the prediction box.

5. The method according to claim 1, wherein The method further includes: The target detection model is trained based on a SAR image dataset using an artificial intelligence algorithm. The SAR image dataset includes multiple SAR images and prediction box information corresponding to the multiple SAR images respectively.

6. The method according to any one of claims 1 to 5, characterized in that, Integrating the prediction box information corresponding to all sub-images of the SAR image to be detected to determine the detection target in the SAR image to be detected includes: Converting the coordinates of the center points of the prediction boxes corresponding to all sub-images in the sub-images to the coordinates in the SAR image to be detected to determine the prediction box information corresponding to the SAR image to be detected, and the prediction box information corresponding to the SAR image to be detected is used to determine the detection target in the SAR image to be detected.

7. The method according to claim 6, wherein After converting the coordinates of the center points of the prediction boxes corresponding to all sub-images in the sub-images to the coordinates in the SAR image to be detected, the method further includes: Performing duplicate removal processing on the prediction boxes corresponding to the SAR image to be detected using a non-maximum suppression algorithm.

8. The method according to claim 1, wherein The method further includes: Based on the SAR image dataset, using a K-means clustering algorithm to determine the input parameters of the detection network, and the input parameters include the initial width and height of the prediction box.

9. An electronic device, characterized in that, It includes a processor and a memory coupled to the processor; the memory is used to store computer instructions, and when the electronic device runs, the processor executes the computer instructions stored in the memory so that the electronic device executes the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a computer program, and when the computer program runs on a computer, it executes the method according to any one of claims 1 to 8.

Images