A multi-scale SAR image target detection method, device, equipment and medium

Abstract

The application provides a multi-scale SAR image target detection method, device, equipment and medium, which utilizes a light-weight convolutional neural network to perform layer-by-layer feature extraction on a SAR image to obtain feature maps of the SAR image at multiple spatial resolutions; the feature maps at the multiple spatial resolutions are fused layer by layer in order from low to high spatial resolution to obtain multi-scale feature maps at the multiple spatial resolutions; the multi-scale feature map at the highest spatial resolution among the multi-scale feature maps at the multiple spatial resolutions is utilized to perform target detection to obtain a center point probability heat map of a target and a size of the target; and a position box of the target in the SAR image is determined according to the center point probability heat map of the target and the size of the target. The application adopts a light-weight network structure and directly utilizes a target center point probability heat map and a size to predict a target position, improves detection speed, and simultaneously performs supervised training based on multiple detection sub-models to ensure target detection accuracy.

Description

Technical Field

[0001] This invention relates to the field of image processing, and in particular to a method, apparatus, device, and medium for multi-scale SAR image target detection. Background Technology

[0002] Because SAR images are unaffected by lighting and weather, SAR-based ship detection has significant application prospects in maritime safety and management tasks. In recent years, with the development of deep neural network technology, deep neural network-based ship detection methods have begun to dominate this field; however, the large number of parameters in these networks leads to lengthy detection processes.

[0003] To address this issue, it is necessary to propose a lightweight detector with a multi-level auxiliary supervision structure. By simplifying the network structure, the detection speed can be improved, and multi-level supervision training can be used to enhance the detection accuracy, thereby achieving efficient multi-scale SAR ship target detection. Summary of the Invention

[0004] In view of the above problems, embodiments of the present invention provide a multi-scale SAR image target detection method, apparatus, device and medium to overcome or at least partially solve the above problems.

[0005] A first aspect of the present invention provides a multi-scale SAR image target detection method, comprising:

[0006] A lightweight neural network is used to extract features from SAR images layer by layer, resulting in feature maps of the SAR images at various spatial resolutions.

[0007] The feature maps of the various spatial resolutions are fused layer by layer in order of increasing spatial resolution to obtain multi-scale feature maps of various spatial resolutions.

[0008] Target detection is performed using the highest spatial resolution multi-scale feature map among the various spatial resolution multi-scale feature maps to obtain a probability heatmap of the target's center point and the target's size.

[0009] The bounding box of the target in the SAR image is determined based on the probability heatmap of the target's center point and the target's size.

[0010] Optionally, target detection using the multi-scale feature map with the highest spatial resolution among the multi-scale feature maps of various spatial resolutions is achieved through a pre-trained detection model corresponding to the detection sub-model with the highest spatial resolution; the target detection model includes multiple detection sub-models that correspond one-to-one with the various spatial resolutions; the method further includes:

[0011] The corresponding detection sub-model is trained using feature map samples of each spatial resolution as training data, and each feature map sample of each spatial resolution carries target label information;

[0012] Based on the total loss function value of the multiple detection sub-models, the model parameters of the multiple detection sub-models are updated to obtain the trained detection model.

[0013] Optionally, each of the plurality of detection sub-models includes a parallel center point probability prediction sub-model and a size prediction sub-model. The center point probability prediction sub-model is used to output a center point probability heatmap of the target, and the size coordinate prediction sub-model is used to output the width and height dimensions of the target. The method further includes:

[0014] The encoder converts feature map samples of each spatial resolution, carrying target labeling information, into the output format of the corresponding detection sub-model.

[0015] Based on the predicted values ​​and transformation results output by each of the multiple detection sub-models, the total loss function value of the center point and the total size loss value of the multiple detection models are calculated according to the following formulas;

[0016] The formula for calculating the total loss function value of the center points of the multiple detection models is as follows:

[0017]

[0018] Among them, y i,j ∈[0,1], is the ground truth value of the pixel at position (i,j) in the center point probability map sample. It is the predicted value of the pixel at position (i,j) in the center point probability map sample, N. pos This represents the total number of pixels with a calibration value of 1, where v is a network hyperparameter. [k] H [k] ) represent the width and height of the feature map at the k-th resolution, respectively;

[0019] The formula for calculating the total size loss value of the multiple detection models is as follows:

[0020]

[0021] Among them, y i,j,m and Let i and j represent the ground truth and predicted values ​​of the pixel at position (i,j) in the feature map sample for each spatial resolution, respectively.

[0022] The total loss function value of the multiple detection sub-models is calculated according to the following formula:

[0023] L total =L cls +λLsize

[0024] Here, λ is a hyperparameter used to balance the two losses.

[0025] Optionally, determining the bounding box of the target in the SAR image based on the probability heatmap of the target's center point and the size of the target includes:

[0026] The peak points are filtered out from the probability heatmap of the center point of the target by the decoder, and the center point of the target is selected according to a pre-set threshold.

[0027] Based on the center point of the target, the bounding box of the target in the SAR image is calculated according to the following formula:

[0028]

[0029]

[0030]

[0031]

[0032] Among them, (b) left ,b top ) and (b right ,b bottom () represents the coordinates of the top-left and bottom-right corners of the bounding rectangle of the target. This represents the rounding operation, (x, y) represents the coordinates of the center point of the target, and t w ,t h The size of the target is defined as follows:

[0033]

[0034] Among them, (W) b H b (W,H) represents the width and height of the bounding rectangle of the target, and (W,H) represents the width and height of the multi-scale feature map with the highest spatial resolution.

[0035] Optionally, feature map samples with multiple spatial resolutions are obtained according to the following steps:

[0036] The annotation information of the feature map samples at the original resolution is scaled multiple times to obtain feature map samples at various spatial resolutions.

[0037] A second aspect of the present invention also provides a multi-scale SAR image target detection device, comprising:

[0038] The feature extraction module is used to extract features from the SAR image layer by layer using a lightweight convolutional neural network to obtain feature maps of the SAR image at multiple spatial resolutions.

[0039] The feature fusion module is used to fuse the feature maps of the multiple spatial resolutions layer by layer in order of increasing spatial resolution to obtain multi-scale feature maps of multiple spatial resolutions.

[0040] The target detection module is used to perform target detection using the highest spatial resolution multi-scale feature map among the multi-scale feature maps of various spatial resolutions, and to obtain a probability heatmap of the target's center point and the size of the target;

[0041] The location determination module is used to determine the bounding box of the target in the SAR image based on the probability heatmap of the target's center point and the size of the target.

[0042] In some embodiments, the apparatus further includes:

[0043] The model training module is used to train the corresponding detection sub-model using feature map samples of each spatial resolution as training data. Each feature map sample of each spatial resolution carries target label information.

[0044] The parameter update module is used to update the model parameters of the multiple detection sub-models based on the total loss function value of the multiple detection sub-models, so as to obtain the trained detection model.

[0045] In some embodiments, the model training module includes:

[0046] The label information conversion module is used to convert the feature map samples carrying target label information at each spatial resolution into the output format of the corresponding detection sub-model through the encoder.

[0047] The loss function value calculation module is used to calculate the total loss function value of the center point and the total loss value of the size coordinates of the multiple detection models based on the predicted values ​​and transformation results output by the multiple detection sub-models.

[0048] A third aspect of this invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the multi-scale SAR image target detection method described in the first aspect of this invention.

[0049] A fourth aspect of this invention also provides a computer-readable storage medium storing a computer program / instructions thereon, which, when executed by a processor, implements the multi-scale SAR image target detection method described in the first aspect of this invention.

[0050] The embodiments of the present invention have the following advantages:

[0051] In this embodiment of the invention, a lightweight convolutional neural network is used to extract features from SAR images layer by layer, obtaining feature maps of multiple spatial resolutions. These feature maps are then fused layer by layer in ascending order of spatial resolution to obtain multi-scale feature maps of multiple spatial resolutions. The highest spatial resolution multi-scale feature map among these multi-scale feature maps is used for target detection, resulting in a target center point probability heatmap and the target's size coordinates. Based on the target center point probability heatmap and the target's size coordinates, the target's bounding box in the SAR image is determined. Because this embodiment of the invention performs multi-level feature fusion on feature maps of different spatial resolutions and uses the multi-scale feature map with the highest spatial resolution after feature fusion for target detection, it compensates for the incomplete feature extraction caused by a small number of parameters, simplifies the network structure, and predicts the target position by directly outputting the center point probability heatmap and the target's size coordinates, omitting prediction based on center point displacement, thus improving detection speed. Furthermore, during training, ground truth templates of different resolutions are used for supervised training in multiple detection sub-models, improving the accuracy of the detection model and ensuring target detection precision. Attached Figure Description

[0052] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a flowchart of the steps of a multi-scale SAR image target detection method provided in an embodiment of the present invention;

[0054] Figure 2 This is a schematic diagram of the network structure of a lightweight detector provided in an embodiment of the present invention;

[0055] Figure 3 This is a schematic diagram of a multi-level supervised training provided in an embodiment of the present invention;

[0056] Figure 4 This is a schematic diagram of the structure of a multi-scale SAR image target detection device provided in an embodiment of the present invention. Detailed Implementation

[0057] Exemplary embodiments of the present invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0058] This invention provides a multi-scale SAR image target detection method, referring to... Figure 1 , Figure 1 A flowchart of a multi-scale SAR image target detection method provided in this embodiment of the invention is shown below. Figure 1 As shown, the method includes:

[0059] Step S101: Use a lightweight convolutional neural network to extract features from the SAR image layer by layer to obtain feature maps of the SAR image with multiple spatial resolutions.

[0060] A lightweight convolutional neural network is used to extract features from SAR images layer by layer. Pooling layers are used to reduce the spatial resolution of the features in order to expand the visual receptive field and effectively extract global semantic information from the image, resulting in feature maps with multiple spatial resolutions. In this case, a convolutional block of one level extracts a corresponding feature map with spatial resolution.

[0061] In this embodiment, the network used for layer-by-layer feature extraction of SAR images is obtained by lightweight pruning of the traditional ResNet network structure, as shown in the following figure. Figure 2 As shown on the left, Figure 2 This is a schematic diagram of the network structure of a lightweight detector provided in this embodiment. Adjacent convolutional layers with the same spatial resolution are called a convolutional block, which is a level, denoted as Bn, where n is the spatial resolution of the features in the convolutional block of the current level. Compared with the traditional ResNet, in order to reduce network parameters, the number of channels in the convolutional blocks of each level is reduced. The number of channels in the convolutional blocks of each level (denoted as B512, B256, B128, B64, B32, B16) is set to 8, 16, 24, 32, 48 and 64 respectively. In order to maintain network depth, the convolutional blocks of the third and fourth levels are stacked three times, and the convolutional blocks of the second and fifth levels are stacked four times.

[0062] Step S102: In order of increasing spatial resolution, the feature maps of the various spatial resolutions are fused layer by layer to obtain multi-scale feature maps of various spatial resolutions.

[0063] In the feature maps of multiple spatial resolutions extracted in step S101, N feature maps of different spatial resolutions corresponding to N levels are selected for multi-level feature fusion processing. N is determined according to actual needs. The N feature maps of different spatial resolutions are input into three cascaded models for fusion processing, thus obtaining multi-scale feature maps of N spatial resolutions. The three cascaded models specifically include: the first model is a feature pyramid model, which integrates high-level semantic features into lower levels, that is, the semantic features in the low-spatial-resolution feature maps are sequentially integrated into the high-spatial-resolution feature maps layer by layer from low to high; the second model is an adaptive feature fusion model, which uses the network's self-learning ability to adjust the fusion weights of each level, that is, adjust the fusion weights of feature maps of different resolutions; the third model is also a feature pyramid model, used to enhance the sharing of high-level semantic features to lower levels, that is, further sharing the semantic features in the low-resolution feature maps to the high-resolution feature maps.

[0064] For example, such as Figure 2 As shown, feature maps with different spatial resolutions from four levels (level 1, level 2, level 3, and level 4) are selected for layer-by-layer feature fusion. The corresponding spatial resolutions are 256, 128, 64, and 32, respectively. The feature maps corresponding to the four levels are input into Model 1 (i.e., the feature pyramid model), Module 2 (i.e., the adaptive feature fusion model), and Module 3 (i.e., the feature pyramid model) for fusion processing. Finally, four multi-scale feature maps with different spatial resolutions are obtained. Among them, the multi-scale feature map with the highest spatial resolution corresponding to level 1 contains the richest semantic and spatial features.

[0065] In this embodiment, by fusing features of multiple spatial resolutions, the representation of target features in the multi-scale feature map is improved. In particular, the multi-scale feature map with the highest spatial resolution contains rich semantic and spatial features, which makes up for the problem of insufficient feature extraction caused by the small number of parameters due to the reduction of the number of channels in each convolutional block during the feature extraction process in step S101.

[0066] Step S103: Target detection is performed using the highest spatial resolution multi-scale feature map among the various spatial resolution multi-scale feature maps to obtain the target's center point probability heatmap and the target's size.

[0067] Since the multi-scale feature map with the highest spatial resolution possesses rich semantic and spatial features, it is used only for target detection. The target detection model performs target detection on the multi-scale feature map and outputs a probability heatmap of the target's center point and the target's size coordinates. The probability heatmap predicts the location of each target's center point in the multi-scale feature map, and the target's size coordinates represent the relationship between the size of each target's bounding box and the current feature map size.

[0068] Optionally, target detection is performed using the multi-scale feature map with the highest spatial resolution among the multi-scale feature maps of the various spatial resolutions. This is achieved by using a detection sub-model corresponding to the highest spatial resolution in a pre-trained detection model. The target detection model includes multiple detection sub-models that correspond one-to-one with the various spatial resolutions.

[0069] Multi-scale feature maps with different spatial resolutions at each level are input into the detection model for detection. Specifically, the multi-scale feature maps with different spatial resolutions for each level are input into the corresponding detection sub-network, with each multi-scale feature map at a different spatial resolution corresponding to a specific detection sub-model. Finally, the highest spatial resolution multi-scale feature map and its corresponding detection sub-model are used for target detection. For example, in... Figure 2 In this process, the multi-scale feature maps of level 1, level 2, level 3 and level 4 are respectively input into the detection sub-models of the same level in the detection model, and the target detection is performed using the level 1 corresponding to the multi-scale feature map with the highest spatial resolution, and the detection result is output using the detection sub-model corresponding to level 1.

[0070] The detection model used for object detection is a pre-trained model. Before performing object detection, the detection model needs to be trained, specifically including:

[0071] The corresponding detection sub-model is trained using feature map samples of each spatial resolution as training data, and each feature map sample of each spatial resolution carries target label information;

[0072] Based on the total loss function value of the multiple detection sub-models, the model parameters of the multiple detection sub-models are updated to obtain the trained detection model.

[0073] When training the object detection module, feature map samples of multiple spatial resolutions are input into the corresponding detection sub-models. Each object detection sub-model receives only one type of feature map sample, ensuring that the detection model can focus on the target region at each level. Simultaneously, the total loss function value of the object detection model (i.e., the total loss function value of all detection sub-models) is calculated, and the parameters of the entire object detection model are optimized based on this total loss function value. Because multiple detection sub-models are trained simultaneously, the detection model can simultaneously focus on target regions at different spatial resolutions, thus achieving mutual supervision among the multiple detection sub-models and ensuring the accuracy of the detection results.

[0074] The feature map samples with multiple spatial resolutions were obtained according to the following steps:

[0075] The annotation information of the feature map samples at the original resolution is scaled multiple times to obtain feature map samples at various spatial resolutions.

[0076] The spatial resolution of the feature map samples corresponds to the corresponding detection sub-model. Each feature map sample with a different spatial resolution contains target label information. Each feature map sample with a different spatial resolution is generated by directly scaling the label information of the original feature map sample to the spatial resolution size of the relevant level.

[0077] For example, such as Figure 3 As shown, Figure 3 This implementation provides a schematic diagram of multi-level supervised training, which uses four different spatial resolution feature map samples (GT_1, GT_2, GT_3, GT_4) to train four levels of detection sub-models. Each different spatial resolution feature map sample corresponds one-to-one with each level of detection sub-model, so that the network can focus on the target region at each level.

[0078] Optionally, each of the plurality of detection sub-models includes a parallel center point probability prediction sub-model and a size coordinate prediction sub-model. The center point probability prediction sub-model is used to output a center point probability heatmap of the target, and the size coordinate prediction sub-model is used to output the width and height dimensions of the target.

[0079] Each detection sub-model contains a parallel center point probability prediction sub-model and a size coordinate prediction sub-model. The center point probability prediction sub-model is used to output the target center point probability and contains a 3×3 convolutional layer with 8 channels and a 1×1 convolutional layer with 1 channel. The size coordinate prediction sub-model is used to output the target size coordinates and contains a 3×3 convolutional layer with 8 channels and a 1×1 convolutional layer with 2 channels.

[0080] The method further includes:

[0081] The encoder converts feature map samples of each spatial resolution, carrying target label information, into the output format of the corresponding detection sub-model.

[0082] The output format of each detection sub-model is three output values ​​(p) corresponding to each pixel in the feature map. c ,t w ,t h ), where p c This is the output value of the center point probability prediction sub-model, representing the probability that the current pixel is the center of the target; t w ,t h This is the output value of the size prediction sub-model, i.e., the size of the target. The transformation process is as follows:

[0083]

[0084] Among them, (W) b H b (W,H) represents the width and height of the bounding rectangle of the target, and (W,H) represents the width and height of the current feature map.

[0085] Based on the predicted values ​​and transformation results output by each of the multiple detection sub-models, the total loss function value of the center point and the total size loss value of the multiple detection models are calculated according to the following formulas;

[0086] The formula for calculating the total loss function value of the center points of the multiple detection models is as follows:

[0087]

[0088] Among them, y i,j ∈[0,1] is the true value at position (i,j) in the probability graph of the center point. N is the network prediction value for that point. pos This represents the total number of pixels with a calibration value of 1, where v is a network hyperparameter. [k] H [k] ) represent the width and height of the feature map in the k-th level, respectively;

[0089] The formula for calculating the total size loss value of the multiple detection models is as follows:

[0090]

[0091] Among them, y i,j,m and Let i and j represent the ground truth and predicted values ​​of the pixel at position (i,j) in the feature map sample for each spatial resolution, respectively.

[0092] The total loss function value of the multiple detection sub-models is calculated according to the following formula:

[0093] L total =L cls +λL size

[0094] Here, λ is a hyperparameter used to balance the two losses.

[0095] In this embodiment, feature map samples with different spatial resolutions are input to train the detection sub-networks at each level. This ensures that the target detection module can focus on the target region at each level. The parameters of the target detection module are optimized based on the total loss function value of multiple detection sub-models, allowing each level of detection sub-model to supervise each other and improve the accuracy of the detection model. When predicting the target, since the multi-scale feature map with the highest spatial resolution has rich semantic and spatial features, only the multi-scale feature map with the highest spatial resolution is used for target detection. The detection sub-model corresponding to the highest spatial resolution is then used for detection, outputting a target center probability heatmap and the target size.

[0096] Step S104: Determine the bounding box of the target in the SAR image based on the probability heatmap of the target's center point and the size of the target.

[0097] When performing target detection, the output from the detection model is a probability heatmap of the target's center point and the target's size. It also needs to be converted into the final detection result by the decoder, which marks the detected target with a rectangle of the corresponding size in the feature map.

[0098] Optionally, determining the bounding box of the target in the SAR image based on the probability heatmap of the target's center point and the size of the target includes:

[0099] A1: The peak point is filtered out in the probability heatmap of the center point of the target by the decoder, and the center point of the target is selected according to the preset threshold.

[0100] Based on the relationship between the probability values ​​of each peak point in the probability heatmap and the threshold, it is determined whether the peak point is the target center point. Specifically, the threshold is set to m. Points with a probability value greater than m are judged as target center points, and points with a probability value less than or equal to m are not target center points. Here, m is determined according to the actual situation.

[0101] A2: Based on the center point of the target, calculate the bounding box of the target in the SAR image according to the following formula:

[0102]

[0103]

[0104]

[0105]

[0106] Among them, (b) left ,b top ) and (b right ,b bottom () represents the coordinates of the top-left and bottom-right corners of the bounding rectangle of the target. This indicates a rounding operation, and (x,y) represents the coordinates of the center point of the target.

[0107] Based on the coordinates of the target center, the coordinates of the top-left and bottom-right corners of the target's circumscribed rectangle can be calculated using the above formula. Then, based on the coordinates of the target center and the top-left and bottom-right corners of the circumscribed rectangle, the target can be marked in the multi-scale feature map. Specifically, for example... Figure 2 As shown, the detected targets are marked with rectangular boxes in the multi-scale feature map.

[0108] In this embodiment, the target position is predicted directly by the position of the center point and the width and height of the target's bounding rectangle, omitting the traditional prediction based on the displacement of the center point. Moreover, it does not require a computationally complex non-maximum suppression algorithm, thus improving detection efficiency.

[0109] This invention also provides a multi-scale SAR image target detection device, referring to... Figure 4 , Figure 4 This is a schematic diagram of the structure of a multi-scale SAR image target detection device provided in an embodiment of this application, as shown below. Figure 4 As shown, the device includes:

[0110] The feature extraction module is used to extract features from the SAR image layer by layer to obtain feature maps of the SAR image at multiple spatial resolutions.

[0111] The feature fusion module is used to fuse the feature maps of the multiple spatial resolutions layer by layer in order of increasing spatial resolution to obtain multi-scale feature maps of multiple spatial resolutions.

[0112] The target detection module is used to perform target detection using the highest spatial resolution multi-scale feature map among the multi-scale feature maps of various spatial resolutions, and to obtain a probability heatmap of the target's center point and the size of the target;

[0113] The location determination module is used to determine the bounding box of the target in the SAR image based on the probability heatmap of the target's center point and the size of the target.

[0114] In one embodiment, the device further includes:

[0115] The model training module is used to train the corresponding detection sub-model using feature map samples of each spatial resolution as training data. Each feature map sample of each spatial resolution carries target label information.

[0116] The parameter update module is used to update the model parameters of the multiple detection sub-models based on the total loss function value of the multiple detection sub-models, so as to obtain the trained detection model.

[0117] In one embodiment, the model training module includes:

[0118] The label information conversion module is used to convert the feature map samples carrying target label information at each spatial resolution into the output format of the corresponding detection sub-model through the encoder.

[0119] The loss function value calculation module is used to calculate the total loss function value of the center point and the total size loss value of the multiple detection models based on the predicted values ​​and transformation results output by the multiple detection sub-models.

[0120] This invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor is configured to execute the multi-scale SAR image target detection method described in this embodiment.

[0121] This invention also provides a computer-readable storage medium storing a computer program that causes a processor to execute the multi-scale SAR image target detection method as described in this invention.

[0122] This invention provides a multi-scale SAR image target detection method, apparatus, device, and medium. The method includes: extracting features from a SAR image layer by layer using a lightweight convolutional neural network to obtain feature maps of the SAR image with multiple spatial resolutions; fusing the feature maps of the multiple spatial resolutions layer by layer in order of increasing spatial resolution to obtain multi-scale feature maps with multiple spatial resolutions; performing target detection using the highest spatial resolution multi-scale feature map among the multi-scale feature maps with multiple spatial resolutions to obtain a probability heatmap of the target's center point and the size coordinates of the target; and determining the bounding box of the target in the SAR image based on the probability heatmap of the target's center point and the size coordinates of the target. In this embodiment of the invention, feature maps with different spatial resolutions are fused at multiple levels, and the multi-scale feature map with the highest spatial resolution after feature fusion is used for target detection. This makes up for the problem of insufficient feature extraction caused by a small number of parameters, simplifies the network structure, and predicts the target position by directly outputting the center point probability heatmap and the target size coordinates during detection, omitting the prediction based on the center point displacement, thus improving the detection speed. At the same time, during the training process, ground truth templates of different resolutions are used for supervised training in multiple detection sub-models, which improves the accuracy of the detection model and thus ensures the target detection accuracy.

[0123] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0124] Embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, electronic devices, and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0125] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The function specified in one or more boxes.

[0126] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0127] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0128] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0129] The present invention provides a detailed description of a multi-scale SAR image target detection method, apparatus, device, and medium. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A multi-scale SAR image target detection method, characterized in that, The method includes: A lightweight neural network is used to extract features from SAR images layer by layer, resulting in feature maps of the SAR images at various spatial resolutions. The feature maps of the various spatial resolutions are fused layer by layer in order of increasing spatial resolution to obtain multi-scale feature maps of various spatial resolutions. Target detection is performed using the highest spatial resolution multi-scale feature map among the various spatial resolution multi-scale feature maps to obtain a probability heatmap of the target's center point and the target's size. Determining the bounding box of the target in the SAR image based on the target's center point probability heatmap and the target's size includes: The peak points are filtered out in the probability heatmap of the center point of the target by the decoder, and the center point of the target is selected according to a pre-set threshold. Based on the center point of the target, the bounding box of the target in the SAR image is calculated according to the following formula: in, and The coordinates represent the top-left and bottom-right corners of the bounding rectangle of the target. This indicates the rounding operation. This represents the coordinates of the center point of the target. , The size of the target is defined as follows: ， in, This represents the width and height of the bounding rectangle of the target. The width and height represent the multi-scale feature map with the highest spatial resolution. Calculate the coordinates of the upper left and lower right corners of the target's bounding rectangle according to the above formula, and then mark the target in the multi-scale feature map based on the coordinates of the target's center point and the coordinates of the upper left and lower right corners of the target's bounding rectangle. The process of filtering out peak points in the probability heatmap of the target's center point using a decoder and selecting the target's center point according to a pre-set threshold includes: setting the threshold to m, and determining points with a probability value greater than m as the target's center point, while points with a probability value less than m are not the target's center point.

2. The method according to claim 1, characterized in that, Target detection using the multi-scale feature map with the highest spatial resolution among the various spatial resolutions is achieved through a pre-trained detection model corresponding to the detection sub-model with the highest spatial resolution; the target detection model includes multiple detection sub-models that correspond one-to-one with the various spatial resolutions; the method further includes: The corresponding detection sub-model is trained using feature map samples of each spatial resolution as training data, and each feature map sample of each spatial resolution carries target label information; Based on the total loss function value of the multiple detection sub-models, the model parameters of the multiple detection sub-models are updated to obtain the trained detection model.

3. The method according to claim 2, characterized in that, Each of the multiple detection sub-models includes a parallel center point probability prediction sub-model and a size prediction sub-model. The center point probability prediction sub-model is used to output a center point probability heatmap of the target, and the size prediction sub-model is used to output the width and height dimensions of the target. The method further includes: The encoder converts feature map samples of each spatial resolution, carrying target labeling information, into the output format of the corresponding detection sub-model. Based on the predicted values ​​and transformation results output by each of the multiple detection sub-models, the total loss function value of the center point and the total size loss value of the multiple detection models are calculated according to the following formulas; The formula for calculating the total loss function value of the center points of the multiple detection models is as follows: in, , is the position of the center point in the probability map sample. The true value of the pixel at that location The position of the center point in the probability map sample The predicted value of the pixel at that location. This represents the total number of pixels with a calibration value of 1. It is a network hyperparameter. Let represent the width and height of the feature map at the k-th resolution, respectively; The formula for calculating the total size loss value of the multiple detection models is as follows: in, and These represent the positions of the feature map samples at each spatial resolution. The ground truth and predicted values ​​of the pixels at that location; The total loss function value of the multiple detection sub-models is calculated according to the following formula: in, It is a hyperparameter used to balance the two losses.

4. The method according to claim 2, characterized in that, Feature map samples with multiple spatial resolutions are obtained according to the following steps: The annotation information of the feature map samples at the original resolution is scaled multiple times to obtain feature map samples at various spatial resolutions.

5. A multi-scale SAR image target detection device, characterized in that, The device includes: The feature extraction module is used to extract features from the SAR image layer by layer to obtain feature maps of the SAR image at multiple spatial resolutions. The feature fusion module is used to fuse the feature maps of the multiple spatial resolutions layer by layer in order of increasing spatial resolution to obtain multi-scale feature maps of multiple spatial resolutions. The target detection module is used to perform target detection using the highest spatial resolution multi-scale feature map among the multi-scale feature maps of various spatial resolutions, and to obtain a probability heatmap of the target's center point and the size of the target; The location determination module is used to determine the bounding box of the target in the SAR image based on the probability heatmap of the target's center point and the size of the target. Specifically, it is used to: filter out peak points in the probability heatmap of the target's center point using a decoder, and select the center point of the target according to a preset threshold; and calculate the bounding box of the target in the SAR image based on the center point of the target according to the following formula: in, and The coordinates represent the top-left and bottom-right corners of the bounding rectangle of the target. This indicates the rounding operation. The coordinates of the center point of the target are indicated. , The size of the target is defined as follows: ， in, This represents the width and height of the bounding rectangle of the target. The width and height represent the multi-scale feature map with the highest spatial resolution. Calculate the coordinates of the upper left and lower right corners of the target's bounding rectangle according to the above formula, and then mark the target in the multi-scale feature map based on the coordinates of the target's center point and the coordinates of the upper left and lower right corners of the target's bounding rectangle. The process of filtering out peak points in the probability heatmap of the target's center point using a decoder and selecting the target's center point according to a pre-set threshold includes: setting the threshold to m, and determining points with a probability value greater than m as the target's center point, while points with a probability value less than m are not the target's center point.

6. The apparatus according to claim 5, characterized in that, The device further includes: The model training module is used to train the corresponding detection sub-model using feature map samples of each spatial resolution as training data. Each feature map sample of each spatial resolution carries target label information. The parameter update module is used to update the model parameters of the multiple detection sub-models based on the total loss function value of the multiple detection sub-models, so as to obtain the trained detection model.

7. The apparatus according to claim 6, characterized in that, The model training module includes: The label information conversion module is used to convert the feature map samples carrying target label information at each spatial resolution into the output format of the corresponding detection sub-model through the encoder. The loss function value calculation module is used to calculate the total loss function value of the center point and the total loss value of the size coordinates of the multiple detection models according to the predicted values ​​and transformation results output by the multiple detection sub-models, using the following formula.

8. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executed, implements the multi-scale SAR image target detection method as described in any one of claims 1-4.

9. A computer-readable storage medium, characterized in that, The stored computer program enables the processor to execute the multi-scale SAR image target detection method as described in any one of claims 1-4.

Images