A SAR image target detection method and device based on data augmentation

Abstract

The application provides a SAR image target detection method and device based on data augmentation, which comprises the following steps: adjusting the size of a SAR image to be tested and performing normalization processing; inputting the size of the SAR image to be tested after the normalization processing into a target detection model to obtain a predicted bounding box, a category probability and a confidence; through the method and device, the accuracy and efficiency of SAR image target detection are improved; the natural fusion of a target subject and a background is ensured, and the robustness of target detection is enhanced; through rotation, flipping and other data augmentation means, the diversity and richness of a data set are increased, and the generalization capability of the model in actual application is effectively improved; an aggregation-distribution mechanism is introduced, effective fusion of multi-scale features is realized, the information loss problem of a traditional YOLO series model in information fusion is solved, and the accuracy and efficiency of target detection are improved.

Description

Technical Field

[0001] This invention belongs to the field of polarimetric synthetic aperture radar (SAR) image technology, specifically relating to a SAR image target detection method and apparatus based on data augmentation. Background Technology

[0002] Synthetic Aperture Radar (SAR) images are widely used in military reconnaissance, disaster monitoring, and environmental assessment due to their high resolution and penetration capabilities. Their imaging principle is based on the relative motion between the radar and the target, using synthetic aperture technology to equate a small antenna aperture to a larger aperture, thus improving imaging resolution. SAR has all-weather, all-time operating capabilities, penetrating clouds, fog, and ground cover to provide high-resolution radar images. By detecting and analyzing SAR images, we can obtain information such as the target's location, shape, size, material, motion status, and surface background characteristics.

[0003] However, due to the high construction cost of SAR systems, the complexity of data processing technology, and the involvement of sensitive areas such as national security, SAR image data is extremely scarce. Therefore, using data augmentation methods is of great significance and practical value in addressing the problem of SAR data scarcity. Currently, multi-target detection datasets in land scenes are relatively scarce, especially the limited sample size of miniSAR datasets, which cannot meet the training requirements of deep learning target detection networks. Traditional data augmentation methods typically involve pasting tiled images of ground targets onto a large scene. This approach inevitably introduces rich original background information and noise, resulting in generated images that do not match reality, thus greatly limiting model performance. In contrast, avoiding background interference and ensuring that generated samples more closely resemble actual conditions is an effective way to improve the quality of data augmentation. This augmentation method not only increases the diversity of the dataset but also allows the model to focus on the essential features of the target during the learning process. By ensuring the authenticity and consistency of the data, the detection accuracy of the model in complex environments can be significantly improved, thus providing more reliable technical support for fields such as military industry.

[0004] SAR images often feature a coexistence of large scenes and small targets, posing a significant challenge to the accurate detection of small targets. While various image processing and deep learning models have been applied to target detection in SAR images, these methods are not specifically optimized for small-sized target detection. This leads to problems such as feature overload, missed detections, and false detections in complex scenes, especially when the target size is much smaller than the background or interfering objects.

[0005] To overcome this problem, improved networks for small target detection enable models to more effectively capture subtle features and extract characteristic information of small targets. This improved strategy not only enhances detection accuracy but also strengthens the model's adaptability in complex environments, providing more reliable technical support for military and security fields and ensuring target detection and response capabilities at critical moments, thus possessing extremely high application value.

[0006] In the prior art, patent application number 202410273172.X in reference document 1 provides a polarimetric SAR image classification method based on a diffusion model, which achieves the PolSAR image classification method based on diffusion data augmentation proposed in this invention. By constructing a diffusion data augmentation model and a convolutional neural network model, the feature matrix corresponding to a small number of selected labeled data is gradually transformed into Gaussian noise through a forward diffusion noise addition process. The noise prediction model learns the distribution of the added noise, and then the learned model parameters are used to perform inverse diffusion denoising on the random noise to generate pseudo-data. Finally, the convolutional neural network model trained on the reconstructed augmented dataset achieves pixel-level classification of PolSAR images. This method effectively avoids the problem of excessive reliance on labeled PolSAR image datasets in existing technologies, and increases the diversity of real data through the pseudo-dataset generated from random noise by the diffusion model, effectively improving the robustness and generalization ability of the polarimetric SAR image classification model. Although this comparative document 1 discusses image separation and fusion, while the method in this comparative document has certain advantages in enhancing data diversity, the data it generates mainly focuses on mimicking the stylistic features of real images and does not strictly constrain the physical causes and imaging mechanisms unique to SAR images.

[0007] In prior art document 2 (application number 202310062565.1), a method for intelligent fruit and vegetable recognition is provided. It achieves the following: 1. It utilizes a large dataset, and through data augmentation, efficiently and rationally utilizes training data, resulting in good model learning performance and high prediction accuracy. 2. It uses a deep learning model for fruit classification, saving labor and time costs. 3. The core principle of the YOLO network is based on CNN, or Convolutional Neural Network. It extracts information from multi-channel images through well-designed convolutional kernels and pooling layers, and then performs classification learning tasks through fully connected layers, adjusting the learning results according to the loss function. YOLO further divides the image into different regions, pre-prescribing possible object ranges before matching, thus improving the network's accuracy and speed. It also eliminates the step of adding extracted features to fully connected layers for regression, directly drawing the object detection region from the image information, greatly improving detection efficiency. Although it also provides data augmentation techniques such as rotation and flipping, the augmentation strategy of Comparison File 2 mainly processes the entire image and does not effectively control the spatial relationship between the target and the background in the image. This may cause the model to learn false features that are irrelevant to the recognition task, such as incorrectly using the spatial relationship between a certain type of background element or target as the basis for judgment.

[0008] In prior art document 3, patent application number 202311537095.6, a target detection method based on partial convolutional embedding and aggregation distribution mechanism is provided. The aggregation distribution mechanism is described, achieving the following: First, the target detection model of this invention improves detection efficiency while enhancing the detection capability of small targets and possessing higher overall accuracy. Second, the Fasternet in this invention includes a basic network submodule, a fast feature fusion submodule, and an efficient upsampling submodule. The basic network submodule includes convolutional layers, batch normalization layers, and activation function layers for feature extraction and nonlinear activation of images. The convolutional layers are responsible for learning local features in the image, the batch normalization layers are used to accelerate the training process and enhance the robustness of the network, and the activation function layers introduce nonlinear factors to increase the network's expressive power. The fast feature fusion submodule is responsible for fusing features from different levels, improving feature expressive power while maintaining speed. The efficient upsampling module is used to upsample the feature map to achieve accurate target location. Upsampling restores high-resolution feature maps to achieve accurate target location, improving positioning accuracy while maintaining speed. Third, the partial convolution in this invention utilizes the redundancy of feature maps, systematically applying regular convolutions to a portion of the input channels without affecting the remaining input channels. Therefore, partial convolution has lower FLOPs (total floating-point operations) and higher FLOPS (floating-point operations per second) than general network structures. By simultaneously reducing redundant computation and memory access, spatial features can be extracted more effectively. Fourth, each Fasternet module in this invention has a partial convolutional layer followed by two convolutions, forming an inverse residual architecture. This results in a larger number of channels in the intermediate layers and a shortcut connection to reuse input features. Furthermore, to maintain feature diversity and reduce latency, the normalization layer and activation layer are placed after the intermediate layers. However, the aggregation strategy in comparative document 3 simply stitches together multi-level feature maps after spatial scale alignment to form a feature aggregation map. This method does not fully fuse the stitched features, resulting in each channel still containing information from different levels, which cannot reasonably explain that this is a global feature (see line 0034 of its specification). Regarding the distribution method: Comparison file 3 uses channel splitting to distribute the aggregated feature maps. However, this strategy has the problems of simple structure and fragmented information: during the splitting process, the semantic continuity of the original feature maps is broken, resulting in incomplete or redundant information propagation. (See line 0037 of its specification for details). Summary of the Invention

[0009] The purpose of this invention is to overcome the above problems and provide a SAR image target detection method and apparatus based on data augmentation.

[0010] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0011] A SAR image target detection method based on data augmentation includes the following steps:

[0012] Adjust the size of the SAR image to be tested and perform normalization processing;

[0013] The normalized SAR image size is input into the target detection model to obtain the predicted bounding box, class probability, and confidence score; where,

[0014] The training method for the target detection model includes the following steps:

[0015] By placing diverse target subjects into background images containing labeled information, and employing an image fusion technique based on alpha blending, the target subjects and background images are naturally blended to generate diverse training data.

[0016] The processed training data is then fed into the improved YOLOv8 object detection network for forward propagation.

[0017] Calculate the loss function of the improved YOLOv8 object detection network, calculate the total loss function based on the loss function, and use the total loss function to backpropagate the training data to update the network weights until the training data loss approaches zero.

[0018] As a further improvement, generating the diverse target entities includes the following steps:

[0019] Preprocessed SAR image and accurate segmentation: The preprocessed SAR image is segmented into the target subject;

[0020] Perform data augmentation operations such as rotation and flipping on the segmented target subject to generate diverse target subjects.

[0021] As a further improvement, obtaining a background image containing annotation information includes the following steps:

[0022] A non-overlapping sliding window method is used to segment the SAR background image into regions, resulting in background images of different regions.

[0023] The background image of different regions is labeled using the ResNet18 network to obtain a background image containing the labeled information.

[0024] As a further improvement, the loss function includes classification loss, bounding box regression loss, and confidence loss.

[0025] As a further improvement, placing diverse target subjects into a background image containing labeled information includes the following steps:

[0026] Based on the binarization mask, a suitable background image region is selected as the candidate region for the target subject;

[0027] The obtained target object is placed in the corresponding candidate region according to its size.

[0028] As a further improvement, the image fusion technique based on alpha blending is used to naturally fuse the target subject with the background image, generating diverse training data, including the following steps:

[0029] Alpha blending image fusion technology is used to adjust the transparency between the target subject and the background image. The target subject and the background image are then naturally blended according to a weighted ratio to obtain diverse training data fused images. The formula for the weighted natural blending is as follows:

[0030] I output =αI foreground +(1-α)I background

[0031] In the formula, I foreground Represents the target subject image, I background The background image is represented by α, and the transparency of the target image is I. output This indicates the final generated image.

[0032] As a further improvement, the step of inputting the processed training data into the improved YOLOv8 object detection network for forward propagation includes the following steps:

[0033] The feature fusion method of the YOLOv8 object detection network is modified to an aggregation-distribution mode, resulting in an improved YOLOv8 object detection network.

[0034] The fused image is input into the improved YOLOv8 object detection network to obtain a multi-scale feature map;

[0035] The multi-scale feature map is input into the aggregation stage of the feature fusion module to obtain an enhanced multi-scale feature map.

[0036] The enhanced multi-scale feature maps are input into the improved YOLOv8 detection head. Each scale feature map is subjected to target classification and bounding box regression through at least two types of convolutional layers. The size of the feature map is changed based on the target classification and the bounding box regression to obtain a multi-scale feature map with a changed size.

[0037] The multi-scale feature map with the changed size is processed, including using non-maximum suppression to remove redundant bounding boxes, retaining the detection results with the highest confidence, and then filtering out the detection results with low confidence based on the confidence threshold.

[0038] As a further improvement, the step of inputting the multi-scale feature map into the aggregation stage of the feature fusion module to obtain an enhanced multi-scale feature map includes the following steps:

[0039] A local feature map is obtained by passing a feature map through a convolutional layer;

[0040] At least two clustered feature maps of equal size are passed through convolutional layers of the same number as the clustered feature maps to obtain at least two enhanced feature maps of equal size; wherein the enhanced feature maps include at least a first enhanced feature map and a second enhanced feature map;

[0041] The first enhanced feature map is processed by the Sigmoid function to obtain a weighting factor. The weighting factor, the first enhanced feature map and the local feature map are multiplied together to obtain a weighted local feature map.

[0042] The weighted local feature map is added to the second enhanced feature map to obtain a distributed feature map, which is then used as the enhanced multi-scale feature map.

[0043] As a further improvement, the calculation of the loss function of the improved YOLOv8 object detection network, and the calculation of the total loss function based on the loss function, includes the following steps:

[0044] Calculate the loss function of the improved YOLOv8 object detection network, which includes: classification loss, bounding box regression loss, and confidence loss;

[0045] The total loss function is calculated based on the weights of the classification loss, the bounding box regression loss, and the confidence loss; the formula for calculating the total loss function is as follows:

[0046] L total =α cls L cls +β box L box +λ con L con

[0047] In the formula, α cls β represents the weighting coefficients of the classification loss. box The weighting coefficients and λ of the bounding box regression loss con L is the weighting coefficient for confidence loss. total L is the total loss function; clsL is the classification loss function; box L is the bounding box regression loss function; con This is the confidence loss function.

[0048] On the other hand, the present invention also provides a SAR image target detection device based on data augmentation, the device including at least one processor and a memory storing instructions, which, when executed by at least one processor, implement the method in the above technical solution.

[0049] A computer-readable storage medium storing a computer program, which is executed by a processor to implement the method described in the steps of the above-described technical solution.

[0050] A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, it implements the method described in the steps of the above-described technical solution.

[0051] The beneficial effects of this invention are:

[0052] This method and apparatus, employing data augmentation techniques and a multi-scale feature fusion mechanism, significantly improves the accuracy and efficiency of target detection in SAR images. First, the U-Net network is used to accurately segment the target subject in the SAR image, ensuring natural integration between the target subject and the background and enhancing the robustness of target detection. Second, data augmentation techniques such as rotation and flipping are used to increase the diversity and richness of the dataset, effectively improving the model's generalization ability in practical applications. Finally, an aggregation-distribution mechanism is introduced to achieve effective fusion of multi-scale features, solving the information loss problem in information fusion of traditional YOLO series models, further improving the accuracy and efficiency of target detection. This invention not only improves the accuracy of target detection but also maintains high detection performance under complex backgrounds, demonstrating broad application prospects. The data augmentation method employed in this invention is based on the SARbake image rendering algorithm. Specifically, it uses a 3D CAD model of the target as input, combining SAR imaging geometry and electromagnetic scattering mechanisms to realistically reconstruct the target and shadow regions in spatial structure. By segmenting the target and shadows, key physical information in the original SAR image can be preserved. In contrast, while the Diffusion method described in the prior art (reference document 1) visually narrows the gap between simulated and real images, it does not address the physical constraints in the SAR image generation process. Modeling bundles carries the risk of artifacts, structural distortion, or failure to accurately reflect target features. Compared to the prior art document 2 in the background, the geometric transformations such as rotation and flipping employed in this invention are specifically targeted at the target domain. That is, they only augment the labeled or extracted target region without changing the background region. This approach effectively avoids confusion or misleading enhancement of the spatial relative relationships between the target and the background, as well as between the target and other interfering objects. Compared to the prior art document 3 in the background, after stitching feature maps of different levels, this invention further introduces multi-scale convolution operations (1×1, 3×3 convolution) to extract new combined features from the stitched features, enhancing the interaction between channels and contextual fusion. Furthermore, a point-by-point addition fusion strategy is adopted, enabling the generated feature map to truly integrate information from different levels, resulting in more consistent semantic expression and stronger representational capabilities. Regarding the distribution method: this invention employs a dual-path convolutional distribution mechanism, applying two convolutional kernels to the aggregated feature map to extract two complementary features: the first convolutional kernel extracts local features and then multiplies them with the original local features using a sigmoid function to explicitly enhance local details; the second convolutional kernel extracts global semantic features and adds them to the weighted local features, achieving dynamic fusion of local and global features. This preserves the fine-grained information of small targets while also considering the global semantic context. This addresses the issue of missing target information that may arise from relying solely on channel weighting in comparison files. Attached Figure Description

[0053] The present invention includes the following figures:

[0054] Figure 1 This is a flowchart of a SAR image target detection method based on data augmentation according to some specific embodiments of the present invention;

[0055] Figure 2 This is a flowchart of the aggregation part of the aggregation-distribution mechanism in the detection method of the present invention;

[0056] Figure 3 This is a flowchart of the distribution part of the aggregation-distribution mechanism in the detection method of the present invention;

[0057] Figure 4 A schematic diagram of a SAR image target detection device based on data augmentation according to the present invention. Detailed Implementation

[0058] The following embodiments further illustrate the content of the present invention, but should not be construed as limiting the present invention. Any modifications or substitutions made to the methods, steps, or conditions of the present invention without departing from the spirit and essence of the invention are within the scope of the present invention.

[0059] like Figure 1-3 As shown, this invention provides a SAR image target detection method based on data augmentation, characterized by the following steps:

[0060] Step S10: Adjust the size of the SAR image to be tested and perform normalization processing;

[0061] Step S11: Input the normalized SAR image size to be tested into the target detection model to obtain the predicted bounding box, class probability and confidence score;

[0062] For example, the process of using a data-augmented SAR image target detection model for SAR image target detection specifically includes the following steps:

[0063] Step 1: Adjust the size of the SAR image used for testing and normalize it to ensure effective adaptation to the model's input requirements.

[0064] Step 2: Input the test image into the trained model. The preprocessed image is then fed into the object detection network for forward inference. First, features are extracted from the image through the backbone network to generate multi-scale feature maps. Then, the feature fusion module, using an aggregation and distribution mechanism, fuses the multi-scale feature maps across layers, enhancing the correlation between target details and contextual information. Finally, based on the fused features, the detection head simultaneously predicts the target's bounding box coordinates (x, y, w, h), confidence score, and class probability, outputting a multi-scale prediction tensor.

[0065] Step 3: The model output undergoes post-processing to generate the final detection results. First, the predicted bounding boxes are filtered according to a preset confidence threshold to remove invalid detections with low confidence. Then, a non-maximum suppression algorithm is used to remove highly overlapping redundant boxes based on the intersection-union ratio (IU) threshold, retaining only the predicted box with the highest confidence for the same target. Finally, the processed results are parsed, outputting the class label, confidence score, and bounding box coordinates for each target.

[0066] The training method for the target detection model includes the following steps:

[0067] Step S20: Place diverse target subjects into a background image containing labeled information, and use an image fusion technique based on alpha mixing to naturally fuse the target subjects with the background image to generate diverse training data.

[0068] For example, firstly, based on the generated binarized mask, a suitable background region is selected as a candidate region for target placement. The target image obtained in step 3 is then placed into the selected background region according to its size. Next, alpha blending is used to achieve a natural fusion of the target and background. Alpha blending is a transparency-based image fusion method that adjusts the transparency between the target and the background to allow for a smooth transition and avoid obvious edge lines. Specific steps include:

[0069] 1) Define the Alpha channel: Each image can have its transparency defined using an Alpha channel. For the target subject image, an appropriate Alpha value needs to be set.

[0070] 2) Image weighted fusion: This method merges the background image and the target subject image according to a certain ratio. The formula is:

[0071] I output =αI foreground +(1-α)I background (1)

[0072] In formula (1), I foreground Represents the target subject image, I background Represents the background image, α is the transparency (alpha value) of the target image, I output This indicates the final generated image. During weighted blending, the transparency can be adjusted as needed to allow the subject to blend naturally into the background.

[0073] Step S21: Input the processed training data into the improved YOLOv8 object detection network for forward propagation;

[0074] For example, the processed dataset is input into an improved YOLOv8 object detection network for forward propagation. Compared to the original YOLOv8, the improved network modifies the feature fusion method to an aggregation-distribution mode. The specific process is as follows:

[0075] 1) Input the dataset processed in step 6 into the YOLOv8 object detection network. First, the input image is processed by the backbone network to extract features and generate multi-scale feature maps of sizes (128×H / 4×H / 4), (256×H / 8×W / 8), (512×H / 16×W / 16), and (1024×H / 32×W / 32).

[0076] 2) Input the multi-scale feature maps into the aggregation stage of the feature fusion module. The specific process is as follows:

[0077] i. Upsample the feature map of (128×H / 4×H / 4) to output a size of (128×H / 8×W / 8). Then, perform adaptive average pooling on the feature maps of (512×H / 16×W / 16) and (1024×H / 32×W / 32) to output sizes of (512×H / 8×W / 8) and (1024×H / 8×W / 8) respectively.

[0078] ii. The feature maps of (128×H / 8×W / 8), (256×H / 8×W / 8), (512×H / 8×W / 8), and (1024×H / 8×W / 8) are concatenated along the channel dimension to generate a high-resolution feature map with an output size of (1920H / 8×W / 8).

[0079] iii. Pass the spliced ​​feature map through a 1×1 convolutional layer to obtain a fused feature of (96×H / 8×W / 8).

[0080] iv. Pass the fused feature map through a batch normalization layer, a 3×3 convolutional layer (padding=1), and a 1×1 convolutional layer respectively. Each convolutional layer has a stride of 1 and is followed by a batch normalization function. The resulting feature maps are then summed, and the output feature map size is (96×H / 8×W / 8).

[0081] v. Pass the summed feature maps through a 1×1 convolutional layer to obtain clustered features.

[0082] 3) The multi-scale feature maps and the high-resolution feature maps after feature aggregation are input into the distribution stage of the feature fusion module. The specific process is as follows:

[0083] i. Pass the (256×H / 8×W / 8) feature map through a 1×1 convolutional layer to obtain a local feature map. Pass the (256×H / 8×W / 8) aggregated feature map through two 1×1 convolutional layers to obtain two enhanced feature maps of size (256×H / 8×W / 8).

[0084] ii. The first enhanced feature map is processed by the sigmoid function to obtain a weighting factor. The weighting factor is multiplied by the first enhanced feature map and the local feature map to obtain the weighted local feature map.

[0085] iii. The weighted local feature map is added to the second enhanced feature map to obtain the distribution feature map. At this time, the distribution feature map retains both the weighted local features and the clustered features containing features from other layers.

[0086] 4) The enhanced multi-scale feature maps are input into the YOLOv8 detection head. The feature maps at each scale are processed by 3×3 convolutional layers and 1×1 convolutional layers for target classification and bounding box regression, respectively, with output sizes of (C+5)×H / 8×W / 8), (C+5)×H / 16×W / 16), and (C+5)×H / 32×W / 32, respectively, where C is the number of classes and 5 represents the coordinates and confidence of the bounding box.

[0087] 5) Post-process the output of the detection head, including using non-maximum suppression to remove redundant bounding boxes, retaining the detection results with the highest confidence, and then filtering out the detection results with low confidence based on the confidence threshold.

[0088] Step S22: Calculate the loss function of the improved YOLOv8 object detection network, calculate the total loss function based on the loss function, and use the total loss function to backpropagate the training data to update the network weights until the training data loss approaches zero.

[0089] For example, calculating the loss function of an object detection network, YOLOv8's loss function consists of three parts: classification loss, bounding box regression loss, and confidence loss. The classification loss uses binary cross-entropy loss to calculate the difference between the predicted and true classes; the bounding box regression loss uses CIoULoss to measure the difference in position and shape between the predicted and true boxes; the confidence loss also uses binary cross-entropy loss to calculate the difference between the confidence score of the predicted box and the true value for whether the predicted box contains the object. The total loss is a weighted sum of the three, as shown in the formula:

[0090] L total =α cls L cls + β box L box +λ con L con (2)

[0091] In formula (2), α cls β box and λ con These are weighting coefficients used to balance the importance of the losses in different parts.

[0092] Backpropagation is performed using the total loss function to update the network weights, and the next training iteration continues.

[0093] Repeat steps S21 to S22 until the loss approaches zero and the network finally converges. Save the optimal model parameters from the training results.

[0094] Generating the diverse target entities includes the following steps:

[0095] Step S30: Preprocessing SAR image and accurate segmentation. The preprocessed SAR image is segmented into the target subject.

[0096] Step S31: Perform data augmentation operations such as rotation and flipping on the segmented target subject to generate diverse target subjects.

[0097] Obtaining a background image containing annotation information includes the following steps:

[0098] Step S40: Use a non-overlapping sliding window to segment the SAR background image into regions, and obtain background images of different regions;

[0099] Step S41: Use the ResNet18 network to annotate the background images of different regions to obtain background images containing annotation information.

[0100] In some embodiments of the present invention, the loss function includes classification loss, bounding box regression loss, and confidence loss.

[0101] The process of placing diverse target subjects onto a background image containing labeled information includes the following steps:

[0102] Step S200: Based on the binarized mask, select a suitable background image region as a candidate region for the target subject;

[0103] Step S201: Place the obtained target body in the corresponding candidate region of the target body according to its size.

[0104] The image fusion technique based on alpha blending is used to naturally fuse the target subject with the background image to generate diverse training data, including the following steps:

[0105] Step S202: Alpha blending image fusion technology is used to adjust the transparency between the target subject and the background image. The target subject and the background image are then naturally fused according to a weighted ratio to obtain diverse training data fused images; wherein, the formula for the weighted natural fusion is as follows:

[0106] I output =αI foreground +(1-α)I background

[0107] In the formula, I foreground Represents the target subject image, I background The background image is represented by α, and the transparency of the target image is I. output This indicates the final generated image.

[0108] The process of inputting the processed training data into the improved YOLOv8 object detection network for forward propagation includes the following steps:

[0109] Step S210: Modify the feature fusion method of the YOLOv8 object detection network to the aggregation-distribution mode to obtain the improved YOLOv8 object detection network;

[0110] Step S211: Input the fused image into the improved YOLOv8 object detection network to obtain a multi-scale feature map;

[0111] Step S212: Input the multi-scale feature map into the aggregation stage of the feature fusion module to obtain the enhanced multi-scale feature map;

[0112] Step S213: Input the enhanced multi-scale feature map into the improved YOLOv8 detection head. Each scale feature map is subjected to target classification and bounding box regression through at least two types of convolutional layers. The size of the feature map is changed based on the target classification and the bounding box regression to obtain a multi-scale feature map with a changed size.

[0113] Step S214: Process the multi-scale feature map with changed size, including using non-maximum suppression to remove redundant bounding boxes, retaining the detection results with the highest confidence, and then filtering out the detection results with low confidence based on the confidence threshold.

[0114] The step of inputting the multi-scale feature map into the aggregation stage of the feature fusion module to obtain the enhanced multi-scale feature map includes the following steps:

[0115] Step S2120: Obtain a local feature map by passing a feature map through a convolutional layer;

[0116] Step S2121: Pass at least two clustered feature maps of equal size through convolutional layers of the same number as the clustered feature maps to obtain at least two enhanced feature maps of equal size; wherein the enhanced feature maps include at least a first enhanced feature map and a second enhanced feature map;

[0117] Step S2122: After processing the first enhanced feature map with the Sigmoid function, a weighting factor is obtained. The weighting factor, the first enhanced feature map, and the local feature map are multiplied together to obtain a weighted local feature map.

[0118] Step S2123: Add the weighted local feature map to the second enhanced feature map to obtain a distributed feature map as the enhanced multi-scale feature map.

[0119] The calculation of the loss function of the improved YOLOv8 object detection network, and the calculation of the total loss function based on the loss function, includes the following steps:

[0120] Step S220: Calculate the loss function of the improved YOLOv8 object detection network, which includes: classification loss, bounding box regression loss, and confidence loss;

[0121] Step S221: Based on the weights of the classification loss, the bounding box regression loss, and the confidence loss, calculate the total loss function; wherein, the formula for calculating the total loss function is as follows:

[0122] L total =α cls L cls +β box L box +λ con L con

[0123] In the formula, α cls β represents the weighting coefficients of the classification loss. box The weighting coefficients and λ of the bounding box regression loss con L is the weighting coefficient for confidence loss. total L is the total loss function; cls L is the classification loss function; box L is the bounding box regression loss function; con This is the confidence loss function.

[0124] This invention presents an improved network for small target detection, enabling the model to more effectively capture subtle features and extract characteristic information of small targets. This improved strategy not only enhances detection accuracy but also strengthens the model's adaptability in complex environments, providing more reliable technical support for military and security fields. It ensures target detection and response capabilities at critical moments, demonstrating extremely high application value.

[0125] Example 1

[0126] Taking the MSTAR dataset collected by a laboratory SAR sensor platform as an example, the MSTAR dataset contains ten types of ground targets: BMP2, BTR70, T72, T62, BRDM2, BTR60, ZSU23 / 4, D7, ZIL131, and 2S1, as well as background images. These images were acquired using an X-band SAR system with a resolution of 0.3m × 0.3m. The ground target size is approximately 128 × 128 pixels, and the background image size is approximately 1400 × 1700 pixels.

[0127] The training process of a SAR image target detection model based on data augmentation includes the following steps:

[0128] Step 1: Read the original SAR image from the training and resize it to 100×100.

[0129] Step 2: Based on the annotated images generated by the SARBake algorithm, train the U-Net network. The U-Net network, through its encoder-decoder structure, can accurately segment the target subject and its shadow region. During training, the input is a SAR image, and the output is a segmentation mask of the target subject and its shadow. By minimizing the Dice coefficient as the loss function to optimize the model parameters, the U-Net network can accurately predict the complete segmentation mask for each input image, obtaining the segmented target subject image. The specific method is as follows:

[0130] 1) The original SAR image with a size of (3×100×100) is input into the U-Net encoder. After two layers of 3×3 convolution, ReLU activation, and batch normalization, a feature map with a size of (64×100×100) is output and then enters the subsequent encoding process.

[0131] 2) Input the feature map of size (64×100×100) into the encoding module, and finally output the feature map of size (512×12×12).

[0132] 3) Input the feature map of size (512×12×12) into the bottleneck layer, pass through two 3×3 convolutional layers, and after each convolutional layer, follow a batch normalization function and a ReLU function, and finally output a feature map of size (1024×6×6) to provide deep features for the decoder.

[0133] 4) Input the feature with size (1024×6×6) into the decoding module, and finally output the feature map with size (64×100×100).

[0134] 5) Input the features of size (64×100×100) into the output layer, and generate the final target body and shadow segmentation mask through a 1×1 convolutional layer. If it is a binary classification task, sigmoid activation is used, and if it is a multi-class classification task, softmax activation is used. The final output segmentation result is of size (3×100×100).

[0135] Step 3: Perform data augmentation operations such as rotation and flipping on the segmented target subject to generate diverse target subject images.

[0136] Step 4: Use non-overlapping sliding windows to segment the background image. First, determine the sliding window size to be 100×100, calculate the number of sliding windows in the horizontal and vertical directions, and divide the non-overlapping sliding windows sequentially starting from the upper left corner of the image.

[0137] Step 5: Input the background region obtained in Step 4 into the ResNet18 network for classification. The ResNet18 network extracts image features through residual modules and uses convolutional layers, batch normalization layers, and the ReLU activation function to learn deep features in the sliding window. Finally, the network outputs the probability of whether the region belongs to a "placeable region" or a "non-placeable region" through a fully connected layer. Classification is performed based on the output probability value. If the output probability is greater than a preset threshold, the region is judged as a "placeable region"; otherwise, it is judged as a "non-placeable region".

[0138] Step 6: First, based on the binarized mask generated in Step 5, select a suitable background region as a candidate region for target placement. The target image obtained in Step 3 is placed into the selected background region according to its size. Next, alpha blending is used to achieve a natural fusion of the target and background. Alpha blending is an image fusion method based on transparency, which adjusts the transparency between the target and the background to allow for a smooth transition and avoid obvious edge lines. Specific steps include:

[0139] 1) Define the Alpha channel: Each image can have its transparency defined using an Alpha channel. For the subject image, set an appropriate Alpha value: 0 for the background area and 1 for the subject.

[0140] 2) Image weighted fusion: This method merges the background image and the target subject image according to a certain ratio. The formula is:

[0141] I output =αI foreground +(1-α)I background (1)

[0142] In formula (1), I foreground Represents the target subject image, I background Represents the background image, α is the transparency (alpha value) of the target image, I output This indicates the final generated image. During weighted blending, the transparency can be adjusted as needed to allow the subject to blend naturally into the background.

[0143] Step 7: Input the dataset processed in Step 6 into the improved object detection network based on YOLOv8. The feature fusion method of this object detection network is modified to a cluster-distribution mode for forward propagation. The specific process is as follows:

[0144] 1) Resize the images in the dataset processed in step 6 to 1216×1216 and input them into the YOLOv8 object detection network. First, the input images are processed by the backbone network to extract features and generate multi-scale feature maps of sizes (128×304×304), (256×152×152), (512×76×76), and (1024×38×38).

[0145] 2) Input the multi-scale feature maps into the aggregation stage of the feature fusion module. The specific process is as follows:

[0146] i. Upsample the (128×304×304) feature map to output a size of (128×152×152). Apply adaptive average pooling to the (512×76×76) and (1024×38×38) feature maps respectively to output sizes of (512×152×152) and (1024×152×152).

[0147] ii. The feature maps of (128×304×304), (256×152×152), (512×152×152), and (1024×152×152) are stitched together along the channel dimension to generate a high-resolution feature map with an output size of (1920×152×152).

[0148] iii. Pass the spliced ​​feature map through a 1×1 convolutional layer to obtain a fused feature of (96×152×152).

[0149] iv. Pass the fused feature map through a batch normalization layer, a 3×3 convolutional layer (padding=1), and a 1×1 convolutional layer respectively. Each convolutional layer has a stride of 1 and is followed by a batch normalization function to obtain different feature maps. Then, add the features and output the feature map with a size of (96×152×152).

[0150] v. Pass the summed feature maps through a 1×1 convolutional layer to obtain clustered features.

[0151] 3) Input the multi-scale feature map and the high-resolution feature map after feature fusion into the distribution stage of the feature fusion module. The specific process is as follows:

[0152] i. Pass the (256×152×152) feature map through a 1×1 convolutional layer to obtain a local feature map. Pass the clustered feature map of size (256×152×152) through two 1×1 convolutional layers to obtain two enhanced feature maps of size (256×152×152).

[0153] ii. The first enhanced feature map is processed by the sigmoid function to obtain a weighting factor. The weighting factor is multiplied by the first enhanced feature map and the local feature map to obtain the weighted local feature map.

[0154] iii. The weighted local feature map is added to the second enhanced feature map to obtain the distribution feature map. At this time, the distribution feature map retains both the weighted local features and the clustered features containing features from other layers.

[0155] 4) The distributed multi-scale feature maps are input into the YOLOv8 detection head. Each scale feature map is processed by a 3×3 convolutional layer and a 1×1 convolutional layer for target classification and bounding box regression, with output sizes of (15×152×152), (15×76×76), and (15×38×38), respectively.

[0156] 5) Post-process the output of the detection head, including using non-maximum suppression to remove redundant bounding boxes, retaining the detection results with the highest confidence, and then filtering out the detection results with low confidence based on the confidence threshold.

[0157] Step 8: Calculate the loss function of the object detection network. The YOLOv8 loss function consists of three parts: classification loss, bounding box regression loss, and confidence loss. The classification loss uses binary cross-entropy loss to calculate the difference between the predicted and true categories; the bounding box regression loss uses CIoULoss to measure the difference in position and shape between the predicted and true bounding boxes; the confidence loss also uses binary cross-entropy loss to calculate the difference between the confidence score of the predicted box and the true value for whether the predicted box contains the object. The total loss is the weighted sum of these three losses, as shown in the formula:

[0158] L total =α cls L cls + β box L box +λ con L con (2)

[0159] Where α cls β box and λ con This is a weighting coefficient used to balance the importance of different loss components. Set α... cls =0.5, β box =7.5, λ con =1.5.

[0160] Step 9: Based on the overall loss function L total Perform backpropagation, update the network weights, and continue with the next training iteration.

[0161] Step 10: Repeat steps 7 through 9 until the loss approaches zero and the network finally converges. Training is performed using the official training parameters provided by Ultralytics, with an initial learning rate of 0.01. Training lasts for a total of 100 epochs. Every 20 epochs, the learning rate is decayed by a decay factor of 0.1, as follows:

[0162]

[0163] The training process is optimized using momentum (momentum = 0.937) and weight decay (weight_decay = 0.0005). Network training continues until the loss approaches zero and convergence occurs. Finally, the model parameters are saved at the end of training.

[0164] The process of using a data-augmented SAR image target detection model for SAR image target detection specifically includes the following steps:

[0165] Step 11: Adjust the size of the SAR image used for testing and normalize it to ensure effective adaptation to the model's input requirements.

[0166] Step 12: Input the test image into the trained model. The preprocessed image is then fed into the object detection network for forward inference. First, features are extracted from the image through the backbone network to generate multi-scale feature maps. Then, the feature fusion module, using an aggregation and distribution mechanism, fuses the multi-scale feature maps across layers, enhancing the correlation between target details and contextual information. Finally, based on the fused features, the detection head simultaneously predicts the target's bounding box coordinates (x, y, w, h), confidence score, and class probability, outputting a multi-scale prediction tensor.

[0167] Step 13: The model output undergoes post-processing to generate the final detection results. First, the predicted bounding boxes are filtered according to a preset confidence threshold to remove invalid detections with low confidence. Then, a non-maximum suppression algorithm is used to remove highly overlapping redundant boxes based on the intersection-union ratio (IU) threshold, retaining only the predicted box with the highest confidence for the same target. Finally, the processed results are parsed, and the class label, confidence score, and bounding box coordinates for each target are output.

[0168] Example 2

[0169] A SAR image target detection method based on data augmentation, including the model training process and the process of using the model to perform SAR image target detection:

[0170] I. The training process of the SAR image target detection model based on data augmentation includes the following steps:

[0171] Step 1: Preprocess the original SAR images used for training by adjusting the image size.

[0172] Step 2: Use the U-Net network to accurately segment the target subject in the SAR image and obtain the segmentation mask of the target subject and its shadow region.

[0173] Step 3: Perform data augmentation operations such as rotation and flipping on the segmented target subject to generate diverse target subject images.

[0174] Step 4: Use a non-overlapping sliding window to segment the background image into regions.

[0175] Step 5: Use the ResNet18 network to classify the background region, determine which regions are suitable for placing the target, and generate a binarized mask.

[0176] Step 6: Place the enhanced target image from Step 3 into a suitable background area, and use an alpha-based image fusion technique to naturally blend the target image with the background image, generating diverse training data.

[0177] Step 7: Input the dataset processed in Step 6 into the object detection network containing the aggregation-distribution mechanism for forward propagation.

[0178] Step 8: Calculate the loss function of the object detection network, including classification loss, bounding box regression loss, and confidence loss.

[0179] Step 9: Use the total loss function for backpropagation, update the network weights, and continue the next training iteration.

[0180] Step 10: Repeat steps 8 through 10 until the loss approaches zero and the network finally converges. Save the optimal model parameters from the training results.

[0181] II. The process of applying the data augmentation-based SAR image target detection model to SAR image target detection specifically includes the following steps:

[0182] Step 11: Adjust the size of the original SAR image used for testing and normalize it.

[0183] Step 12: Input the test image into the trained object detection model to obtain the predicted bounding box, class probability, and confidence score.

[0184] This method and apparatus, employing data augmentation techniques and a multi-scale feature fusion mechanism, significantly improves the accuracy and efficiency of target detection in SAR images. First, the U-Net network is used to accurately segment the target subject in the SAR image, ensuring natural integration of the target subject with the background and enhancing the robustness of target detection. Second, data augmentation techniques such as rotation and flipping are used to increase the diversity and richness of the dataset, effectively improving the model's generalization ability in practical applications. Finally, an aggregation-distribution mechanism is introduced to achieve effective fusion of multi-scale features, solving the information loss problem in traditional YOLO series models during information fusion, further improving the accuracy and efficiency of target detection. This invention not only improves the accuracy of target detection but also maintains high detection performance in complex backgrounds, demonstrating broad application prospects.

[0185] In some specific embodiments, such as Figure 4 As shown, a SAR image target detection device based on data augmentation is disclosed. The device includes at least one processor and a memory storing instructions to implement any of the above-described SAR image target detection methods based on data augmentation.

[0186] In some specific embodiments, a computer-readable storage medium stores a computer program that is executed by a processor to implement the methods of the steps in the above-described technical solutions.

[0187] In some specific embodiments, a computer program product includes a computer program, which, when executed by a processor, implements the steps in the above-described technical solution.

[0188] The embodiments and functional operations of the subject matter described in this specification can be implemented in the following ways: digital electronic circuits, tangibly implemented computer software or firmware, computer hardware, including the structures disclosed in this specification and their equivalents, or combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, that is, one or more modules of computer program instructions encoded on one or more tangible non-transitory program carriers, for execution by a data processing device or to control the operation of the data processing device.

[0189] Alternatively or additionally, program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are then generated as coded information to be transmitted to an appropriate receiver device executed by data processing equipment. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or one or more combinations of the above.

[0190] Computer programs (which may also be referred to or described as programs, software, software applications, modules, software modules, scripts, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be expanded in any form, including as standalone programs or as modules, components, subroutines, or other units suitable for use in a computing environment. A computer program may, but must not, correspond to a file in a file system. A program may be stored as a portion of a file containing other programs or data, for example, as one or more scripts in a markup language document; in a single file dedicated to the related program; or in multiple co-files, for example, a file storing one or more modules, subroutines, or code portions. A computer program can be expanded to execute on one or more computers located in one place or distributed across multiple locations and interconnected via a communication network.

[0191] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

[0192] Although the present invention has been described in detail above with general descriptions, specific embodiments, and experiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. A SAR image target detection method based on data augmentation, characterized in that, Includes the following steps: Adjust the size of the SAR image to be tested and perform normalization processing; The normalized SAR image size is input into the target detection model to obtain the predicted bounding box, class probability, and confidence score; where, The training method for the target detection model includes the following steps: By placing diverse target subjects into background images containing labeled information, and employing an image fusion technique based on alpha blending, the target subjects and background images are naturally blended to generate diverse training data. The processed training data is then fed into the improved YOLOv8 object detection network for forward propagation. Calculate the loss function of the improved YOLOv8 object detection network, calculate the total loss function based on the loss function, and use the total loss function to backpropagate the training data to update the network weights until the training data loss approaches zero. The process of inputting the processed training data into the improved YOLOv8 object detection network for forward propagation includes the following steps: The feature fusion method of the YOLOv8 object detection network is modified to an aggregation-distribution mode, resulting in an improved YOLOv8 object detection network. The fused image is input into the improved YOLOv8 object detection network to obtain a multi-scale feature map; The multi-scale feature map is input into the aggregation stage of the feature fusion module to obtain an enhanced multi-scale feature map. The enhanced multi-scale feature maps are input into the improved YOLOv8 detection head. Each scale feature map is subjected to target classification and bounding box regression through at least two types of convolutional layers. The size of the feature map is changed based on the target classification and the bounding box regression to obtain a multi-scale feature map with a changed size. The multi-scale feature map with the changed size is processed, including using non-maximum suppression to remove redundant bounding boxes, retaining the detection results with the highest confidence, and then filtering out the detection results with low confidence based on the confidence threshold. The step of inputting the multi-scale feature map into the aggregation stage of the feature fusion module to obtain the enhanced multi-scale feature map includes the following steps: A local feature map is obtained by passing a feature map through a convolutional layer; At least two clustered feature maps of equal size are passed through convolutional layers of the same number as the clustered feature maps to obtain at least two enhanced feature maps of equal size; wherein the enhanced feature maps include at least a first enhanced feature map and a second enhanced feature map; The first enhanced feature map is processed by the Sigmoid function to obtain a weighting factor. The weighting factor, the first enhanced feature map and the local feature map are multiplied together to obtain a weighted local feature map. The weighted local feature map is added to the second enhanced feature map to obtain a distributed feature map, which is then used as the enhanced multi-scale feature map.

2. The SAR image target detection method based on data augmentation according to claim 1, characterized in that, Generating the diverse target entities includes the following steps: Preprocessed SAR image and accurate segmentation: The preprocessed SAR image is segmented into the target subject; The segmented target subject is subjected to data augmentation operations to generate diversified target subjects; the data augmentation operations are rotation and flipping.

3. The SAR image target detection method based on data augmentation according to claim 1, characterized in that, Obtaining the background image containing annotation information includes the following steps: A non-overlapping sliding window method is used to segment the SAR background image into regions, resulting in background images of different regions. The background image of different regions is labeled using the ResNet18 network to obtain a background image containing the labeled information.

4. The SAR image target detection method based on data augmentation according to claim 1, characterized in that, The loss function includes classification loss, bounding box regression loss, and confidence loss.

5. The SAR image target detection method based on data augmentation according to claim 1, characterized in that, The process of placing diverse target subjects onto a background image containing labeled information includes the following steps: Based on the binarization mask, a suitable background image region is selected as the candidate region for the target subject; The obtained target object is placed in the corresponding candidate region according to its size.

6. The SAR image target detection method based on data augmentation according to claim 1, characterized in that, The image fusion technique based on alpha blending is used to naturally fuse the target subject with the background image to generate diverse training data, including the following steps: Alpha blending image fusion technology is used to adjust the transparency between the target subject and the background image. The target subject and the background image are then naturally blended according to a weighted ratio to obtain diverse training data fused images. The formula for the weighted natural blending is as follows: In the formula, I foreground Represents the target subject image. I background Represents the background image. α It refers to the transparency of the target image. I output This indicates the final generated image.

7. The SAR image target detection method based on data augmentation according to claim 1, characterized in that, The calculation of the loss function of the improved YOLOv8 object detection network, and the calculation of the total loss function based on the loss function, includes the following steps: Calculate the loss function of the improved YOLOv8 object detection network, which includes: classification loss, bounding box regression loss, and confidence loss; The total loss function is calculated based on the weights of the classification loss, the bounding box regression loss, and the confidence loss; the formula for calculating the total loss function is as follows: In the formula, α cls These are the weighting coefficients for the classification loss; β box The weighting coefficients of the bounding box regression loss and λ con The weighting coefficients for confidence loss; This is the total loss function; The classification loss function; The bounding box regression loss function; This is the confidence loss function.

8. A SAR image target detection device based on data augmentation, the device comprising at least one processor and a memory storing instructions, characterized in that, When the instruction is executed by at least one processor, a SAR image target detection method based on data augmentation as described in any one of claims 1-7 is implemented.

Images