Data enhancement method for deep learning-based SAR image target rotation box detection

Abstract

The data enhancement method based on the deep learning SAR image target rotation frame detection first randomly rotates or flips each target in a single image at an arbitrary angle, obtains a single image after data enhancement, then performs the above operation on all images in the data set, that is, a data set after the data enhancement operation is obtained, the neural network is trained by using the data set after the data enhancement, then the trained neural network is used for detecting the target in the SAR image, and the final detection result is output. The application expands the number of sample images under the same scene under the condition that the sample is limited, so that the neural network learns more about the characteristics of the target, and finally improves the detection accuracy.

Description

Technical Field

[0001] This invention belongs to the field of intelligent processing of satellite remote sensing information, and in particular relates to SAR remote sensing data processing, specifically to a data augmentation method for rotating bounding box detection of targets in SAR images based on deep learning. Background Technology

[0002] Currently, when training a deep learning model, the neural network learns the features of the input data and then continuously adjusts and optimizes the network parameters to achieve the optimal point with the lowest model loss. Mainstream neural networks require training on tens of thousands of image datasets to obtain good output results. However, when the amount of available data is limited or the image dataset is too small, directly using this limited data to train the network often leads to overfitting, or the network may be unable to learn sufficient features from the limited data, resulting in the network's output on the test data failing to achieve the desired effect.

[0003] In the absence of a large amount of original training data, data augmentation is a solution that can effectively address the problem of insufficient data to a certain extent. For deep learning models in computers, the same image becomes a new image after some modifications, and the computer can continue to learn from its features. The essence of data augmentation is to increase the amount of data by introducing prior knowledge, thereby improving the generalization ability of the model.

[0004] The patent "Data-Driven Heterogeneous Multi-Target Intelligent Detection Method and System" (CN202110000652.5) ​​involves data augmentation methods, but its data augmentation methods only include conventional transformations of image scale and color, such as rotation, scaling, translation, and contrast adjustment. These methods transform an image at the overall level, without changing the shape or distribution of the targets in the image. The neural network learns more about the overall features of the image during the above data augmentation process, and learns very limited features about individual targets. Summary of the Invention

[0005] The technical problem solved by this invention is to overcome the shortcomings of the prior art and propose a data augmentation method for target rotation box detection based on deep learning SAR images. This method fully expands the morphological distribution of targets in SAR images, enriches the features learned by the neural network for individual targets, and lays the foundation for subsequent high-precision SAR image target detection.

[0006] The solution of the present invention is:

[0007] In a first aspect, the present invention provides a data augmentation method for target rotation box detection based on deep learning SAR images, the method comprising the following steps:

[0008] S1: In the original image, obtain the coordinates (x, y) of the center point of the target calibration box and the length h and width w of the calibration box;

[0009] S2: Extract the image within the outer circle of the calibration frame, denoted as Cut. circle ;

[0010] S3: Cut circle The image Cut is obtained by randomly rotating the circle by any angle with the center as the rotation point. circle_r ;

[0011] S4: Cut circle_r Align the center of the circle with (x,y) and cut the circle. circle_r Place it within the outer circle of the original calibration box to obtain an enhanced image for the current target;

[0012] S5: For each target in a single original image, repeat S1-S4 to obtain a data-enhanced image for each target;

[0013] S6: Perform S5 on all images in the original dataset to obtain a new dataset after data augmentation.

[0014] S7: Use the dataset after data augmentation to train a deep learning-based target detection model, and use the trained model to detect targets in SAR images.

[0015] Preferably, in step S2, the radius of the circumcircle corresponding to the calibration frame is calculated using h,w, and the center of the circumcircle is obtained by combining (x,y), thereby obtaining the range of the circumcircle.

[0016] Preferably, in step S1, the coordinates (x, y) of the center point of the target calibration box and the length h and width w of the calibration box are obtained using the target's annotation information file.

[0017] Preferably, the target annotation information file is a YAML or XML file.

[0018] Secondly, regarding the data augmentation method for target rotation box detection based on deep learning in SAR images, the steps of this method include:

[0019] S1: In the original image, obtain the coordinates (x, y) of the center point of the target calibration box and the length h and width w of the calibration box;

[0020] S2: Extract the image within the calibration box. The extracted image is a rectangular region containing a single target, denoted as Cut.rectangle ;

[0021] S3: Cut rectangle Flipping along its major or minor axis respectively yields the corresponding Cut. rectangle_x Or Cut rectangle_y The major axis refers to the axis passing through Cut. rectangle The axis whose center is parallel to its long side, the minor axis is the axis passing through Cut. rectangle The center of the axis is parallel to its shorter side;

[0022] S4: Cut rectangle_x Or Cut rectangle_y Align the center of the cut with the original calibration frame center point coordinates (x, y), and then... rectangle_x Or Cut rectangle_y Put it back in its original position;

[0023] S5: For each target in a single original image, repeat S1-S4 to obtain a data-enhanced image for each target;

[0024] S6: Perform S5 on all images in the original dataset to obtain a new dataset after data augmentation.

[0025] S7: Train a deep learning-based target detection model using the new dataset, and use the trained model to detect targets in SAR images.

[0026] Preferably, in step S1, the coordinates (x, y) of the center point of the target calibration box and the length h and width w of the calibration box are obtained using the target's annotation information file.

[0027] Preferably, the target annotation information file is a YAML or XML file.

[0028] Thirdly, a terminal device includes:

[0029] Memory, used to store at least one instruction executed by a processor;

[0030] A processor is configured to execute instructions stored in memory to perform the methods described in the first or second aspect above.

[0031] Fourthly, a computer-readable storage medium storing computer instructions that, when executed on a computer, cause the computer to perform the methods described in the first or second aspect.

[0032] The advantages of this invention compared to the prior art are:

[0033] (1) The method of this invention is different from the conventional method of performing relevant scale transformation or color transformation on the whole image. Instead, it transforms the shape and position orientation of a single target in the image. Compared with traditional data augmentation methods, this invention can effectively expand the diversity of samples. Because the rotation angle and flip are random, a large number of data samples can be expanded to change and expand the target shape and target distribution, enriching the features learned by the neural network for individual targets as much as possible, laying the foundation for subsequent high-precision SAR image target detection.

[0034] (2) The data augmentation method proposed in this invention can effectively expand the original dataset when SAR image data is insufficient. The proposed method is for deep learning-based rotating box target detection, which can effectively improve the detection accuracy of related target detection neural networks such as Fast R-CNN and RetinaNet in SAR image target detection. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the data augmentation method 1 proposed in this invention;

[0036] Figure 2 The following are examples of the results of the data augmentation method 1 proposed in this invention, where (a) is the original image, (b) is the image within the outer circle of the extracted calibration box, (c) is the image after random rotation of (b), and (d) is the augmented image.

[0037] Figure 3 This is a schematic diagram of the data augmentation method 2 proposed in this invention;

[0038] Figure 4 The following are examples of the results of the data augmentation method 2 proposed in this invention, where (a) is the original image, (b) is the data-augmented image after the target is flipped along the major axis, and (c) is the data-augmented image after the target is flipped along the minor axis. Detailed Implementation

[0039] The present invention will be further described below with reference to the embodiments.

[0040] This invention proposes a data augmentation method for rotating bounding box target detection in SAR images based on deep learning. This method can generate more samples with different morphological distributions from the same image, thereby generating more images with only a small number of SAR images. Compared with traditional methods of scaling and color transformation, the two methods of this invention can more effectively expand the shortcomings of target morphology in the samples, so that the relevant neural network can better detect targets at different angles in the image, and thus obtain better SAR image rotating bounding box target detection results.

[0041] The specific working principle is as follows:

[0042] First, the data augmentation method proposed in this invention is applied to each target in a single image to obtain a data-augmented single image. Then, the above operation is performed on all images in the dataset to obtain a dataset after data augmentation. The neural network is trained using the data-augmented dataset, and then the trained neural network is used to detect targets in SAR images to output the final detection results.

[0043] This invention provides a data augmentation method for target rotation box detection in SAR images based on deep learning. Method 1 achieves this by randomly rotating the target in the SAR image at an arbitrary angle, while Method 2 achieves this by randomly flipping the target in the SAR image. This expands the angular distribution of the target in the SAR image, thereby increasing the richness of the training samples. This allows the neural network to be effectively trained on the augmented image, thus improving the detection accuracy of the neural network for rotating box detection of SAR targets.

[0044] This invention discloses a data augmentation method for rotating bounding box detection of targets in SAR images based on deep learning, the form of which is shown below. Figure 1 The data augmentation method involved in the invention follows the following process:

[0045] S1: Using the target's label information, the coordinates (x, y) of the target's bounding box center point, as well as the length h and width w of the bounding box, can be obtained, such as... Figure 1 As shown. Obtain the coordinates (x, y) of the center point of the target calibration box and the h, w of the calibration box. This information can be obtained from the original annotation information file, such as a yaml or xml file.

[0046] S2: Original image Figure 2 Taking (a) as an example, the image within the outer circle of the calibration box is extracted and denoted as Cut. circle ,like Figure 2 As shown in (b). The image within the circumcircle of the calibration frame is extracted. The radius of the circumcircle corresponding to the calibration frame can be calculated using h and w. Combined with (x, y), the center of the circumcircle can be obtained. Using these two pieces of information, the image within the circular region can be obtained. This image within the circular region is extracted and denoted as Cut. circle .

[0047] S3: Cut circle The image Cut is obtained by randomly rotating the circle by any angle with the center as the rotation point. circle_r ,like Figure 2 As shown in (c);

[0048] S4: Cut circle_rAlign the center of the circle with (x,y) to obtain a new enhanced image, such as... Figure 2 As shown in (d);

[0049] S5: For each target in a single image, repeat S1-S4 to obtain a data-enhanced image for each target;

[0050] S6: Perform S5 on all images in the original dataset to obtain a new dataset after data augmentation.

[0051] S7: Use the data-augmented dataset to train a deep learning-based object detection model to detect objects in images.

[0052] This invention discloses a data augmentation method for rotating bounding box detection of targets in SAR images based on deep learning, the form of which is shown below. Figure 3 The data augmentation method involved in the invention follows the following process:

[0053] S1: Using the target's label information, the coordinates (x, y) of the target's bounding box center point, as well as the length h and width w of the bounding box, can be obtained, such as... Figure 3 As shown. Obtain the coordinates (x, y) of the center point of the target calibration box and the h, w of the calibration box. This information can be obtained from the original annotation information file, such as a yaml or xml file.

[0054] S2: Original image Figure 4 Taking (a) as an example, the image within the calibration box is extracted and denoted as Cut. rectangle .

[0055] S3: Cut rectangle Randomly select to flip along the major or minor axis to obtain Cut. rectangle_x Or Cut rectangle_y As shown in the annotation in the figure;

[0056] S4: Cut rectangle_x Or Cut rectangle_y Align the center of the cut with the original calibration frame center point coordinates (x, y), and then... rectangle_x Or Cut rectangle_y The image is then returned to its original position to obtain the data-augmented image, such as... Figure 4 As shown in (b) and (c)

[0057] S5: For each target in a single image, repeat S1-S4 to obtain a data-enhanced image for each target;

[0058] S6: Perform S5 on all images in the original dataset to obtain a new dataset after data augmentation.

[0059] S7: Train a deep learning-based object detection model using the new dataset to detect objects in images.

[0060] This invention provides a terminal device, comprising:

[0061] Memory, used to store at least one instruction executed by a processor;

[0062] The processor is used to execute instructions stored in memory to perform the first or second method described above.

[0063] The present invention provides a computer-readable storage medium storing computer instructions that, when executed on a computer, cause the computer to perform the first or second method described above.

[0064] This invention relates to a data augmentation method for target rotation box detection based on deep learning. It fully expands the morphological distribution of targets in SAR images by rotating and randomly flipping targets in SAR images at arbitrary angles, thereby effectively expanding SAR images. Then, it uses neural networks to achieve more accurate rotation box target detection in SAR images.

[0065] This invention can expand the number of sample images in the same scene under the condition of limited samples, thereby generating more training sample images labeled with rotated boxes, effectively increasing the number of samples, and improving the diversity of training samples to a certain extent. This allows the neural network to learn more features about the target, ultimately improving the detection accuracy.

[0066] The contents not described in detail in this specification are common knowledge to those skilled in the art.

Claims

1. A data augmentation method for deep learning-based SAR image target rotation box detection, characterized in that The steps of the method comprise: S1: in the original image, obtaining target calibration box center point coordinates (x, y) and the length h and width w of the calibration box; S2: Take out the image within the circumscribed circle range of the calibration frame, denoted as Cut circle ; S3: Cut circle Randomly rotate the image Cut by any angle with the center of the circle as the rotation center circle_r ; S4: align the center of the circle of Cut circle_r with (x, y), and place Cut circle_r into the circumscribed circle range of the original calibration frame to obtain the enhanced image for the current target. S5: for each target in a single original image, repeating steps S1-S4 to obtain data enhanced images for each target; S6: performing step S5 on all images in the original data set to obtain a new data set after data enhancement operation; S7: training a target detection model based on deep learning using the new data set after data enhancement operation, and detecting targets in a SAR image using the trained model.

2. The data enhancement method for deep learning-based SAR image target rotation box detection according to claim 1, characterized in that: In step S2, the radius of the circumscribed circle corresponding to the calibration box is calculated by h and w, and the center of the circumscribed circle is obtained in combination with the target calibration box center point coordinates (x, y), and then the circumscribed circle range is obtained.

3. The data enhancement method for deep learning-based SAR image target rotation box detection according to claim 1, characterized in that: In step S1, the target calibration box center point coordinates (x, y) and the length h and width w of the calibration box are obtained by using the target annotation information file.

4. The data enhancement method for deep learning-based SAR image target rotation box detection according to claim 3, characterized in that: The target annotation information file is a yaml or xml file.

5. A data augmentation method for deep learning-based SAR image target rotation box detection, characterized in that The steps of the method comprise: S1: in the original image, obtaining target calibration box center point coordinates (x, y) and the length h and width w of the calibration box; S2: take out the image within the calibration frame range, the taken-out image is a rectangular region containing a single target, denoted as Cut rectangle ; S3: flip Cut rectangle along its long axis or short axis, respectively, to obtain Cut rectangle_x or Cut rectangle_y ; wherein the long axis refers to an axis passing through the center of Cut rectangle and parallel to its long side, and the short axis refers to an axis passing through the center of Cut rectangle and parallel to its short side; S4: Align the center of Cut rectangle_x or Cut rectangle_y with the center point coordinate (x, y) of the original calibration frame, and put Cut rectangle_x or Cut rectangle_y back to the original position; S5: for each target in a single original image, repeating steps S1-S4 to obtain data enhanced images for each target; S6: performing step S5 on all images in the original data set to obtain a new data set after data enhancement operation; S7: training a target detection model based on deep learning using the new data set, and detecting targets in a SAR image using the trained model.

6. The data enhancement method for deep learning-based SAR image target rotation box detection according to claim 5, characterized in that: In step S1, the target calibration box center point coordinates (x, y) and the length h and width w of the calibration box are obtained by using the target annotation information file.

7. The data enhancement method for deep learning-based SAR image target rotation box detection according to claim 6, characterized in that: The target annotation information file is a yaml or xml file.

8. A terminal device, comprising: It comprises: A memory for storing instructions executed by at least one processor; A processor for executing instructions stored in the memory to perform the method of any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions, when the computer instructions run on the computer, make the computer execute the method of any one of claims 1-7.

Images