A rotation target detection method based on improved Orientedrcnn

Abstract

The present invention relates to the technical field of rotated image target detection, and in particular to a rotated target detection method based on an improved Oriented RCNN. The method comprises the following steps: inputting an image; image preprocessing: adjusting each image to a fixed size, normalizing the fixed-size images, and dividing the images into a training set, a validation set, and a test set; inputting a network model for training: inputting the training set into the improved Oriented RCNN model for training; inputting a test set, and outputting a detection result. The present invention uses a six-parameter method different from that in the prior art to define a rotated anchor frame, and uses different polarization functions to respectively extract different features required for classification tasks and positioning tasks. In addition, an SPP module is introduced to realize the fusion of local features and global features, thereby overcoming detection interference caused by the inconsistency of features required for classification and regression, effectively extracting different features required for different tasks, and more accurately classifying and positioning remote sensing image targets.

Description

Technical Field

[0001] The present invention relates to the technical field of rotation image target detection, and in particular to a rotation target detection method based on an improved Orientedrcnn. Background Art

[0002] Object detection in remote sensing images is a primary task in remote sensing image processing. Its primary task is to automatically identify regions of interest (ROIs) within a given remote sensing image dataset and assign specific object categories. However, objects in remote sensing images often have varying orientations and small scales, and conventional object detection methods, which rely on horizontal anchor boxes, cannot accurately fit the objects.

[0003] To accurately predict target orientation, many researchers have proposed rotated object detection. This method introduces orientation parameters into the RPN module and generates oriented anchor boxes for regression and classification. For example, RotateRPN sets 54 anchor boxes of varying angles, scales, and aspect ratios at each anchor point, improving accuracy when oriented objects are sparsely distributed. Xie X et al. proposed OrientedRCNN to address computational cost issues and further improve accuracy.

[0004] In order to obtain accurate information about rotated targets, researchers have introduced rotating anchor frames into existing target detection models to achieve more accurate positioning. For example, the patent application with publication number CN112800955A, entitled “Remote sensing image rotation target detection method and system based on weighted bidirectional feature pyramid”, discloses a method for detecting rotated targets in remote sensing images based on weighted bidirectional feature pyramid, which introduces BiFPN to enhance the model’s cross-scale feature fusion capabilities. The patent application with publication number CN110378242A, entitled “A remote sensing target detection method based on a dual attention mechanism”, discloses a remote sensing target detection method based on a dual attention mechanism, which uses the dual attention mechanism to redefine the feature map.

[0005] However, due to the inconsistency of the required features between classification and regression, existing rotated image object detection methods have difficulty in accurately extracting different features required for different tasks. Summary of the Invention

[0006] The object of the present invention is to provide a rotation target detection method based on improved Orientedrcnn to solve the problems raised in the above background technology.

[0007] The technical solution of the present invention is: a rotating target detection method based on improved Orientedrcnn, comprising the following steps:

[0008] Step 1. Input image: Select a remote sensing image dataset whose annotation file contains orientation information as the input image, and randomly flip and fill the input image;

[0009] Step 2: Image preprocessing: resize each image to a fixed size of 1024×1024; normalize the fixed-size images and divide them into training, validation, and test sets in a 5:2:3 ratio;

[0010] Step 3: Input the network model for training: Input the training set in step 2 into the improved Orientedrcnn model for training;

[0011] Step 4: Input the test set and output the detection results: Use the trained improved Orientedrcnn model to detect the remote sensing image and obtain the result image of the target framed by the rotating frame;

[0012] Among them, the operations of training the improved Orientedrcnn model in step 3 include: inputting the training set into the backbone network ResNet50 for feature extraction to obtain features C2-C5 of different sizes; inputting the extracted features into the SPP-FPN module for feature fusion to obtain featuremap, wherein the SPP-FPN module outputs C5 of the deepest layer of the backbone network through the SPP module to obtain M5.

[0013] Preferably, the specific operations of step 3 include:

[0014] Step 3.1: Input the training set into the backbone network ResNet50 for feature extraction to obtain features C2-C5 of different sizes;

[0015] Step 3.2: Input the extracted features into the SPP-FPN module for feature fusion to obtain featuremap;

[0016] Step 3.3: Input the feature map into the rotation proposal region generation module oriented RPN, and after encoding and decoding, output the proposal region proposals;

[0017] Step 3.4: Input the feature map obtained in step 3.2 and the proposals obtained in step 3.3 into the improved detection head module PAM-head for the final classification and positioning operations, and output the remote sensing target recognition and positioning results.

[0018] Preferably, in step 3.2, the specific workflow of the SPP-FPN module includes: passing the deepest output C5 of the backbone network through the SPP module to obtain M5, summing the elements of the result obtained by upsampling M5 and horizontally connecting it with C4 to obtain M4, summing the elements of the result obtained by upsampling M4 and horizontally connecting it with C3 to obtain M3, and so on to obtain M2-M5, and respectively subjecting M2-M5 to 3×3 convolution to obtain improved FPN outputs P2-P5.

[0019] Preferably, the SPP module realizes the fusion of local features and global features, processes the feature maps using pooling of different sizes, and finally splices them to obtain the output result.

[0020] Preferably, in the step 3.3, the specific operation of the rotation proposal region generation module orientedRPN includes: the output feature map of step 3.2 is changed to 6A after the number of convolution channels, A represents the number of anchor frames generated at each anchor point, 6 represents that 6 parameters are required to define a rotated anchor frame, and the 6 parameters are (x, y, w, h, Δα, Δβ), where x and y represent the coordinates of the center point of the generated horizontal anchor frame, w and h represent the width and height of the generated horizontal anchor frame, and Δα and Δβ represent the offset between two adjacent vertices of the rotated anchor frame and the midpoints of two adjacent sides of the horizontal anchor frame.

[0021] Preferably, the specific operations of the improved detection head PAM-head module in step 3.4 include: processing the input feature map through the polarized attention module PAM, generating different feature pyramids for classification tasks and positioning tasks, avoiding feature interference between different tasks, and effectively extracting different key features required for different tasks, sending the obtained different features to the fully connected layer for classification and regression, and outputting the final classification and positioning results.

[0022] Preferably, the polarized attention module PAM has a dual-branch structure, and the input feature map uses different feature representation functions after passing through the attention module (channel attention module and spatial attention module in parallel). The classification branch uses an excitation function to obtain highly responsive global features, and the positioning branch uses an inhibition function to focus only on boundary features and suppress irrelevant highly activated areas.

[0023] Preferably, the experimental configuration of the training model of the rotating target detection method includes the MMdetectionV2 framework, the experimental environment is Python3.8, Pytorch1.7.0, torchvision0.7.0, the batch size is 2, the initial value of the learning rate is set to 0.001, the maximum number of training epochs is 12, and the learning rate drops to 1×10-4 and 1×10-5 after iterating to the 9th and 11th epochs, respectively.

[0024] Preferably, the experimental hardware equipment for the training model of the rotating target detection method is Core TM i9-10900XCPU, NVIDIARTX3080Ti graphics card.

[0025] Preferably, the size of the input image is adjusted to 1024×1024 pixels, and the accuracy AP of each category of targets in the dataset and the average accuracy mAP of all categories of targets are used as the measurement indicators of the experiment.

[0026] The present invention provides a rotation target detection method based on improved Orientedrcnn by improvement, which has the following improvements and advantages compared with the prior art:

[0027] The present invention uses a six-parameter method different from the existing technology to define the rotating anchor frame, and uses different polarization functions to extract different features required for classification tasks and positioning tasks respectively. In addition, the SPP module is introduced to realize the fusion of local features and global features, which can overcome the detection interference caused by the inconsistency of the features required for classification and regression, effectively extract different features required for different tasks, and more accurately classify and locate remote sensing image targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The present invention will be further explained below in conjunction with the accompanying drawings and embodiments:

[0029] Figure 1 It is a flow chart of the entire network framework of the present invention.

[0030] Figure 2 This is the structural diagram of the SPP-FPN in the present invention.

[0031] Figure 3 This is a structural diagram of the SPP module involved in the present invention.

[0032] Figure 4 Schematic diagram of the six parameters representing the anchor box in the oriented RPN of the present invention.

[0033] Figure 5 This is a structural diagram of the PAM-head in the present invention.

[0034] Figure 6 This is a schematic diagram of the remote sensing image detection results obtained by the present invention. DETAILED DESCRIPTION

[0035] The present invention is described in detail below, clearly and completely describing the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0036] The present invention provides a rotating target detection method based on improved Orientedrcnn by improvement. The technical solution of the present invention is:

[0037] like Figure 1 As shown, a rotation target detection method based on improved Orientedrcnn includes the following steps:

[0038] Step 1. Input image: Select a remote sensing image dataset whose annotation file contains orientation information as the input image, and randomly flip and fill the input image;

[0039] Step 2: Image preprocessing: resize each image to a fixed size of 1024×1024; normalize the fixed-size images and divide them into training, validation, and test sets in a 5:2:3 ratio;

[0040] Step 3: Input the network model for training: Input the training set in step 2 into the improved Orientedrcnn model for training;

[0041] Step 4: Input the test set and output the detection results: Use the trained improved Orientedrcnn model to detect the remote sensing image and obtain the result image of the target framed by the rotating frame. The result image is as follows: Figure 6 As shown;

[0042] Among them, the operations of training the improved Orientedrcnn model in step 3 include: inputting the training set into the backbone network ResNet50 for feature extraction to obtain features C2-C5 of different sizes; inputting the extracted features into the SPP-FPN module for feature fusion to obtain featuremap, wherein the SPP-FPN module outputs C5 of the deepest layer of the backbone network through the SPP module to obtain M5.

[0043] The specific operations of step 3 include:

[0044] Step 3.1: Input the training set into the backbone network ResNet50 for feature extraction to obtain features C2-C5 of different sizes;

[0045] Step 3.2: Input the extracted features into the SPP-FPN module for feature fusion to obtain featuremap;

[0046] Step 3.3: Input the feature map into the rotation proposal region generation module oriented RPN, and after encoding and decoding, output the proposal region proposals;

[0047] Step 3.4: Input the feature map obtained in step 3.2 and the proposals obtained in step 3.3 into the improved detection head module PAM-head for the final classification and positioning operations, and output the remote sensing target recognition and positioning results.

[0048] Among them, in the step 3.2, the specific workflow of the SPP-FPN module includes: passing the deepest output C5 of the backbone network through the SPP module to obtain M5, summing the elements of the result obtained by upsampling M5 and horizontally connecting it with C4 to obtain M4, summing the elements of the result obtained by upsampling M4 and horizontally connecting it with C3 to obtain M3, and so on to obtain M2-M5, and respectively subjecting M2-M5 to 3×3 convolution to obtain improved FPN outputs P2-P5, as shown Figure 2 shown.

[0049] Furthermore, the SPP module realizes the fusion between local features and global features, processes the feature maps using pooling of different sizes, and finally splices them to obtain the output results, such as Figure 3 shown.

[0050] Among them, in the step 3.3, the specific operation of the rotation proposal region generation module oriented RPN includes: the output feature map of step 3.2 is converted into 6A after the number of convolution channels, A represents the number of anchor boxes generated at each anchor point, 6 represents that 6 parameters are required to define a rotation anchor box, and the 6 parameters are (x, y, w, h, Δα, Δβ), where x, y represent the coordinates of the center point of the generated horizontal anchor box, w, h represent the width and height of the generated horizontal anchor box, Δα, Δβ represent the offset between two adjacent vertices of the rotation anchor box and the midpoints of the two adjacent sides of the horizontal anchor box, such as Figure 4 As shown. The formula for regressing the anchor box using the above 6 parameters is:

[0051]

[0052] Where v1, v2, v3, and v4 represent the four vertices of the rotation anchor box.

[0053] Furthermore, since orientedRPN generates a large number of anchor boxes, it is necessary to select N anchor boxes with higher scores as subsequent inputs. In this paper, the DIoU score is used as the positive sample allocation strategy. The DIoU expression is shown in Formula 2:

[0054]

[0055] where d 2 Represents the square of the distance between the center point of the predicted box and the real box, c 2 Represents the square of the diagonal length of the minimum enclosing rectangle of the predicted box and the true box.

[0056] The specific operation of the improved detection head PAM-head module in step 3.4 includes: processing the input feature map through the polarized attention module PAM, generating different feature pyramids for classification tasks and positioning tasks, avoiding feature interference between different tasks, and effectively extracting different key features required for different tasks, sending the obtained different features to the fully connected layer for classification and regression, and outputting the final classification and positioning results; wherein, the PAM-head module structure is as follows Figure 5 The total loss function of the model is shown in Equation 3:

[0057]

[0058] Among them L cls The cross entropy loss is used, L reg Using Smooth L1 Loss. cls As shown in formula 4, L reg As shown in Formula 5:

[0059]

[0060]

[0061] where p i is the output of the RPN classification branch, indicating the probability that the proposal is a foreground, is the label of the i-th true value, t i Indicates the offset value of the positioning branch regression, Indicates the offset value of the real box, Smooth L1 The function definition is shown in Equation 6:

[0062]

[0063] Furthermore, the polarized attention module (PAM) has a dual-branch structure. After the input feature map passes through the attention module (channel attention module and spatial attention module in parallel), different feature representation functions are used. The classification branch uses an excitation function to obtain highly responsive global features, and the localization branch uses an inhibition function to focus only on boundary features and suppress irrelevant high-activation areas. The expression of the excitation function is as follows:

[0064]

[0065] Where η is the excitation coefficient. The suppression function expression is as follows:

[0066]

[0067] In the present invention, the experimental configuration of the training model of the rotating target detection method includes the following: based on the MMdetectionV2 framework, the experimental environment is Python3.8, Pytorch1.7.0, torchvision0.7.0, the batch size is 2, the initial value of the learning rate is set to 0.001, the maximum number of training epochs is 12, and the learning rate drops to 1×10-4 and 1×10-5 after iterating to the 9th and 11th epochs, respectively.

[0068] Among them, the experimental hardware equipment of the training model of the rotating target detection method is Core TM i9-10900XCPU, NVIDIARTX3080Ti graphics card.

[0069] Considering the large size of remote sensing images, the size of the input image is adjusted to 1024×1024 pixels, and the accuracy AP of each type of target in the dataset and the average accuracy mAP of all types of targets are used as the measurement indicators of the experiment.

[0070] The method of the present invention uses a six-parameter method different from the existing technology to define the rotation anchor frame, and uses different polarization functions to extract different features required for classification tasks and positioning tasks respectively. In addition, the SPP module is introduced to realize the fusion of local features and global features, which can overcome the detection interference caused by the inconsistency of the required features between classification and regression, enhance the feature representation of small remote sensing targets, and has good network performance and strong model generalization ability.

[0071] The above description is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A rotating target detection method based on an improved Oriented RCNN, characterized by: The following steps are involved: Step 1. Input image: Select a remote sensing image dataset whose annotation file contains orientation information as the input image, and randomly flip and fill the input image; Step 2: Image preprocessing: resize each image to a fixed size of 1024×1024; Normalize the fixed-size images and divide them into training set, validation set, and test set according to the ratio of 5:2:3; Step 3: Input the network model for training: Input the training set in step 2 into the improved Oriented rcnn model for training. The specific operations include: Step 3.1: Input the training set into the backbone network ResNet50 for feature extraction to obtain features C2-C5 of different sizes; Step 3.2: Input the extracted features into the SPP-FPN module for feature fusion to obtain the feature map; Step 3.3, input the feature map into the rotation proposal region generation module oriented RPN, after encoding and decoding, output the proposal region proposals, the specific operation of the rotation proposal region generation module oriented RPN includes: step 3.2 output feature map after convolution channel number becomes 6A, A represents the number of anchor boxes generated at each anchor point, 6 means that 6 parameters are required to define a rotation anchor box, 6 parameters are (x, y, w, h, Δα, Δβ), where x, y represent the coordinates of the center point of the generated horizontal anchor box, w, h represent the width and height of the generated horizontal anchor box, Δα, Δβ represent the offset between two adjacent vertices of the rotation anchor box and the midpoints of two adjacent sides of the horizontal anchor box; Step 3.4: Input the feature map obtained in step 3.2 and the proposals obtained in step 3.3 into the improved detection head module PAM-head to perform the final classification and positioning operations, and output the remote sensing target recognition and positioning results; Step 4: Input the test set and output the detection results: Use the trained improved Oriented rcnn model to detect the remote sensing image and obtain the result image of the target framed by the rotating frame; Among them, the operations of training the improved Oriented rcnn model in step 3 include: inputting the training set into the backbone network ResNet50 for feature extraction to obtain features C2-C5 of different sizes; inputting the extracted features into the SPP-FPN module for feature fusion to obtain a feature map, wherein the SPP-FPN module outputs C5 of the deepest layer of the backbone network through the SPP module to obtain M5.

2. The rotating target detection method based on the improved Oriented RCNN according to claim 1, characterized in that: In step 3.2, the specific workflow of the SPP-FPN module includes: passing the deepest output C5 of the backbone network through the SPP module to obtain M5, summing the elements of the result obtained by upsampling M5 and horizontally connecting it with C4 to obtain M4, summing the elements of the result obtained by upsampling M4 and horizontally connecting it with C3 to obtain M3, and so on to obtain M2-M5, and respectively subjecting M2-M5 to 3×3 convolution to obtain improved FPN outputs P2-P5.

3. The rotating target detection method based on the improved Oriented RCNN according to claim 2, characterized in that: The SPP module realizes the fusion of local features and global features, processes the feature maps using pooling of different sizes, and finally splices them to obtain the output results.

4. The rotating target detection method based on the improved Oriented RCNN according to claim 2, characterized in that: The specific operations of the improved detection head PAM-head module in step 3.4 include: processing the input feature map through the polarized attention module PAM, generating different feature pyramids for classification tasks and positioning tasks, avoiding feature interference between different tasks, and effectively extracting different key features required for different tasks, sending the obtained different features to the fully connected layer for classification and regression, and outputting the final classification and positioning results.

5. The rotating target detection method based on improved Oriented RCNN according to claim 4, characterized in that: The polarized attention module (PAM) has a dual-branch structure. After the input feature map passes through the attention module (channel attention module and spatial attention module in parallel), different feature representation functions are used. The classification branch uses an excitation function to obtain highly responsive global features, and the localization branch uses an inhibition function to focus only on boundary features and suppress irrelevant high-activation areas.

6. A rotating target detection method based on an improved Oriented RCNN according to any one of claims 1 to 5, characterized in that: The experimental configuration of the training model of the rotation object detection method includes the following: based on the MMdetection V2 framework, the experimental environment is Python 3.8, Pytorch 1.7.0, torchvision 0.7.0, the batch size is 2, the initial value of the learning rate is set to 0.001, the maximum number of training epochs is 12, and the learning rate drops to 1×10-4 and 1×10-5 after the 9th and 11th epochs, respectively.

7. The rotating target detection method based on improved Oriented RCNN according to claim 6, characterized in that: The experimental hardware equipment for the training model of the rotating object detection method is Intel® Core™ i9-10900X CPU and NVIDIA RTX3080Ti graphics card.

8. The rotating target detection method based on improved Oriented RCNN according to claim 7, characterized in that: The size of the input image is adjusted to 1024×1024 pixels, and the accuracy AP of each type of target in the dataset and the average accuracy mAP of all types of targets are used as the measurement indicators of the experiment.

Images