A method for detecting targets in any direction in remote sensing images based on deep learning

Abstract

The present invention discloses a remote sensing image arbitrary direction target detection method based on deep learning, comprising: obtaining a remote sensing image data set, and dividing it into a training set and a test set; performing data enhancement and labeling on the training set; constructing a deep learning network model with a SAN structure; designing an OVAL_IOU loss function based on the constructed network model, and training the network model using the training set; testing the test set using the trained network model to obtain a detection result; post-processing the detection result to obtain the final target position information. The method provided by the present invention overcomes the problems of high false alarm rate and high missed detection rate in the prior art, and improves detection accuracy and detection efficiency.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and in particular relates to a method for detecting targets in any direction of remote sensing images based on deep learning. Background Art

[0002] With the rapid development of imaging technology, the information obtained by drones, satellites and other equipment is becoming more and more comprehensive, the methods of processing data are becoming more and more perfect, the resolution of images is getting higher and higher, and remote sensing images are being used more and more widely in various fields. Remote sensing images are different from natural images. The background of remote sensing images is more complex, the lighting is changeable, and the perspective is changeable; the targets in remote sensing images are small in size, usually with a large aspect ratio, such as ships, trucks, etc., with arbitrary directions, and the foreground and background are less distinguishable. Due to these characteristics of remote sensing images, the current deep learning-based target detection method for natural scenes cannot achieve ideal results when directly applied to remote sensing images, and the obtained target frame contains more redundant background information. Therefore, it is necessary to design a method suitable for remote sensing image target detection to improve the recall rate and precision of target detection in remote sensing images.

[0003] In order to solve the problem of low accuracy in remote sensing image target detection, Patent Document 1 (publication number CN112102241A) proposed a single-stage remote sensing image target detection algorithm, which uses YOLO V3 as the benchmark network, replaces the feature pyramid structure of YOLOV3 with a path aggregation network, and upsamples the feature map through transposed convolution to improve the detection rate of small targets. However, this method uses a horizontal rectangular frame to calibrate the position, and the detection results contain a lot of redundant information, and this method misses detection seriously when the targets are closely arranged.

[0004] Patent document 2 (publication number CN110674674A) proposes a rotation target detection method based on YOLO V3. This method uses the 5-parameter method defined by OpenCV to represent a rectangle of any direction. At the same time, the anchor frame generation method and the positive and negative sample allocation method are redesigned so that the algorithm can obtain more positive samples, making the network easier to train. Finally, the smooth_L1 loss function is used to regress the position, realizing the detection of targets in any direction in remote sensing images. However, this method greatly increases the number of anchor frames, which reduces the detection speed; at the same time, this method is also based on the rectangular frame for position calibration, which contains more redundant information in the detection results, affecting the detection accuracy. In addition, the smooth_L1 loss function will cause a sudden increase in the loss value when regressing the target position due to the periodicity of the angle and the interchangeability of width and height, which increases the difficulty of regression and thus affects the detection efficiency and accuracy.

[0005] In summary, the existing remote sensing image target detection methods have problems such as high false alarm rate, high missed detection rate and low detection efficiency. Summary of the invention

[0006] In order to solve the above problems existing in the prior art, the present invention provides a method for detecting targets in any direction in remote sensing images based on deep learning. The technical problem to be solved by the present invention is achieved through the following technical solutions:

[0007] A method for detecting targets in any direction in remote sensing images based on deep learning, comprising:

[0008] Obtain remote sensing image datasets and divide them into training sets and test sets;

[0009] Performing data augmentation and labeling on the training set;

[0010] Construct a deep learning network model with SAN structure;

[0011] Designing an OVAL_IOU loss function based on the constructed network model, and training the network model using the training set;

[0012] Using the trained network model to test the test set, and obtaining a test result;

[0013] The detection result is post-processed to obtain final target position information.

[0014] In one embodiment of the present invention, performing data enhancement and labeling on the training set includes:

[0015] Cut the data of the training set into appropriate sizes and perform data augmentation;

[0016] Use OpenCV representation to mark the target boxes in the training set;

[0017] Convert the OpenCV representation to the long edge representation, and the target box in the training set is marked as:

[0018] {x c ,y c ,w_l,h_l,θ_l};

[0019]

[0020] Among them, (x c ,y c) is the center coordinate of the target box, w represents the first edge encountered by the x-axis clockwise rotation, h is the other adjacent edge, w_l>=h_l, θ is the angle of rotation, θ∈(0°,90°], θ_l represents the angle of rotation from the x-axis clockwise to w_l, θ_l∈(0°,180°].

[0021] In one embodiment of the present invention, constructing a deep learning network model with a SAN structure includes: using darknet53 as a backbone feature extraction network to build a deep learning network framework, which includes a feature extraction module, a SAN module, a feature fusion module, and a feature integration module; wherein,

[0022] The feature extraction module takes the image to be inspected as input, and after feature extraction, outputs a feature map to the SAN module;

[0023] The SAN module is used to perform feature enhancement on the feature map output by the feature extraction module, and output the enhanced feature map to the feature fusion module;

[0024] The feature fusion module is used to upsample the deep features rich in semantic information and fuse them with the shallow features rich in contour information according to the feature map output by the attention module to obtain the fused features;

[0025] The feature integration module is used to integrate the fusion features to obtain detection results of different scales.

[0026] In one embodiment of the present invention, the SAN module includes a CAM module and an S-SAM module to perform weight division on the feature map from different angles, thereby achieving feature enhancement.

[0027] In one embodiment of the present invention, the loss function is:

[0028] Loss = L conf +L cls +L attention +L reg

[0029] Among them, L conf represents the confidence loss, L cls represents the category loss, L attention represents the attention loss, L reg represents the position regression loss.

[0030] In one embodiment of the present invention, the confidence loss L conf The calculation formula is:

[0031]

[0032] Where N represents the number of all prediction boxes; Indicates whether there is an object in the k-th prediction box, 1 if yes, 0 otherwise; Indicates whether there is no target in the k-th prediction box, if not, it is 1, otherwise, it is 0; p k represents the probability that the kth prediction box contains the target, y k Indicates whether there is a target label value in the k-th prediction box, y k =1 means there is a target, y k =0 means no target; BCE is the cross entropy loss function.

[0033] In one embodiment of the present invention, the class loss L cls The calculation formula is:

[0034]

[0035] Among them, N represents the number of all prediction boxes, is a binary value, Indicates that the nth prediction box contains the target, Indicates that the nth prediction box does not contain the target, BCE is the cross entropy loss function, p n (c) represents the probability that the nth prediction box is of the cth class, t n (c) is the corresponding label value.

[0036] In one embodiment of the present invention, the attention loss L attention The calculation formula is:

[0037]

[0038] Among them, h and w represent the height and width of the foreground score map after the feature layer passes through the S-SAM module, BCE is the cross entropy loss function, and u ij represents the foreground score of the pixel at position i,j, Indicates the label value of the corresponding position, when the position belongs to the foreground otherwise

[0039] In one embodiment of the present invention, the position regression loss L reg The calculation formula is:

[0040]

[0041] Where N represents the number of all prediction boxes; is a binary value, Indicates that the nth prediction box contains the target, It means that the nth predicted box does not contain the target. OVAL_IOU is a function that uses an ellipse to approximate the IOU of two rectangular boxes in any direction.

[0042] In one embodiment of the present invention, post-processing the test result to obtain the location information of the target includes:

[0043] Calculating the confidence value of each grid according to the detection result;

[0044] Calculate the prediction frame according to the detection result and the anchor frame position information;

[0045] The prediction box is screened based on the confidence value to obtain final target location information.

[0046] Beneficial effects of the present invention:

[0047] The present invention effectively reduces noise information and enhances target information by adding a supervised attention structure in a deep learning network. At the same time, an OVAL_IOU loss function is designed to regress the rotated rectangular box, eliminating the sudden increase of the loss value in the existing method, and making the optimization regression task of the rotated rectangular box position consistent with the metric standard of the evaluation method, making the target position regression more direct and effective, overcoming the problems of high false alarm rate and high missed detection rate in the prior art, and improving the detection accuracy and efficiency.

[0048] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a flowchart of a method for detecting targets in any direction in remote sensing images based on deep learning provided by an embodiment of the present invention;

[0050] Figure 2 It is a schematic diagram of the OpenCV representation method of the rotated rectangular frame provided by an embodiment of the present invention;

[0051] Figure 3 is a schematic diagram of a method for representing the long side of a rotated rectangular frame provided by an embodiment of the present invention;

[0052] Figure 4 It is a schematic diagram of a deep learning network structure with a SAN structure provided by an embodiment of the present invention;

[0053] Figure 5 It is a schematic diagram of the SAN network structure provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0055] Embodiment 1

[0056] See also Figure 1 , Figure 1 1 is a flowchart of a method for detecting targets in any direction in remote sensing images based on deep learning provided by an embodiment of the present invention, which includes:

[0057] Step 1: Obtain a remote sensing image dataset and divide it into a training set and a test set.

[0058] In this embodiment, data can be obtained from public remote sensing image datasets such as UCAS_AOD, HRSC2016, DOTA, etc. Then the obtained dataset is divided into a training set and a test set in a ratio of 7:3.

[0059] Step 2: Perform data augmentation on the training set and label it.

[0060] First, the training set data is cut into appropriate sizes and data augmentation is performed on it.

[0061] The specific cutting size can be set as needed, and the data enhancement method can be implemented by referring to the existing related technologies, which will not be introduced in detail in this embodiment.

[0062] Then, the target boxes in the training set are marked using OpenCV representation, such as Figure 2 shown.

[0063] Generally, the annotation method of the rotated rectangular box in public datasets such as DOTA and UCAS_AOD is {x 1 ,y 1 ,x 2 ,y 2 ,x 3 ,y 3 ,x 4 ,y 4}, that is, the 8-parameter method, where (x i ,y i ), i = 1, 2, 3, 4, representing the coordinates of the four vertices of the rotated rectangular frame. In this embodiment, the 8-parameter representation needs to be converted into a 5-parameter representation.

[0064] Specifically, we use the cv.minAreaRect operator in OpenCV to convert the 8-parameter representation into the OpenCV 5-parameter representation, that is, {x c ,y c ,w,h,θ}, where (x c ,y c ) is the center coordinate of the rotated rectangular box, that is, the center coordinate of the target box, w represents the first edge encountered by the x-axis clockwise rotation, h is the other adjacent edge, θ is the angle of rotation, θ∈(0°,90°].

[0065] Finally, convert the OpenCV representation to the long edge representation, such as Figure 3 As shown, the target box in the training set is marked as:

[0066] {x c ,y c ,w_l,h_l,θ_l};

[0067]

[0068] Among them, w_l>=h_l, θ_l represents the angle through which the x-axis rotates clockwise to w_l, θ_l∈(0°,180°].

[0069] Step 3: Build a deep learning network model with SAN structure.

[0070] See also Figure 4 , Figure 4 is a schematic diagram of a deep learning network structure with a SAN structure provided by an embodiment of the present invention. In this embodiment, darknet53 is used as the backbone feature extraction network to build a deep learning network framework, which includes a feature extraction module, a SAN (Supervised Attention Net) module, a feature fusion module, and a feature integration module; wherein,

[0071] The feature extraction module takes the image to be inspected as input, and after feature extraction, outputs the feature map to the SAN module;

[0072] The SAN module is used to enhance the feature map output by the feature extraction module and output the enhanced feature map to the feature fusion module;

[0073] The feature fusion module is used to upsample the deep features rich in semantic information and fuse them with the shallow features rich in contour information according to the feature map output by the attention module to obtain the fused features;

[0074] The feature integration module is used to integrate the fused features to obtain detection results of different scales.

[0075] Specifically, Figure 4 As shown in , this embodiment uses darknet53 as the backbone feature extraction network, and obtains feature maps of 5 scales after the residual block. Figure 3 , 4, 5, and then add a supervised attention structure (SupervisedAttention Net, SAN) to get three new feature layers 6, 7, and 8; then the 8th feature map is convolved and upsampled and fused with the 7th feature map, and the 11th feature map obtained after convolution is upsampled and fused with the 6th feature map; finally, the three feature layers are convolved to get the detection results of three scales. Taking the detection result with the smallest scale as an example, its shape is 13*13*[(5+1+n)*3], where 13*13 means that the original image is divided into 13*13 grids; [(5+1+n)*3] represents its number of channels, where *3 means that each grid has three prediction boxes, so there are 13*13*3 prediction boxes in the original image, 5 represents the offset of the prediction box relative to the center position, width, height and angle of the anchor box, 1 represents the probability of having an object in each prediction box, n represents the number of categories, and the corresponding probability of the object in the prediction box being each category.

[0076] For further information, see Figure 5 , Figure 5 It is a schematic diagram of the SAN network structure provided by an embodiment of the present invention, which includes two modules, a channel attention module (CAM) and a supervised-spatial attention module (S-SAM), which weight feature maps from different angles to achieve feature enhancement.

[0077] Specifically, for input features of size 256*52*52 Figure 3 , which are processed by the CAM module and the SAM module respectively to obtain enhanced features.

[0078] For the CAM module, the size of the features after maximum pooling and average pooling is 256*1*1, and then they are passed through the fully connected layer to obtain new 256*1*1 features. Then, these two features are added element by element and activated by the sigmoid function to output 256*1*1 features.

[0079] For the S-SAM module, after maximum pooling and average pooling, the features of 2*52*52 are concatenated, and then after convolution and sigmoid function activation, the features of 1*52*52 are obtained.

[0080] Finally, the features obtained by the CAM module are compared with the features obtained by the S-SAM module and the input Figure 3 By multiplying them, we can get feature map 6, which has the same size as the input feature map, 256*52*52.

[0081] The following takes the network input as an example of a 3*416*416 image to describe its processing process in detail.

[0082] First, the 3*416*416 image passes through the conv2D Block to obtain the feature map 0 of size 64*208*208;

[0083] Then, after Res Block, we get the feature maps of 5 scales, where Res Block*n represents the stacking of n Res Blocks.

[0084] After the feature map 0 passes through Res Block*1, the feature size is 64*208*208. Figure 1 ;

[0085] feature Figure 1 After Res Block*2, the feature size is 128*104*104 Figure 2 ;

[0086] feature Figure 2 After Res Block*8, the feature size is 256*52*52 Figure 3 ;

[0087] feature Figure 3 After Res Block*8, the feature size is 512*26*26 Figure 4 ;

[0088] feature Figure 4 After Res Block*4, the feature size is 1024*13*13 Figure 5 .

[0089] In Features Figure 3 , 4 , 5, and then add a supervised attention net (SAN) to obtain feature maps 6, 7, and 8 respectively; SAN does not change the size of the feature map.

[0090] After convolution of feature map 8, feature map 9 with a size of 512*13*13 is obtained;

[0091] After convolution, feature map 9 obtains the detection result of large-scale targets, with a shape of 13*13*[(5+1+n)*3];

[0092] Feature map 9 is upsampled to obtain feature map 10 with a size of 512*26*26;

[0093] Feature map 10 is fused with feature map 7 to obtain feature map 11 with a size of 768*26*26;

[0094] After convolution of feature map 11, feature map 12 with a size of 256*26*26 is obtained;

[0095] After convolution, feature map 12 obtains the detection result of the medium-scale target, with a shape of 26*26*[(5+1+n)*3];

[0096] Feature map 12 is upsampled to obtain feature map 13 with a size of 128*52*52;

[0097] Feature map 13 is fused with feature map 6 to obtain feature map 14 with a size of 384*52*52;

[0098] After convolution of feature map 14, feature map 15 with a size of 128*52*52 is obtained;

[0099] After convolution, feature map 15 obtains the detection result of small-scale targets, with a shape of 52*52*[(5+1+n)*3].

[0100] In this embodiment, a SAN structure is added after the main feature layer. The structure includes two modules, CAM and S-SAM, which can effectively reduce noise information and enhance target information, thereby improving detection accuracy to a certain extent.

[0101] Step 4: Design the OVAL_IOU loss function based on the constructed network model and train the network model using the training set.

[0102] First, in order to solve the problem of sudden increase in loss value caused by the periodicity of angles and the interchangeability of width and height when the smooth_L1 loss function regresses the target position, this embodiment defines the OVAL_IOU loss function to regress the rotated rectangular box.

[0103] Specifically, the OVAL_IOU loss function is expressed as:

[0104] Loss = L conf +L cls +L attention +L reg

[0105] Among them, L conf Represents the confidence loss, and the calculation formula is:

[0106]

[0107] Where N represents the number of all prediction boxes; Indicates whether there is an object in the k-th prediction box, 1 if yes, 0 otherwise; Indicates whether there is no target in the k-th prediction box, if not, it is 1, otherwise, it is 0; p krepresents the probability that the kth prediction box contains the target, y k Indicates whether there is a target label value in the k-th prediction box, y k =1 means there is a target, y k =0 means no target; BCE is the cross entropy loss function.

[0108] In this embodiment, the calculation formula of BCE is:

[0109] BCE(p k ,y k )=-[y k ·log(p k )+(1-y k )·log(1-p k )]

[0110] L cls Represents the category loss, and the calculation formula is:

[0111]

[0112] Among them, N represents the number of all prediction boxes, is a binary value, Indicates that the nth prediction box contains the target, Indicates that the nth prediction box does not contain the target, p n (c) represents the probability that the nth prediction box is of the cth class, t n (c) is the corresponding label value.

[0113] L attention represents the attention loss, and the calculation formula is:

[0114]

[0115] Among them, h and w represent the height and width of the foreground score map after the feature layer passes through the S-SAM module, BCE is the cross entropy loss function, and u ij represents the foreground score of the pixel at position i,j, Indicates the label value of the corresponding position, when the position belongs to the foreground otherwise

[0116] L reg Represents the position regression loss, and the calculation formula is:

[0117]

[0118] Among them, OVAL_IOU is a function that uses an ellipse to approximate the IOU of two rectangular boxes in any direction. The specific calculation process is as follows:

[0119] Any ellipse in a plane can be represented as:

[0120]

[0121] Where a is the major axis, b is the minor axis, α is the angle between the major axis and the x axis α∈(0°,180°], {x c ,y c ,w_l,h_l,θ_l} represented by the rotation rectangle is approximately represented by an ellipse:

[0122]

[0123] definition:

[0124]

[0125]

[0126]

[0127] Among them, (i, j) represents the coordinates of the pixel point, and oval represents {x c ,y c ,w_l,h_l,θ_l}, and use P(i,j,oval) to determine whether the pixel point p(i,j) is inside the ellipse. If oval P(i,j,oval)=1, it means that the pixel point p(i,j) is inside the oval, otherwise it is not. Then the intersection and union of the two ellipses is:

[0128]

[0129]

[0130] Because P(i,j,oval) is not differentiable, a continuous differentiable function F(i,j,oval) is used to approximate P(i,j,oval). The expression of F(i,j,oval) is as follows:

[0131] F(i,j,oval)=K(k_1+k_2,0.25)

[0132] in k is an adjustable parameter that controls the sensitivity of the target pixel.

[0133] The intersection and union formula is updated to:

[0134]

[0135]

[0136] The OVAL_IOU calculation formula is:

[0137]

[0138] This paper proposes a new loss function, OVAL_IOU Loss. OVAL_IOU uses an ellipse to approximate a rectangular box in any direction, and then calculates the IOU of two ellipses as the IOU of any two rotated rectangular boxes. OVAL makes the optimization regression task of the rotated rectangular box position consistent with the metric standard of the evaluation method, that is, using the IOU of two target boxes, making the target position regression more direct and effective.

[0139] Then, based on the constructed OVAL_IOU loss function, the network model constructed in step 3 is trained using the training set data to obtain a trained network model.

[0140] Step 5: Use the trained network model to test the test set and obtain the test results.

[0141] Specifically, assuming that the width and height of the input image are 416×416, a total of 13×13×3+26×26×3+52×52×3=10647 detection results can be obtained.

[0142] Step 6: Post-process the detection results to obtain the final target location information.

[0143] 61) Calculate the confidence value of each grid based on the detection results.

[0144] Specifically, after dividing the image into n×n grids (n∈{13,26,52}), the prediction result of the (i,j)th grid is (t ijkx ,t ijky ,t ijkw ,t ijkh ,t ijkθ ,p ijkobj ,p ijkc1 ,p ijkc2 ,...,p ijkcn ).

[0145] Among them, (i=1,2,…,n;j=1,2,…,n), k=1,2,3 means that each grid has three prediction boxes, t ijkx ,t ijky ,t ijkw ,t ijkh ,t ijkθ They represent the offset of the k-th prediction box of the (i, j)-th grid relative to the center position, width, height and angle of the anchor box, respectively. ijkobj The probability that the k-th prediction box of the (i, j)-th grid has an object, n represents the number of categories, p ijkcmDenote the probability that the object in the \(k\)-th prediction box of the \((i, j)\)-th grid belongs to the \(m\) (\(m = 1, 2, \cdots, n\))-th class.

[0146] Then the confidence calculation formula is:

[0147] conf ijk = p ijkobj * max(p ijkc1 , p ijkc2 , \cdots, p ijkcn )

[0148] Set a confidence threshold conf_thr, remove the prediction boxes with conf ijk < conf_thr, and convert the prediction results to \((t ijkx , t ijky , t ijkw , t ijkh , t ijkθ , conf ijk , cls_id), where cls_id (cls_id = 0, 1, \cdots, n - 1) represents the class number corresponding to the maximum value in \((p ijkc1 , p ijkc2 , \cdots, p ijkcn ).

[0149] (62) Calculate the prediction box according to the detection result and the anchor box position information.

[0150] Specifically, according to the predicted t ijkx , t ijky , t ijkw , t ijkh , t ijkθ and the center, width, height and angle of the anchor box \((x aijk , y aijk , w aijk , h aijk , θ aijk ), obtain the prediction box \((x pijk , y pijk , w pijk , h pijk , θ pijk ), and the calculation formula is as follows:

[0151] x pijk = x aijk + t ijkx

[0152] y pijk = y aijk + t ijky

[0153] w pijk = w ajk*exp(t ijkw )

[0154] h pijk =h ajk *exp(t ijkh )

[0155] θ pijk =θ aijk +t ijkθ

[0156] 63) The prediction box is filtered based on the confidence value to obtain the final target location information.

[0157] Since there are many repeated prediction boxes in step 62), they need to be removed. This embodiment uses the NMS (non-maximum suppression) algorithm to remove redundant prediction boxes and retain the prediction box with the highest confidence. The specific implementation steps are as follows:

[0158] a): Set the IOU threshold iou_thr for deleting redundant prediction boxes;

[0159] b): Sort all prediction boxes by cls_id;

[0160] c): Take out the prediction box of one category each time, and arrange them in descending order according to the confidence of the prediction box to get the subsequent box list;

[0161] d): Select the box B with the highest confidence and add it to the output list, and delete it from the candidate box list;

[0162] e): When the subsequent box list is not empty, calculate the IOU value of box B with all boxes in the candidate box list, and delete the subsequent boxes whose IOU is greater than iou_thr;

[0163] f): Repeat steps d) and e) until the candidate box list is empty;

[0164] g): Repeat steps c), d), e), and f) until all categories are processed and the output list is returned.

[0165] At this point, the final target location information is obtained.

[0166] The present invention effectively reduces noise information and enhances target information by adding a supervised attention structure in a deep learning network. At the same time, an OVAL_IOU loss function is designed to regress the rotated rectangular box, eliminating the sudden increase of the loss value in the existing method, and making the optimization regression task of the rotated rectangular box position consistent with the metric standard of the evaluation method, making the target position regression more direct and effective, overcoming the problems of high false alarm rate and high missed detection rate in the prior art, and improving the detection accuracy and efficiency.

[0167] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A method for detecting targets in any direction in remote sensing images based on deep learning, characterized in that: include: Obtain remote sensing image datasets and divide them into training sets and test sets; Performing data augmentation and labeling on the training set; Construct a deep learning network model with SAN structure; An OVAL_IOU loss function is designed based on the constructed network model, and the network model is trained using the training set; wherein the OVAL_IOU loss function is expressed as: Loss=L conf +L cls +L attention +L reg Where Loss represents OVAL_IOU loss, L conf represents the confidence loss, L cls represents the category loss, L attention represents the attention loss, L reg Represents the position regression loss; the position regression loss L reg The calculation formula is: Where N represents the number of all prediction boxes; is a binary value, Indicates that the nth prediction box contains the target, Indicates that the nth prediction box does not contain the target, oval_p n and oval_t n They represent ellipse prediction boxes in two directions respectively. OVAL_IOU represents a function that uses ellipses to approximate the IOU of two rectangular boxes in any direction. The specific calculation process includes: approximating any two rotated rectangular boxes with two ellipses, calculating the intersection and union of the two ellipses according to whether the target point is within the two ellipses, and then calculating the IOU of the two ellipses as the IOU of any two rotated rectangular boxes, so that the optimization regression task of the rotated rectangular box position is consistent with the metric standard of the evaluation method; Using the trained network model to test the test set, and obtaining a test result; The detection result is post-processed to obtain final target position information.

2. The method for detecting targets in any direction in remote sensing images based on deep learning according to claim 1, characterized in that: Performing data augmentation and labeling on the training set includes: Cut the data of the training set into a set size and perform data augmentation on it; Use OpenCV representation to mark the target boxes in the training set; Convert the OpenCV representation to the long edge representation, and the target box in the training set is marked as: {x c ,y c ,w_l,h_l,θ_l}; Among them, (x c ,y c ) is the center coordinate of the target box, w represents the first edge encountered by the x-axis clockwise rotation, h is the other adjacent edge, w_l>=h_l, θ is the angle of rotation, θ∈(0o,90o], θ_l represents the angle of rotation from the x-axis clockwise to w_l, θ_l∈(0o,180o].

3. The method for detecting targets in any direction in remote sensing images based on deep learning according to claim 1, characterized in that: Building a deep learning network model with a SAN structure includes: Darknet53 is used as the backbone feature extraction network to build a deep learning network framework, which includes feature extraction module, SAN module, feature fusion module and feature integration module; among them, The feature extraction module takes the image to be inspected as input, and after feature extraction, outputs a feature map to the SAN module; The SAN module is used to perform feature enhancement on the feature map output by the feature extraction module, and output the enhanced feature map to the feature fusion module; The feature fusion module is used to upsample the deep features rich in semantic information and fuse them with the shallow features rich in contour information according to the feature map output by the attention module to obtain the fused features; The feature integration module is used to integrate the fusion features to obtain detection results of different scales.

4. The method for detecting targets in any direction in remote sensing images based on deep learning according to claim 3, characterized in that: The SAN module includes a CAM module and an S-SAM module to weight the feature map from different angles, thereby achieving feature enhancement.

5. The method for detecting targets in any direction in remote sensing images based on deep learning according to claim 1, characterized in that: The confidence loss L conf The calculation formula is: Where N represents the number of all prediction boxes; Indicates whether there is an object in the k-th prediction box, 1 if yes, 0 otherwise; Indicates whether there is no target in the k-th prediction box, if not, it is 1, otherwise, it is 0; p k represents the probability that the kth prediction box contains the target, y k Indicates whether there is a target label value in the k-th prediction box, y k =1 means there is a target, y k =0 means no target; BCE is the cross entropy loss function.

6. The method for detecting targets in any direction in remote sensing images based on deep learning according to claim 1, characterized in that: The class loss L cls The calculation formula is: Among them, N represents the number of all prediction boxes, is a binary value, Indicates that the nth prediction box contains the target, Indicates that the nth prediction box does not contain the target, BCE is the cross entropy loss function, p n (c) represents the probability that the nth prediction box is of the cth class, t n (c) is the corresponding label value.

7. The method for detecting targets in any direction in remote sensing images based on deep learning according to claim 1, characterized in that: The attention loss L attention The calculation formula is: Among them, h and w represent the height and width of the foreground score map after the feature layer passes through the S-SAM module, BCE is the cross entropy loss function, and u ij represents the foreground score of the pixel at position i,j, Indicates the label value of the corresponding position, when the position belongs to the foreground otherwise 8. The method for detecting targets in any direction in remote sensing images based on deep learning according to claim 1, characterized in that: Post-processing the detection results to obtain final target location information includes: Calculating the confidence value of each grid according to the detection result; Calculate the prediction frame according to the detection result and the anchor frame position information; The prediction box is screened based on the confidence value to obtain final target location information.

Images