Rotation target fine detection method and system based on Bhattacharyya distance metric loss function

Abstract

The present invention discloses a method and system for fine detection of rotating targets based on Bhattacharyya distance metric loss function, including: obtaining a sample data set for detecting rotating targets in visible light remote sensing images, unifying the label format of the sample data set, and dividing it into a training sample set and a test sample set; constructing a fine detection network for rotating targets; using the training sample set as input, constructing a Bhattacharyya distance metric loss function as an objective function to train the fine detection network for rotating targets, and obtaining a trained fine detection network for rotating targets; inputting the test sample set into the trained fine detection network for rotating targets, and outputting the precise position and category of the target. The present invention solves the technical problem that the feature map extracted by the deep learning network is strongly interfered by densely distributed targets, and that the refined position features of densely distributed rotating targets are difficult to effectively extract, thereby realizing the refined extraction of target features, suppressing the interference of complex background on the target, and accurately locating the target rotation frame.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence theory and algorithm technology, and in particular relates to a method and system for fine detection of rotating targets based on Bhattacharyya distance metric loss function. Background Art

[0002] In recent years, deep learning and convolutional neural networks (CNNs) have made significant progress in the interpretation of satellite remote sensing images. Since convolutional neural networks have a strong ability to represent target features, a large number of satellite remote sensing image target detection methods using deep learning have been proposed, such as Yan D, Li G, Li X, et al. An improved Faster R-CNN method to detect tailings ponds from high-resolution remote sensing images [J]. Remote sensing, 2021, 13 (11): 2052. and have achieved good detection results. These advanced target detection methods usually use horizontal prediction boxes to detect targets. However, due to the changes in the attitude of satellite remote sensing imaging and the different distribution states of superimposed ground targets, the targets in satellite remote sensing images often have directional diversity and show the characteristics of random angle rotation, which is very different from the horizontal perspective satellite remote sensing images in life scenes. High-precision target detection in satellite remote sensing images requires the prediction of accurate bounding boxes. Traditional target detectors use horizontal prediction boxes to locate targets, which has certain limitations in predicting target angle and position information, resulting in the area enclosed by the boundary being larger than the actual target shape. Compared with the horizontal prediction box, the rotated bounding box adds rotation angle information and can detect the rotation direction of the target, so it can accurately match the target's direction information. Therefore, target detection based on rotated bounding boxes has gradually become a topic worthy of research in the field of satellite remote sensing images.

[0003] The current CNN-based rotation target detection schemes can be mainly summarized into three categories: data enhancement strategy, defining regression angles, and defining vertex offsets.

[0004] Among them, the data enhancement strategy rotates and expands the satellite remote sensing images to make the directions of the learning samples diverse, so that the network has a certain degree of rotation invariance. This method cannot effectively extract the refined spatial angle features for densely distributed rotating targets.

[0005] Defining the regression angle is to regress the angle parameter information of the target in the positioning stage, which is a processing method from continuous to discrete angle. This method will produce a large loss in the process of target positioning, resulting in a certain loss of spatial position information.

[0006] Defining vertex offsets is to sort the four vertices of the detection box, and then calculate the offset distance of each vertex in its sliding direction, and the angle parameter does not need to be directly involved. This method is improved based on the horizontal prediction box method. It is necessary to predict the horizontal box first and then regress the offset distance. The prediction is difficult and not conducive to model training.

[0007] In addition, there are methods that model Gaussian distribution for rotated targets, and then calculate the distribution metric distance between the predicted box and the true box, which solves the problem that the intersection over union (IoU) cannot be back-propagated at non-differentiable points due to the discontinuity of the predicted box. However, such methods do not have scale invariance or computational symmetry when calculating distribution distances. Summary of the invention

[0008] In view of the above technical problems, the present invention provides a method and system for fine detection of rotated targets based on Bhattacharyya distance metric loss function. The method solves the technical problem that the feature map extracted by the deep learning network is usually strongly interfered by densely distributed targets, and the refined position features of densely distributed rotated targets are difficult to effectively extract. The present invention can minimize the loss of spatial position information in the bounding box regression process, effectively realize the fine extraction of features, and improve the accuracy of rotated target detection.

[0009] In order to achieve the above object, the present invention provides a method for fine detection of rotation targets based on Bhattacharyya distance metric loss function, the method comprising:

[0010] Step 1: Obtain a sample data set of rotation target detection in visible light remote sensing images, unify the label format of the sample data set, and divide it into a training sample set and a test sample set;

[0011] Step 2: Construct a rotating target fine detection network based on convolutional neural network;

[0012] Step 3: Take the training sample set as input, construct the Bhattacharyya distance metric loss function as the objective function to train the rotation target fine detection network, and obtain the trained rotation target fine detection network;

[0013] Step 4: Input the test sample set into the trained rotating target fine detection network and output the precise position and category of the target.

[0014] The visible light remote sensing image rotation target detection sample data set in step 1 is composed of samples and labels corresponding to the samples;

[0015] The sample is an image region including the target to be detected cut out from a visible light remote sensing image taken by a satellite;

[0016] The label is used to mark the image area position of the target to be detected in the sample using a rotated bounding box; the rotated bounding box is in the format of a label.

[0017] The format of the label in step 1 is used to represent the position information of the image area of ​​the target to be detected in the sample, specifically: (x, y, w, h, θ);

[0018] Among them, x represents the horizontal coordinate of the center point of the bounding box; y represents the vertical coordinate of the center point of the bounding box; w represents the width of the bounding box, h represents the height of the bounding box; θ represents the acute angle between the bounding box and the horizontal direction.

[0019] The rotation target fine detection network constructed based on the convolutional neural network in step 2 includes: a feature pyramid network, a feature aggregation module, and a classification and position regression subnetwork;

[0020] Feature pyramid network, used to extract feature maps of different scales in the sample set, and input the extracted feature maps of different scales into the feature aggregation module;

[0021] The feature aggregation module is used to aggregate the feature maps of different scales output by the feature pyramid network through upward propagation and skip connections to generate global features for classification and position regression; the global features are input into the classification and position regression subnetworks;

[0022] The classification and position regression subnetwork is used to determine the category of the target based on the input global features through the classification subnetwork; at the same time, the Bhattacharyya distance learning module based on Gaussian distribution regresses the input global features to the target position.

[0023] Among them, the Bhattacharyya distance learning module based on Gaussian distribution is a learnable regression module, which is used as a similarity constraint between two Gaussian distributions in arbitrary directions, and the constraint information is the position information of the image area of ​​the target to be detected in the sample.

[0024] Furthermore, the step of regressing the input global features to the position of the target in the Bhattacharyya distance learning module based on Gaussian distribution includes: modeling the bounding box of the target in the input global features as a two-dimensional Gaussian distribution and then performing a distribution measurement calculation of the Bhattacharyya distance, thereby regressing the input global features to the position information of the image area of ​​the target to be detected in the sample;

[0025] The formula for calculating the distribution metric of the Bhattacharyya distance after modeling the bounding box of the target in the input global feature as a two-dimensional Gaussian distribution includes:

[0026]

[0027] Among them, BD(N P ,NG ) is the normalized Bhattacharyya distance; τ is the offset hyperparameter; f(.) is the nonlinear expression of the distance function, including logarithmic or square operations; BD(N P ||N G ) is the Gaussian distribution N of the prediction box P P (u p ,A p ) and the Gaussian distribution N of the true box G G (u g ,A g );Specifically expressed as:

[0028]

[0029] Where A is the covariance matrix; (.) T Represents the transpose of a matrix.

[0030] In the step 3, the training sample set is used as input, and the Bhattacharyya distance metric loss function is constructed as the objective function to train the rotation target fine detection network to obtain a trained rotation target fine detection network, which specifically includes:

[0031] The training sample set is input into the rotating target fine detection network, and the Adam optimizer is used for gradient descent. The number of iterations, initial learning rate, and batch size are set to gradually reduce the value of the Bhattacharyya distance metric loss function. At the same time, the weights of the rotating target fine detection network are updated. After the iteration is completed, the trained rotating target fine detection network is obtained.

[0032] The Bhattacharyya distance measurement loss function L in step 3 specifically includes:

[0033]

[0034] Among them, the first hyperparameter λ1 and the second hyperparameter λ2 control the weights of the cross entropy loss function and the regression loss function; N is the number of anchor boxes input to the classification and position regression subnetwork; L cls is the cross entropy loss function; p n represents the probability distribution of each category calculated by the Sigmoid function in the classification and position regression subnetwork, g n is the true label of the nth target of the sample in the training sample set; Represents the positive and negative sample index of the current anchor box; if the nth anchor box is a positive sample, then The value is 1, otherwise it is 0; L reg is the regression loss function; b n Represents the nth bounding box in the location regression subnetwork.

[0035] Among them, L regThe regression loss function is:

[0036] L reg =1-BD(N P ,N G );

[0037] Among them, BD(N P ,N G ) is the normalized Bhattacharyya distance.

[0038] In the fourth step, the test sample set is input into the trained rotating target fine detection network, and the precise position and category of the target are output, which specifically includes:

[0039] Step S110: The feature pyramid network in the trained rotation target fine detection network extracts a multi-scale feature map F in the test sample set. i ; The feature aggregation module extracts the multi-scale feature map F from bottom to top i Perform re-enhancement transmission to obtain the feature aggregation module feature layer R i , and splice and fuse; denoted as O s =Det[S;∑ * (ψ(F i ,R i ))];

[0040] Among them, O s is the detection process of the target in the test sample set in the trained rotation target fine detection network; Det(.) represents the detection process of the target in the trained rotation target fine detection network; S is the test sample set; ∑ * Represents the bottom-up transfer and fusion operation of the feature layer in the feature aggregation module; ψ represents the operation of transferring the feature map of the feature pyramid network to the feature aggregation module layer by layer;

[0041] Step S120: Each feature layer R of the feature aggregation module i After the classification and position regression subnetwork, the category and position of the rotating box are accurately predicted within the constrained range, and the precise position and category of the target are finally output; it is represented by O s =Det{S,C[∑ * (ψ(F i ,R i ))]};

[0042] Among them, C(.) represents the loss constraint of the classification and position regression sub-network.

[0043] The present invention also provides a system for fine detection of rotating targets based on Bhattacharyya distance metric loss function. The system includes at least one computer device, including a processor, a memory, and an executable program in the memory. After the program is loaded, the method for fine detection of rotating targets based on Bhattacharyya distance metric loss function is implemented.

[0044] The beneficial effects of the present invention are as follows:

[0045] The present invention proposes a method and system for fine detection of rotated targets based on the Bhattacharyya distance metric loss function. The designed loss function based on the Bhattacharyya distance metric can dynamically adjust the gradient through the horizontal and vertical coordinates, length, frame, and rotation angle of the center point of the bounding box, and is applied to gradient descent and parameter updating to effectively and accurately locate the target bounding box.

[0046] The designed feature aggregation module FAM can completely retain the feature map with spatial features and semantic information, realize the refined extraction of target features, effectively suppress the interference of complex background with strong texture in remote sensing images on the target, and improve the detection accuracy of targets under complex background.

[0047] The Bhattacharyya distance learning module based on Gaussian distribution in the classification and position regression sub-network is used to model the rotated bounding box of the target as a two-dimensional Gaussian distribution. This can solve the loss error problem caused by the angle periodicity and the change of the length and short sides of the rotating target, make the loss function measurement more accurate, and reduce the difficulty of network training. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A flow chart of a method for fine detection of rotation targets based on Bhattacharyya distance metric loss function proposed by the present invention;

[0049] Figure 2 This is a schematic diagram of the structure of the rotating target fine detection network proposed by the present invention;

[0050] Figure 3 This is a schematic diagram of the feature aggregation module proposed in the present invention;

[0051] Figure 4 is a test sample in an embodiment of the present invention;

[0052] Figure 5 It is a schematic diagram of the detection effect of the precise position and category of the target output in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] Embodiment 1

[0054] refer to Figure 1 As shown, the first embodiment of the present invention provides a method for fine detection of a rotating target based on a Bhattacharyya distance metric loss function, the method comprising:

[0055] Step 1: Obtain a sample data set of rotation target detection in visible light remote sensing images, unify the label format of the sample data set, and divide it into a training sample set and a test sample set;

[0056] Step 2: Construct a rotating target fine detection network based on convolutional neural network;

[0057] Step 3: Take the training sample set as input, construct the Bhattacharyya distance metric loss function as the objective function to train the rotation target fine detection network, and obtain the trained rotation target fine detection network;

[0058] Step 4: Input the test sample set into the trained rotating target fine detection network and output the precise position and category of the target.

[0059] The visible light remote sensing image rotation target detection sample data set in step 1 is composed of samples and labels corresponding to the samples;

[0060] The sample is an image region including the target to be detected cut out from a visible light remote sensing image taken by a satellite;

[0061] The label is used to mark the image area position of the target to be detected in the sample using a rotated bounding box; the rotated bounding box is in the format of a label.

[0062] The format of the label in step 1 is used to represent the position information of the image area of ​​the target to be detected in the sample, specifically: (x, y, w, h, θ);

[0063] Among them, x represents the horizontal coordinate of the center point of the bounding box; y represents the vertical coordinate of the center point of the bounding box; w represents the width of the bounding box, h represents the height of the bounding box; θ represents the acute angle between the bounding box and the horizontal direction.

[0064] The rotating target fine detection network in step 2 includes: Feature Pyramid Networks (FPN), Feature Aggregation Module (FAM) and classification and position regression subnetwork;

[0065] Feature pyramid network, used to extract feature maps of different scales in the sample set, and input the extracted feature maps of different scales into the feature aggregation module;

[0066] Specifically, it includes: Feature Pyramid Network (FPN), which first uses five consecutive convolutional layers to extract features from bottom to top, retaining the features of four different scales output by the second to fifth convolutional layers. Then, the features generated by the fifth convolutional layer are added with 1×1 convolution to obtain the fifth feature map, and the fifth feature map is upsampled three times. Then, the results of the three upsamplings are connected with the features generated by the fourth, third, and second convolutional layers using 1×1 convolution to obtain the fourth, third, and second feature maps. The feature pyramid network obtains a total of four feature maps of different scales, from the second to the fifth feature maps from bottom to top.

[0067] The feature aggregation module FAM is used to aggregate the feature maps of different scales output by the feature pyramid network through upward propagation and skip connections to generate global features for classification and position regression; the global features are input into the classification and position regression subnetworks;

[0068] Specifically, it includes: Feature Aggregation Module (FAM), which aggregates the feature maps of different scales output by the feature pyramid network through upward propagation and jump connection to avoid feature loss and generate global features for classification and position regression: First, the second feature map of the lowest layer output by the feature pyramid network FPN is used as the second lowest feature layer of the feature aggregation module, and the third feature map output by the feature pyramid network is horizontally connected and added with the corresponding second feature layer in the feature aggregation module FAM to generate the third feature layer of the feature aggregation module, and the feature mapping space dimensions at both ends of the horizontal connection are ensured to be the same, both 256, and the feature is transferred from bottom to top. The fourth and fifth feature layers are generated in this way, and the second, third, and fourth feature layers of the feature aggregation module are jump-connected with the fifth feature layer. Then, the second to fifth feature layers are spliced, and then the number of channels of the spliced ​​features is compressed through one-dimensional convolution to generate global features for classification and regression tasks.

[0069] The feature aggregation module FAM module retains the features in the shallow network through upward propagation, which contains a large amount of target edge contour information. The spatial features and semantic information in this information will be of great help to the angle regression of the rotation target, and can achieve refined extraction of target features. The feature layer directly propagates the features of the local branches upward through jump connections, which can efficiently integrate each feature layer into a more representative feature, and can suppress the interference of the complex background with strong textures in remote sensing images on the target.

[0070] The classification and position regression subnetwork is used to determine the category of the target through the classification subnetwork based on the input global features; at the same time, the Bhattacharyya distance learning module based on Gaussian distribution regresses the input global features to the target position. The classification and position regression subnetwork is based on the classification subnetwork and position regression subnetwork of RetinaNet, and the position regression subnetwork is optimized to a Bhattacharyya distance learning module (BDLM) based on Gaussian distribution, which can regress the target position information more accurately.

[0071] Among them, the Bhattacharyya distance learning module based on Gaussian distribution is a learnable regression module, which is used as a similarity constraint between two Gaussian distributions in arbitrary directions, and the constraint information is the position information of the image area of ​​the target to be detected in the sample.

[0072] Furthermore, the step of regressing the input global features to the position of the target in the Bhattacharyya distance learning module based on Gaussian distribution includes: modeling the bounding box of the target in the input global features as a two-dimensional Gaussian distribution and then performing a distribution measurement calculation of the Bhattacharyya distance, thereby regressing the input global features to the position information of the image area of ​​the target to be detected in the sample;

[0073] The formula for calculating the distribution metric of the Bhattacharyya distance after modeling the bounding box of the target in the input global feature as a two-dimensional Gaussian distribution includes:

[0074]

[0075] Among them, BD(N P ,N G ) is the normalized Bhattacharyya distance; τ is the offset hyperparameter; f(.) is the nonlinear expression of the distance function, including logarithmic or square operations; BD(N P ||N G ) is the Gaussian distribution N of the prediction box P P (u p ,A p ) and the Gaussian distribution N of the true box G G (u g ,A g );Specifically expressed as:

[0076]

[0077] Where A is the covariance matrix; (.) T Represents the transpose of a matrix.

[0078] The target's rotated bounding box (x, y, w, h, θ) is modeled as a two-dimensional Gaussian distribution N(u, A). Using a two-dimensional Gaussian distribution instead of a bounding box for training can solve the problem of loss errors caused by the periodicity of the angle and the change of the length and short sides of the bounding box during rotation. For example: for the label (x, y, w, h, θ) of the training sample set, construct a two-dimensional Cartesian coordinate system rotation matrix M(θ), Using orthogonal matrix multiplication to represent the transformation of the rotation box, we get a two-dimensional Gaussian distribution N(u,A), where the variable mean u of the Gaussian distribution and the covariance matrix A are expressed as in It is the diagonal matrix of the rotation eigenvalues, which represents the geometric stretching relationship of the rectangular box during the rotation process.

[0079] The distribution metric calculation of Bhattacharyya distance includes: using the position regression subnetwork to regress the predicted box, and converting the regressed predicted box P and the true box G into two two-dimensional Gaussian distributions: N P (u p ,A p ), N G (u g ,A g ), then the distance between two Gaussian distributions is:

[0080] This formula represents the distance between two Gaussian distributions, and is used to further represent the similarity measure with angle information between two rotating frames. A nonlinear relationship is established between the distance function f(.) and BD(.||.). BD(.||.) is converted into a fractional form and normalized to obtain a new similarity measure, namely the normalized Bhattacharyya distance:

[0081]

[0082] Among them, τ is the offset hyperparameter, and f(.) is the nonlinear expression of the distance function. In general, f(.) has two forms, namely, taking the logarithm operation or the square operation. This formula is symmetrical for the distance calculation between the two distributions, indicating that the BMLD module can perform symmetrical and stable distribution measurement calculations on the spatial relative position information through the normalized Bhattacharyya distance.

[0083] Among them, the predicted box is the box output by the classification and position regression subnetwork; the real box is the rotated bounding box of the target annotation of the training sample set.

[0084] In the step 3, the training sample set is used as input, and the Bhattacharyya distance metric loss function is constructed as the objective function to train the rotation target fine detection network to obtain a trained rotation target fine detection network, which specifically includes:

[0085] The training sample set is input into the rotating target fine detection network, and the Adam optimizer is used for gradient descent. The number of iterations, initial learning rate, and batch size are set to gradually reduce the value of the Bhattacharyya distance metric loss function. At the same time, the weights of the rotating target fine detection network are updated. After the iteration is completed, the trained rotating target fine detection network is obtained.

[0086] The loss function L of the Bhattacharyya distance metric in step 3 specifically includes:

[0087]

[0088] Among them, the first hyperparameter λ1 and the second hyperparameter λ2 control the weights of the cross entropy loss function and the regression loss function; N is the number of anchor boxes input to the classification and position regression subnetwork; L cls is the cross entropy loss function; p n represents the probability distribution of each category calculated by the Sigmoid function in the classification and position regression subnetwork, g n is the true label of the nth target of the sample in the training sample set; Represents the positive and negative sample index of the current anchor box; if the nth anchor box is a positive sample, then The value is 1, otherwise it is 0; L reg is the regression loss function; b n Represents the nth bounding box in the position regression subnetwork. An anchor box is a box with multiple scales and multiple aspect ratios. In the output of FAM, it is a box of different proportions and sizes. If the IOU between the anchor and the real box is greater than 0.5, it is considered a positive sample, otherwise it is a background.

[0089] Among them, L reg The regression loss function is:

[0090] L reg =1-BD(N P ,N G );

[0091] Among them, BD(N P ,N G ) is the normalized Bhattacharyya distance.

[0092] The loss function based on the Bhattacharyya distance metric can dynamically adjust the gradient through the five elements of the bounding box (x, y, w, h, θ), the horizontal and vertical coordinates of the center point, length, frame, and rotation angle, and is applied to gradient descent and parameter update to effectively and accurately locate the target rotation box.

[0093] In the fourth step, the test sample set is input into the trained rotating target fine detection network, and the precise position and category of the target are output, which specifically includes:

[0094] Step S110: The feature pyramid network in the trained rotation target fine detection network extracts a multi-scale feature map F in the test sample set. i ; The feature aggregation module extracts the multi-scale feature map F from bottom to top i Perform re-enhancement transmission to obtain the feature aggregation module feature layer R i , and splice and fuse; denoted as O s =Det[S;∑ * (ψ(F i ,R i ))];

[0095] Among them, O s is the detection process of the target in the test sample set in the trained rotation target fine detection network; Det(.) represents the detection process of the target in the trained rotation target fine detection network; S is the test sample set; ∑ * Represents the bottom-up transfer and fusion operation of the feature layer in the feature aggregation module; ψ represents the operation of transferring the feature map of the feature pyramid network to the feature aggregation module layer by layer;

[0096] Step S120: Each feature layer R of the feature aggregation module i After the classification and position regression subnetwork, the category and position of the rotating box are accurately predicted within the constrained range, and the precise position and category of the target are finally output; it is represented by O s =Det{S,C[∑ * (ψ(F i ,R i ))]};

[0097] Among them, C(.) represents the loss constraint of the classification and position regression sub-network.

[0098] An embodiment of the present invention also provides a system for fine detection of rotating targets based on Bhattacharyya distance metric loss function. The system includes at least one computer device, including a processor, a memory, and an executable program in the memory. After the program is loaded, the method for fine detection of rotating targets based on Bhattacharyya distance metric loss function is implemented.

[0099] In order to more fully understand the present invention, a specific embodiment is now provided:

[0100] In an embodiment, step 1, use the DIOR dataset as a sample dataset for target visible light remote sensing image rotation target detection. The DIOR dataset is a large-scale public dataset of optical remote sensing images, which contains different imaging conditions, weather, seasons and other environmental factors. The dataset has 23,463 800×800 pixel images and 192,472 target instances, with a total of 20 target categories, including airplanes, airports, baseball fields, basketball courts, bridges, chimneys, dams, highway service areas, highway toll stations, ports, golf courses, ground track and field stadiums, overpasses, ships, stadiums, storage tanks, tennis courts, railway stations, vehicles and windmills. The label format of the sample dataset is unified and divided into a training sample set and a test sample set, of which the training sample set used for training is 11,725 ​​and the test sample set used for testing is 11,738.

[0101] Step 2: Construct a rotating target fine detection network based on a convolutional neural network. The network structure diagram is shown in Figure 2 :The rotating target fine detection network consists of three parts: Feature Pyramid Networks (FPN), Feature Aggregation Module (FAM), and classification and position regression sub-network. Among them: The constructed feature pyramid network is used to extract feature maps of different scales in the sample set, and the extracted feature maps of different scales are input into the feature aggregation module;

[0102] The feature aggregation module is used to aggregate the feature maps of different scales output by the feature pyramid network through upward propagation and skip connections to generate global features for classification and position regression; the global features are input into the classification and position regression subnetworks;

[0103] The classification and position regression subnetwork is used to determine the category of the target based on the input global features through the classification subnetwork; at the same time, the Bhattacharyya distance learning module based on Gaussian distribution regresses the input global features to the target position.

[0104] Step 3: Take the training sample set as input, construct the Bhattacharyya distance metric loss function as the objective function to train the rotation target fine detection network, and obtain the trained rotation target fine detection network.

[0105] Specifically, it includes: Feature Pyramid Network (FPN), which is used to extract feature map information of different scales: for the samples in the input training sample set, four consecutive convolution layers are first used to extract features from the bottom to the top to generate features of four different scales. Then, a 1×1 convolution is added to the features generated by the top convolution layer to obtain feature map F5, and then F5 is upsampled three times by a factor of 2. Then, the results of these three times of upsampling are added element by element to the first three different scales of features extracted from the bottom to the top to obtain multi-scale feature maps F4, F3, and F2. The feature pyramid network obtains a total of four feature maps F of different scales. i (i=5,4,3,2).

[0106] Feature Aggregation Module (FAM), see Figure 3 , which is used to aggregate the feature maps of different scales output by the feature pyramid network through upward propagation and skip connections to avoid feature loss and generate global features for classification and position regression: First, the feature map F2 output by FPN is used as the lowest feature layer R2 of the feature aggregation module, and each layer of feature map F i (i=3,4,5) and the corresponding R in FAM i-1 Lateral concatenation and summing generate R i , and ensure that the spatial dimensions of the feature maps at both ends of the horizontal connection are the same, both 256, for bottom-up feature transfer. i (i=2,3,4) and the top feature layer R5 are connected by skipping. i The concatenation of (i=2,3,4,5) is performed, and then the number of channels of the concatenated features is compressed through one-dimensional convolution to generate global features for classification and regression tasks.

[0107] The FAM module retains a large amount of target edge contour information in the shallow network through upward propagation, which is very helpful for the angle regression of the rotated target. The feature layer directly propagates the features of the local branch upward through jump connections, which can efficiently integrate each feature layer into a more representative feature.

[0108] The global features generated by FAM are input into the optimized classification and position regression subnetwork to obtain the target category and position. The optimized classification and position regression subnetwork is based on the classification and position regression subnetwork of RetinaNet, and the position regression subnetwork is optimized to a Bhattacharyya distance learning module (BDLM) based on Gaussian distribution, so that the position regression subnetwork can more accurately regress the position angle and coordinates of the target.

[0109] The BMLD module is a learnable regression model used as a similarity constraint for two Gaussian distributions in arbitrary directions. The constraint information includes position angle information and coordinate information. The BMLD module regresses the input global features to the target position, which mainly includes two steps: modeling the target bounding box as a two-dimensional Gaussian distribution, and calculating the distribution metric of the Bhattacharyya distance.

[0110] The target bounding box is modeled as a two-dimensional Gaussian distribution. Specifically, for the label B(x, y, w, h, θ) of the training sample set, a two-dimensional Cartesian coordinate rotation matrix M(θ) is constructed. Using orthogonal matrix multiplication to represent the transformation of the rotation box, we get a two-dimensional Gaussian distribution N(u,A), where the variable mean u of the Gaussian distribution and the covariance matrix A are expressed as in It is the diagonal matrix of the rotation eigenvalues, which represents the geometric stretching relationship of the rectangular box during the rotation process.

[0111] Distribution metric calculation of Bhattacharyya distance. Convert the regressed predicted box P and the true box G into two two-dimensional Gaussian distributions: N P (u p ,A p ), N G (u g ,A g ), then the distance between two Gaussian distributions is:

[0112] This formula represents the distance between two Gaussian distributions, and is used to further represent the similarity measure with angle information between two rotating frames. A nonlinear relationship is established between the distance function f(.) and BD(.||.). BD(.||.) is converted into a fractional form and normalized to obtain a new similarity measure, namely the normalized Bhattacharyya distance:

[0113]

[0114] Among them, τ is the offset hyperparameter, and f(.) is the nonlinear expression of the distance function. In general, f(.) has two forms, namely, taking the logarithm operation or the square operation. This formula is symmetrical for the distance calculation between the two distributions, indicating that the BMLD module can perform symmetrical and stable distribution measurement calculations on the spatial relative position information through the normalized Bhattacharyya distance.

[0115] In this embodiment, Lenovo P710 workstation, Quadro GV100 graphics card, 32G video memory are used. The backbone network of this method and each comparative method is ResNet50, and the Adam optimizer is used for gradient descent, iterates 282,000 steps, the initial learning rate is set to 0.0005, and the batch size is set to 8. In addition, the hyperparameters λ1 and λ2 in the total loss function L are set to 1 and 2 respectively. After the iteration is completed, a trained rotation target fine detection network is obtained.

[0116] Step 4: Input the test sample set into the trained rotating target fine detection network and output the detection results of the precise position and category of the target.

[0117] In this embodiment, the samples used for testing in the DIOR dataset are used for testing, and the average precision AP (%) and mean average precision mAP (%) of different rotation object detectors in the DIOR dataset are compared. The model performance comparison results are shown in Table 1.

[0118] Table 1

[0119]

[0120] It can be seen from Table 1 that the detection accuracy and average detection accuracy of most categories of the present invention are superior to other existing methods, and the detection performance has been effectively improved. Figure 5 , Figure 4 For the test sample, Figure 5 This is the detection result of the present invention. The detected bounding box accurately surrounds the true shape of the target and has rotation angle information.

[0121] An embodiment of the present invention also provides a system for fine detection of rotating targets based on Bhattacharyya distance metric loss function. The system includes at least one computer device, including a processor, a memory, and an executable program in the memory. After the program is loaded, the method for fine detection of rotating targets based on Bhattacharyya distance metric loss function is implemented.

[0122] The beneficial effects of the embodiments of the present invention are as follows:

[0123] The present invention proposes a method and system for fine detection of rotated targets based on the Bhattacharyya distance metric loss function. The designed loss function based on the Bhattacharyya distance metric can dynamically adjust the gradient through the horizontal and vertical coordinates, length, frame, and rotation angle of the center point of the bounding box, and is applied to gradient descent and parameter updating to effectively and accurately locate the target bounding box.

[0124] The designed feature aggregation module FAM can completely retain the feature map with spatial features and semantic information, realize the refined extraction of target features, effectively suppress the interference of complex background with strong texture in remote sensing images on the target, and improve the detection accuracy of targets under complex background.

[0125] The Bhattacharyya distance learning module based on Gaussian distribution in the classification and position regression sub-network is used to model the rotated bounding box of the target as a two-dimensional Gaussian distribution. This can solve the loss error problem caused by the angle periodicity and the change of the length and short sides of the rotating target, make the loss function measurement more accurate, and reduce the difficulty of network training.

Claims

1. A method for fine detection of rotating targets based on Bhattacharyya distance metric loss function, characterized in that: The method includes: Step 1: Obtain a sample data set of rotation target detection in visible light remote sensing images, unify the label format of the sample data set, and divide it into a training sample set and a test sample set; Step 2: construct a rotating target fine detection network based on a convolutional neural network, the rotating target fine detection network includes: a feature pyramid network, a feature aggregation module and a classification and position regression subnetwork; the feature pyramid network is used to extract feature maps of different scales in the sample set, and the extracted feature maps of different scales are input into the feature aggregation module; the feature aggregation module is used to perform feature aggregation on the feature maps of different scales output by the feature pyramid network through upward propagation and jump connection to generate global features for classification and position regression; the global features are input into the classification and position regression subnetwork; the classification and position regression subnetwork is used to judge the category of the target through the classification subnetwork according to the input global features; at the same time, the Bhattacharyya distance learning module based on Gaussian distribution regresses the input global features to the position of the target; Step 3: Take the training sample set as input, construct the Bhattacharyya distance metric loss function as the objective function to train the rotation target fine detection network, and obtain the trained rotation target fine detection network; Bhattacharyya distance measurement loss function L, specifically includes: Among them, the first hyperparameter λ1 and the second hyperparameter λ2 control the weights of the cross entropy loss function and the regression loss function; N is the number of anchor boxes input to the classification and position regression subnetwork; L cls is the cross entropy loss function; p n represents the probability distribution of each category calculated by the Sigmoid function in the classification and position regression subnetwork, g n is the true label of the nth target of the sample in the training sample set; Represents the positive and negative sample index of the current anchor box; if the nth anchor box is a positive sample, then The value is 1, otherwise it is 0; L reg is the regression loss function; b n Represents the nth bounding box in the position regression subnetwork; Step 4: Input the test sample set into the trained rotating target fine detection network and output the precise position and category of the target.

2. The method according to claim 1, characterized in that The visible light remote sensing image rotation target detection sample data set in step 1 is composed of samples and labels corresponding to the samples; The sample is an image region including the target to be detected cut out from a visible light remote sensing image taken by a satellite; The label is used to mark the image area position of the target to be detected in the sample using a rotated bounding box; the rotated bounding box is in the format of the label.

3. The method according to claim 1 or 2, characterized in that The format of the label in step 1 is used to represent the position information of the image area of ​​the target to be detected in the sample, specifically: (x, y, w, h, θ); Among them, x represents the horizontal coordinate of the center point of the bounding box; y represents the vertical coordinate of the center point of the bounding box; w represents the width of the bounding box, h represents the height of the bounding box; θ represents the acute angle between the bounding box and the horizontal direction.

4. The method according to claim 1, characterized in that The Bhattacharyya distance learning module based on Gaussian distribution is a learnable regression module, which is used as a similarity constraint between two Gaussian distributions in arbitrary directions, and the constraint information is the position information of the image area of ​​the target to be detected in the sample.

5. The method according to claim 1 or 4, characterized in that The step of regressing the input global features to the position of the target in the Bhattacharyya distance learning module based on Gaussian distribution includes: modeling the bounding box of the target in the input global features as a two-dimensional Gaussian distribution and then performing a distribution measurement calculation of the Bhattacharyya distance, thereby regressing the input global features to the position information of the image area of ​​the target to be detected in the sample; The formula for calculating the distribution metric of the Bhattacharyya distance after modeling the bounding box of the target in the input global feature as a two-dimensional Gaussian distribution includes: Among them, BD(N P ,N G ) is the normalized Bhattacharyya distance; τ is the offset hyperparameter; f(.) is the nonlinear expression of the distance function, including logarithmic or square operations; BD(N P ||N G ) is the Gaussian distribution N of the prediction box P P (u p ,A p ) and the Gaussian distribution N of the true box G G (u g ,A g );Specifically expressed as: Where A is the covariance matrix; (.) T Represents the transpose of a matrix.

6. The method according to claim 1, characterized in that In the step 3, the training sample set is used as input, and the Bhattacharyya distance metric loss function is constructed as the objective function to train the rotation target fine detection network to obtain a trained rotation target fine detection network, which specifically includes: The training sample set is input into the rotating target fine detection network, and the Adam optimizer is used for gradient descent. The number of iterations, initial learning rate, and batch size are set to gradually reduce the value of the Bhattacharyya distance metric loss function. At the same time, the weights of the rotating target fine detection network are updated. After the iteration is completed, the trained rotating target fine detection network is obtained.

7. The method according to claim 5, characterized in that L reg The regression loss function is: L reg =1-BD(N P ,N G ); Among them, BD(N P ,N G ) is the normalized Bhattacharyya distance.

8. The method according to claim 1, characterized in that In the fourth step, the test sample set is input into the trained rotating target fine detection network, and the precise position and category of the target are output, which specifically includes: Step S110: The feature pyramid network in the trained rotation target fine detection network extracts a multi-scale feature map F in the test sample set. i ; The feature aggregation module extracts the multi-scale feature map F from bottom to top i Perform re-enhancement transmission to obtain the feature aggregation module feature layer R i , and splice and fuse; denoted as O s =Det[S;∑ * (ψ(F i ,R i ))]; Among them, O s is the detection process of the target in the test sample set in the trained rotation target fine detection network; Det(.) represents the detection process of the target in the trained rotation target fine detection network; S is the test sample set; ∑ * Represents the bottom-up transfer and fusion operation of the feature layer in the feature aggregation module; ψ represents the operation of transferring the feature map of the feature pyramid network to the feature aggregation module layer by layer; Step S120: Each feature layer R of the feature aggregation module i After the classification and position regression subnetwork, the category and position of the rotating box are accurately predicted within the constrained range, and the precise position and category of the target are finally output; it is represented as O s =Det{S,C[Σ * (ψ(F i ,R i ))]}; Among them, C(.) represents the loss constraint of the classification and position regression sub-network.

9. A system for fine detection of rotating targets based on Bhattacharyya distance metric loss function, the system comprising at least one computer device, including a processor, a memory and an executable program in the memory, which, after loading, implements the method for fine detection of rotating targets based on Bhattacharyya distance metric loss function as described in any one of claims 1 to 8.

Images