Aerial image target detection method and system based on difference feature learning

Abstract

The application relates to an aerial image target detection method and system based on difference feature learning, and belongs to the field of computer vision. The method adopts an anchor frame-free detection algorithm as a benchmark network framework, sets the anchor frame size and quantity, and uses an hourglass as a backbone network to extract features; three parallel branches are used to realize prediction of target size, category and difference features, high-quality features are collected in real time during network training and used for learning of the difference features in the next training of the specified network, and finally, the difference features are used to correct the wrong features so that the whole network inputs a more accurate feature; a head-tail sampling method is used to realize division of head-tail category targets so that the network can learn the features of the head-tail two types of targets during each iteration. The application can effectively solve the problem of incomplete feature information of the target itself caused by low resolution, shielding and blurring.

Description

Technical Field

[0001] This invention belongs to the field of computer vision, specifically the field of deep learning object detection, and specifically relates to a method and system for aerial image object detection based on differential feature learning. Background Technology

[0002] Object detection is a crucial research method in computer vision and a foundation for other complex visual tasks. Aerial image object detection has long been a challenging yet hotly debated area in computer vision. Influenced by shooting angle, altitude, and speed, these targets are constantly changing, leading to problems such as low resolution, occlusion, and blurring. In our daily lives, aerial image object detection is widely used in agriculture, industry, and other fields, where higher accuracy is required, as any omission or erroneous detection can have serious consequences.

[0003] Existing object detection algorithms are mainly divided into two categories: one-stage object detection algorithms and two-stage object detection algorithms. One-stage object detection algorithms directly predict the target without needing to predict coarse bounding boxes beforehand, while two-stage detection algorithms add a Region Proposal Network (RPN) to predict coarse bounding boxes, and then classify and correct the positions of these boxes. Two-stage object detection algorithms significantly improve accuracy, but their detection speed decreases. Both of these detection frameworks are anchor-based methods. When detecting small targets (such as aerial images), the performance of these methods degrades because it is difficult to set the proportion and number of anchor points.

[0004] In recent years, with the emergence of anchorless methods, the detection performance of small targets has been improved. Anchorless methods transform the object detection task into the prediction of key points, mainly including two categories: corner-based detection and center-based detection. These methods have innovated the detection process, but it is still difficult to learn effective features for detecting aerial images in complex environments, and the detection effect of targets is therefore limited. Therefore, a lot of work is currently focused on improving the network's ability to learn features, such as the earliest use of image pyramids and feature pyramids to improve the extraction of features of small targets. Subsequently, a large number of feature fusion methods have emerged to improve the utilization of features, mainly divided into three categories: (1) combining shallow and deep features by fusing features at different scales; (2) utilizing detailed features by fusing global and local features; (3) selecting effective features and suppressing ineffective features through attention mechanisms.

[0005] Current methods aggregate the features of each target into a single point or candidate box through numerous convolutions, without considering the inherent features of the target itself. For targets in aerial images, most suffer from feature loss; for example, occluded targets cannot be effectively extracted regardless of fusion methods, leading to false positives or false negatives.

[0006] Therefore, this invention designs a new aerial image target detection algorithm to solve the following problems existing in the prior art: 1) Target detection in aerial images belongs to small target detection in complex environments. Affected by shooting angle, height and speed, these targets will be constantly changing, and there are problems such as low resolution, occlusion and blur. Therefore, existing target detection algorithms cannot learn effective features; 2) There is a serious long tail problem in aerial datasets, and the long tail distribution of different datasets is very different. Summary of the Invention

[0007] In view of this, the purpose of the present invention is to provide a target detection method and system for aerial images based on differential feature learning, which is used to correct missing or erroneous features on the target and can effectively solve the problem of incomplete feature information of the target itself due to low resolution, occlusion and blur.

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] Solution 1: An aerial image target detection method based on differential feature learning. This method employs an anchor-free detection algorithm as the baseline network framework, setting the size and number of anchor boxes, and using hourglass as the backbone network to extract features. Three parallel branches are used to predict target size, category, and differential features. High-quality features are collected in real-time during network training and used to specify the learning of differential features in subsequent training iterations. Finally, differential features are used to correct erroneous features, resulting in a more accurate input to the entire network. Furthermore, to collect more balanced features across classes in each iteration, a head-and-tail sampling method is used to separate head and tail target categories, ensuring the network learns features from both categories in each iteration.

[0010] Furthermore, the method specifically includes the following steps:

[0011] S1: Divide the dataset into two subsets and randomly select the same number of images from each subset as input;

[0012] S2: Train a specific object detection network using the corresponding data, and collect the high-quality features extracted by the network in real time during the training process; the specific object detection network uses the Hourglass network as the backbone network to extract features, and uses three parallel branches to extract object size features, category features and differential features;

[0013] S3: Predict the differential features of target size, category, and corresponding location;

[0014] S4: Summing the difference features and the categorical features yields the feature. To achieve the correction of categorical features, and then to implement The convolution operation yields a new classification heatmap. ;

[0015] S5: Utilize and The algorithm filters candidate boxes from the prediction results of the category prediction branch to obtain candidate boxes. ;

[0016] S6: Heatmap obtained by predicting branches using differential features For candidate boxes The corresponding scores will be updated in the final analysis.

[0017] Furthermore, in step S2, three parallel branches are used to extract target size features, category features, and differential features. Specifically, this includes using two parallel detection heads to extract target size features and category features, obtaining the category features. and target size features And input Convolution; additionally, differential features can be obtained directly using a single branch. ;

[0018] The category prediction branch and the size prediction branch will combine the features from the previous layer. Send in Convolution yields prediction results and ,in, Each position represents a target. c Scores for each category Each position represents the height and width of the target at that position. b This indicates the batch size during network training. w Indicates the width of the feature map. h The height of the feature map is represented; then, a score heatmap is obtained based on the ground truth for each target. The target's category and size are regressed using the focal loss function and the L1 loss function, respectively; furthermore, the differential feature prediction branch is executed only once. Convolution yields a differential feature .

[0019] Furthermore, in step S2, because Different prediction errors can occur at these locations, such as foreground and background errors. This invention requires different loss regressions for these locations. Firstly, this invention needs to determine the maximum class score obtained from the class prediction branch. and the highest category score of the true label The locations are divided and different loss functions are obtained, as shown in the equation:

[0020]

[0021] in, This indicates the classification of locations on the feature map that meet the corresponding conditions. This represents the actual labeled score information. This indicates the score information of the network prediction. and This represents the threshold set when collecting high-quality features. This represents the true class of the target when the foreground is incorrect; then, different loss functions are set for different positions in the differential feature branch, as shown in the equation:

[0022]

[0023] in, Indicates the loss of differential features. n This indicates the number of feature vectors that need to be subjected to differential feature regression. This represents the loss corresponding to foreground error in the difference feature loss. This represents the loss corresponding to background error in the differential feature loss. This represents the difference feature loss at the remaining positions excluding the first two cases in the difference feature loss, where SmoothL1Loss is used as the final loss function in all cases. Indicates the first c High-quality features of class targets, This represents the differential features learned by the network. This indicates the features that need to be corrected in the category prediction branch. This represents the weight corresponding to each position;

[0024] Furthermore, during training, high-quality features are collected in each iteration. In the category prediction branch, not every position can learn effective features; therefore, this invention selectively retains high-quality features, first selecting all features in the current iteration that meet the following conditions:

[0025]

[0026] in, This represents the mean of the features that meet the conditions in the current iteration of the network. and These represent the maximum class score at each location obtained by applying the max function to the heatmap output by the category prediction branch and the actually labeled heatmap, respectively. This represents the threshold for category collection; then, high-quality features are updated:

[0027]

[0028] in, Indicates the first High-quality features from the next iteration. Indicates the preceding The next iteration collects more high-quality features. It is the weighted weight for updating high-quality features.

[0029] Furthermore, in step S6, the heatmap output by the difference feature branch is utilized. For candidate boxes Update the scores of those with excessively low scores:

[0030]

[0031] in, and Representing the target boxes The scores before and after the update. Indicates based on the target box central position and categories get H The score for the corresponding position in the middle. This indicates that the score updates the weight.

[0032] Option 2: An aerial image target detection system based on differential feature learning, comprising:

[0033] The dataset partitioning module is used to partition specific datasets in a targeted manner.

[0034] The feature extraction module is used to extract features required by downstream detection tasks using the Hourglass backbone network;

[0035] The target category prediction module is used to predict the target category at each location by utilizing the features learned by the upstream network.

[0036] The target size prediction module is used to predict the height and width of the target at each location by utilizing the features learned by the upstream network.

[0037] The difference feature prediction module is used to predict the difference features at each location by utilizing the features learned by the upstream network.

[0038] The high-quality feature collection module is used to collect high-quality features for each category obtained from the category prediction module;

[0039] The target bounding box filtering module is used to filter the target bounding boxes from the outputs of the target category prediction module and the target size prediction module.

[0040] The score update module is used to correct the category prediction features using the features output by the difference feature module, and then update the scores of the target boxes that do not meet the criteria for low scores.

[0041] Furthermore, the differential feature prediction module learns the differential features using the following formula:

[0042]

[0043] in, Indicates the loss of differential features. n This indicates the number of feature vectors that need to be subjected to differential feature regression. This represents the loss corresponding to foreground error in the difference feature loss. This represents the loss corresponding to background error in the differential feature loss. This represents the difference feature loss at the remaining positions excluding the first two cases in the difference feature loss, where SmoothL1Loss is used as the final loss function in all cases. Indicates the first c High-quality features of class targets, This represents the differential features learned by the network. This indicates the features that need to be corrected in the category prediction branch. This represents the weight corresponding to each position, as shown in the equation:

[0044]

[0045] Furthermore, the high-quality feature collection module obtains high-quality features using the following formula:

[0046]

[0047]

[0048] in, Indicates the first High-quality features from the next iteration. Indicates the preceding The next iteration collects more high-quality features. This represents the features that meet the conditions in the current iteration of the network. It is the weighted weight for updating high-quality features.

[0049] Furthermore, the target box filtering module updates the score using the following formula:

[0050]

[0051]

[0052] in, and Representing the target boxes The scores before and after the update. Indicates based on the target box central position and categories get H The score for the corresponding position in the middle. This indicates that the score updates the weight.

[0053] The beneficial effects of this invention are as follows: This invention can be used to correct missing or erroneous features on a target. This invention considers the learning of target features from a new perspective, that is, it uses the difference between high-quality features and other features to guide the network to learn features that other branches have not learned, rather than forcing the network to learn the global features of each target. This can effectively solve the problem of incomplete feature information of the target itself due to low resolution, occlusion, and blur.

[0054] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0055] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0056] Figure 1 This is an overall flowchart of the aerial image target detection method based on differential feature learning of the present invention;

[0057] Figure 2This is a framework diagram of an aerial image target detection method based on differential feature learning (training phase).

[0058] Figure 3 This is a framework diagram (inference stage) for an aerial image target detection method based on differential feature learning.

[0059] Figure 4 Visualize the differences in features;

[0060] Figure 5 shows a visualization of the prediction results of the HTS method of the present invention;

[0061] Figure 6 Visualize the prediction results of the DFL method of this invention;

[0062] Figure 7 This is a block diagram of the aerial image target detection system based on differential feature learning according to the present invention;

[0063] Figure 8 This is a block diagram of the terminal section. Detailed Implementation

[0064] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0065] Example 1

[0066] like Figure 1 The diagram shown is an overall flowchart of the aerial image target detection method based on differential feature learning of the present invention, which specifically includes the following steps:

[0067] S1: Divide the dataset into two subsets and randomly select the same number of images from each subset as input.

[0068] This embodiment uses the VisDrones aerial photography dataset as an example to detail the specific implementation process. First, due to the significant differences in category distribution across different datasets, this invention analyzes the target category distribution within the dataset and designates categories with fewer targets as tail categories and others as head categories. Then, this invention partitions the dataset, sending images containing more than 5 tail targets to the tail subset and the rest to the head subset. During the training phase, this invention selects images from each of these two categories... The images are processed, randomly cropped, and then sent to the network.

[0069] S2: Train a specific object detection network using the corresponding data, and collect the high-quality features extracted by the network in real time during the training process.

[0070] In this invention, a baseline network is constructed for target detection. Within this baseline network, the Hourglass network is selected as the backbone to extract features. Hourglass was initially proposed for human pose estimation and subsequently used in anchorless methods such as CornerNet and CenterNet. Here, the detection of small targets can be approximated as keypoint detection. Aerial image target detection is also a type of small target detection, hence the selection of Hourglass as the backbone network, which demonstrates good detection performance. Furthermore, this invention uses two parallel detection heads to extract target size and category features, obtaining feature category features. and target size features And input Convolution. Additionally, differential features are obtained directly using a single branch. .

[0071] like Figure 2 The category prediction branch and the size prediction branch will combine the features from the previous layer. Send in Convolution yields prediction results and ,in Each position represents the target's scores in c categories. Each position represents the height and width of the target at that position. Then, a score heatmap is obtained based on the ground truth of each target. This invention uses focus loss and L1 loss functions to regress the target's category and size, respectively; for details, please refer to CornerNet. Furthermore, in the differential feature prediction branch, this invention only executes the function once. Convolution yields a differential feature Differences The 256-dimensional feature vector at each location and the features used for score prediction Each position corresponds one-to-one with a 256-dimensional feature vector. Because This can cause different prediction errors in the image, such as foreground errors and background errors. This invention requires different loss regressions for these locations. Firstly, this invention needs to... Output the corresponding score heatmap Then, the score for each position is calculated using the max function. and true labeled score The location is divided. If the category prediction branch of this invention predicts a target (i.e. 1) If the predicted score is below a certain threshold, this invention refers to it as a foreground error. Furthermore, this threshold corresponds to the threshold used when collecting high-quality features. Therefore, this invention only performs differential feature learning on features with scores below the threshold to prevent side effects on features that would otherwise have higher scores and be more accurate. If the category prediction branch of this invention will target the background (i.e....) Predicting a foreground error, i.e., its classification score. In this invention, such errors are referred to as background errors. Similarly, this invention only performs differential feature learning on features of targets with a classification score greater than 0.1, because when filtering candidate boxes, target boxes with scores less than 0.1 will be directly deleted. The final error type classification is as follows:

[0072]

[0073] in, and This invention represents the collection of the first The threshold set when considering high-quality features of the target class, ultimately The value is set to 0.5. Then, this invention sets different loss functions for different errors in the differential feature branch. Foreground errors are mostly due to occlusion or interference from similar targets, resulting in missing or incorrect features. Therefore, their categorical features and high-quality features differ to some extent. Ultimately, the difference between high-quality features and categorical features is used to determine the learning of these differential features. Secondly, for background errors, due to the variability of background features, this invention cannot collect a unified background feature set. Therefore, this invention regresses the differential features of these errors to the inverse of the categorical features. This is because if... converges to After performing the operation using the following formula, the score will be... With a value less than 0.1, the background error is well suppressed. ,in express Convolution operation, This indicates that the maximum score for each category is obtained at each position. Furthermore, since the features at other positions do not require adjustment, the difference features at these positions are regressed to 0 to avoid interfering with the category features. The correct features are identified. Finally, based on different errors, a three-part loss function is obtained, as shown in the equation:

[0074]

[0075] in, This represents the SmoothL1Loss loss function. This represents the high-quality features of the c-th target, meaning that for foreground errors, this invention needs to learn the high-quality features corresponding to the true class c. Between and categorical features The differences. To make the network pay more attention to features that are not predicted accurately, this invention assigns corresponding weights to each location-difference feature. For example, in the case of a foreground error, when the target score... The closer The smaller the weight, the more the network will focus on learning the differential features of targets with lower scores. Based on its score It is calculated from the threshold, as shown in the equation:

[0076]

[0077] Among them, when In order to conform to the prediction of heatmap in Focal Loss, that is, the balance between positive and negative samples, this invention calculates the weights based on the true labeled scores. .

[0078] Finally, we obtain the final total loss, as shown in the equation: ,in, This represents the loss weight of the target size prediction branch, which is ultimately set to 0.1 in this invention.

[0079] Furthermore, during training, this invention collects high-quality features for each target class based on the predicted scores in the target class detection branch. And weighted by the features already collected. For example... Figure 2 In the category prediction branch, the penultimate layer of features in this invention Enter one Convolutional layers produce score heatmaps for each location in the image. .therefore, The 256-dimensional feature vector at each position directly affects the final score prediction. If the features... The accuracy is insufficient; the object of this invention cannot obtain the correct score. Furthermore, the feature map... Each position The 256-dimensional feature vector in Convolution operations are unaffected by features at other locations, so they can independently represent the features of the target at each location. This invention will... Features with the same category are retained and their mean is taken as high-quality features. Because not every location can learn effective features, this invention selectively retains high-quality features. First, this invention applies a max function to the heatmap output by the category prediction branch and the actually labeled heatmap to obtain the maximum category score for each location. and For each category, if the score Above the threshold And in the heatmap of the actual image annotation The score was also higher than the threshold. The present invention addresses these features. The best features in the current iteration are obtained by retaining and averaging them, as shown in the equation:

[0080]

[0081] However, the score distribution for each category is different and changes with the number of iterations. Therefore, this invention needs to select a suitable score threshold for each category when collecting high-quality features. Ultimately, this invention first sets a small threshold and continuously increases the threshold until it reaches a maximum threshold, so as to collect better features with the increase of the number of iterations.

[0082] This invention will use the high-quality features of the current iteration The updated effect is achieved by weighting the data with the already collected high-quality features, as shown in the equation:

[0083]

[0084] in, and They represent the first The second iteration and the first High-quality features in the next iteration This represents the features that meet the conditions in the current iteration of the network. These are weighted weights for updating high-quality features. At the beginning of training, this invention sets a small weight because the features learned by the network are unstable at this stage. As the number of iterations increases, the network learns more accurate features and tends to stabilize, so this invention increases the weight to improve the overall quality of the high-quality features. In the later stages of training, since good features have been collected, this invention reduces the weight to fine-tune the high-quality features. The effect is as follows. Therefore, the weights obtained by the Gaussian distribution function in this invention are as follows:

[0085]

[0086] in, and These are two adjustable parameters of the function, which determine the upper and lower limits of the weights, respectively. and These are also two adjustable parameters of the function, which determine the trend of weight changes as the network iterates. In this invention, they are ultimately set to 0.4, 0.1, 4, and 1500, respectively. This represents the value of the iteration number divided by 10000. Furthermore, since the features learned at the beginning of network training are not representative, this invention begins collecting features at the 10kth iteration of the network.

[0087] Step S3: Predict the target size, category, and location-specific features.

[0088] Using the network weights trained by this invention, after inputting an image, the network will output a category score vector of length c, the height and width of the target, and a difference feature vector of length 256 at each position of the image.

[0089] Step S4: Summing the difference features and the categorical features yields the feature. To achieve the correction of categorical features, and then to implement The convolution operation yields the classification heatmap. .

[0090] In step S4, this invention utilizes high-quality features and categorical features to guide the learning of differential features. Therefore, the output of the differential feature branch in this invention is actually an offset value. Figure 3 , to differentiate features With category features Adding them together yields the corrected features. For targets with flawed prospects, Approaching high-quality features. Furthermore, for targets with incorrect backgrounds, Approaching zero, while other positions Approaching Ultimately, all the features that were predicted incorrectly can be corrected accordingly.

[0091] Step S5: Utilize and The algorithm filters candidate boxes from the prediction results of the category prediction branch to obtain candidate boxes. .

[0092] The method of this invention is based on an anchorless bounding box approach, therefore, a classification score and the height and width of the target bounding box are predicted at each location in the image. Thus, as... Figure 3 This invention filters candidate boxes based on the scores of their category prediction branches. Finally, it uses the top-k algorithm to select the 1500 boxes with the highest scores among all target boxes, and then executes the NMS algorithm to suppress duplicate boxes, obtaining the final result. A candidate box.

[0093] Step S6: Predict the heatmap obtained by using differential features. For candidate boxes The corresponding scores will be updated in the final analysis.

[0094] In step S6, because all candidate boxes obtained by this invention come from the category prediction branch, there are many candidate boxes with inaccurate scores. At this point, this invention needs to fine-tune the scores of these target boxes using the classification scores obtained from the differential feature learning branch, but this invention does not directly... The topk and NMS algorithms are used because the network also has some errors when learning differential features. Ultimately, based on the score range set during differential feature regression, this invention selects to only apply the scores... Fine-tune the target bounding box. For example... Figure 3 The present invention first obtains the difference features Execute a The convolution operation, specifically, this convolution and the last layer of the class prediction branch. Convolutional layers share weights and ultimately output a score heatmap. Where c represents the number of categories. Finally, the scores are updated with weights, as shown in the equation:

[0095]

[0096] in, and Representing the target boxes The scores before and after the update. Indicates according to Category and central position get Score for corresponding position , This indicates that the score updates the weight, and the final weight in this invention is set to 0.4.

[0097] In practical applications, this invention uses metrics such as mAP / AP50 / AP75 / AR on the VisDrones dataset to evaluate its model. VisDrones is a classic aerial image dataset that suffers from severe problems such as low resolution, occlusion, blurring, data imbalance, and multiple views. The training set of this dataset contains 6471 images, the validation set contains 548 images, and the test challenge set contains 1580 images, with image resolutions ranging from... arrive The dataset contains 12 categories: "ignore", "pedestrian", "person", "bicycle", "car", "van", "truck", "tricycle", "awning-tricycle", "bus", "motor", and "others". Ultimately, this invention only uses the first 11 categories, excluding the "ignore" category in the final AP calculation, and evaluating performance only on objects within these 10 categories. Table 1 shows the statistical distribution of target category numbers in the VisDrones dataset. It can be seen that "tricycle", "awning-tricycle", and "bus" have the fewest targets; therefore, these three categories are used as the tail category subset, and the others as the head category subset. After partitioning, the head subset contains 5370 images, and the tail subset contains 1101 images. Finally, this invention selects Hourglass as the backbone network and uses an anchor-free method to predict the category and size of each target, serving as the baseline network for this invention.

[0098] Table 1. Distribution of the number of target categories in the dataset

[0099]

[0100] First, the performance of the baseline network was evaluated. Then, the head-and-tail sampling method and the differential feature learning method were gradually added to the dataset, and their performance was evaluated, as shown in Table 2. Here, "Multi-scale" indicates multi-scale prediction during testing. Furthermore, the resampling method and differential feature learning method of this invention are represented by HTS and DFL in the table. The table shows that after adding the HTS method to the baseline network, this invention achieved performance improvements of 1.72%, 2.72%, 1.87%, and 1.58% in mAP, AP50, AP75, and AR, respectively. Then, the DFL method of this invention further improved the network performance, achieving improvements of 35.21%, 64.22%, 33.97%, and 51.33% in mAP, AP50, AP75, and AR, respectively.

[0101] Table 2 Performance evaluation of the benchmark network and the method of this invention on various metrics.

[0102]

[0103] Furthermore, the present invention further compares the improvement of mAP for each category by the method of the present invention, as shown in Table 3. The results show that the HTS method of the present invention can effectively improve the network's detection performance for each category, with the most significant improvement for tail categories, namely "tcy", "awn", and "bus", with mAP improvements of 3.1%, 3.0%, and 5.32%, respectively. In addition, the DFL method of the present invention is mainly used to solve problems such as low resolution, occlusion, and blurring, and can also improve the network's detection performance. For example, categories that are prone to the above problems, such as "pedestrian", "person", "tricycle", "awning-tricycle", and "motor", show significant improvements in mAP, at 0.85%, 1.76%, 0.76%, 1.59%, and 2.54%, respectively.

[0104] Table 3 Performance evaluation of the baseline network and the method of this invention for each category of mAP index.

[0105]

[0106] The differential features learned by the differential feature branch in this invention are visualized, such as... Figure 4 For the first image, the occluded objects scored low in prediction, but the differential feature learning branch predicted more prominent features at these locations, indicating that these objects lacked more features. Meanwhile, the visualization of the second image reveals the corrected features. This allows for the output of more accurate scores. The invention then uses visualization to compare the differences in detection results between the invention's method and the baseline method. Figure 5 As shown, the baseline network exhibits both false positives and false negatives in its detection results for tail categories such as "tcy" and "awn," while the invented HTS method demonstrates better detection performance. Furthermore, the invented DFL method shows better detection results for low-resolution, blurry, and occluded objects, such as... Figure 6 .

[0107] The above experiments and related results analysis verify the effectiveness of the aerial image target detection method proposed in this invention.

[0108] Example 2:

[0109] Reference Figure 7 A target detection system is provided to implement the aerial image target detection method in the above embodiments. The system includes:

[0110] The dataset partitioning module is used to partition specific datasets in a targeted manner.

[0111] The feature extraction module is used to extract features required for downstream detection tasks using the Hourglass backbone network.

[0112] The target category prediction module is used to predict the target category at each location by utilizing the features learned by the upstream network.

[0113] The target size prediction module is used to predict the height and width of the target at each location using features learned from the upstream network.

[0114] The difference feature prediction module is used to predict the difference features at each location by utilizing the features learned by the upstream network.

[0115] The high-quality feature collection module is used to collect high-quality features for each category obtained from the category prediction module.

[0116] The target bounding box filtering module is used to filter the target bounding boxes from the outputs of the target category prediction module and the target size prediction module.

[0117] The score update module is used to correct the category prediction features using the features output by the difference feature module, and then update the scores of the target boxes that do not meet the criteria for low scores.

[0118] Specifically, the differential feature prediction module uses the following formula to learn differential features:

[0119]

[0120] in, This represents the differential features learned by the network. This indicates the features that need to be corrected in the category prediction branch. To represent the high-quality features of category c, this invention assigns corresponding weights to each positional difference feature. For example, the equation:

[0121]

[0122] Specifically, the feature collection module uses the following formula to obtain high-quality features:

[0123]

[0124]

[0125] in, and They represent the first The second iteration and the first High-quality features in the next iteration This represents the features that meet the conditions in the current iteration of the network. It is the weighted weight for updating high-quality features.

[0126] Specifically, the target box filtering module updates the score using the following formula:

[0127]

[0128]

[0129] in, and Representing the target boxes The scores before and after the update. Indicates according to Category and central position get Score for corresponding position , This indicates that the score updates the weight.

[0130] Example 3:

[0131] The present invention also provides a terminal, such as Figure 8 As shown, for ease of explanation, only the parts related to the embodiments of the present invention are illustrated. The terminal may include any terminal device such as a mobile phone, tablet, computer, or POS machine; a computer is used as an example.

[0132] Figure 5 This diagram shows a partial structural representation of a computer associated with a terminal provided in an embodiment of the present invention. (Reference) Figure 5 A computer includes components such as: memory, input devices, display devices, audio circuitry, wireless fidelity (WIFI) module, processor, and power supply. Those skilled in the art will understand that... Figure 5 The computer architecture shown does not constitute a limitation on the computer and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0133] The following is combined with Figure 5 A detailed introduction to the various components of a computer:

[0134] A memory can be used to store software programs and modules. The processor executes various computer functions and data processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc.; the data storage area can store data created based on computer usage (such as audio data, phone books, etc.). In this invention, the data storage area can store the original image to be used to generate adversarial examples. Furthermore, the memory can include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0135] An input device can be used to receive input numeric or character information, and to generate key signal inputs related to user settings and function control of a computer. The input device in this invention can be used to receive raw image data from which adversarial examples are to be generated.

[0136] Output devices are used to display information input by the user or provided to the user, as well as various computer menus. Output devices may include display panels, optionally configured as liquid crystal displays (LCDs), organic light-emitting diodes (OLEDs), or similar devices.

[0137] Aerial photography equipment can capture images of the environment to be monitored in real time using devices such as drones and output corresponding images. It can also dynamically adjust the shooting angle and altitude to obtain richer and more reliable data.

[0138] Network devices, belonging to data transmission technology, enable computers to help users send and receive emails, browse web pages, and access streaming media in real time, providing users with broadband internet access.

[0139] The processor is the control center of a computer. It connects various parts of the computer via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in memory, and by accessing data stored in memory, thereby providing overall monitoring of the computer. Optionally, the processor may include one or more processing units; preferably, the processor may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into the processor.

[0140] The power source (such as a battery) is preferably connected to the processor logic through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0141] Although not in Figure 8 As shown in the diagram, the computer may also include a camera, Bluetooth module, etc., which will not be elaborated here.

[0142] Specifically, in this embodiment, the processor in the terminal loads the executable files corresponding to the processes of one or more applications into the memory according to the following instructions, and the processor runs the applications stored in the memory to achieve various functions:

[0143] The original image data is divided, and the target detection model is trained.

[0144] The image to be predicted is input into the trained object detection model to obtain the object bounding box category and size at each location, as well as the corresponding differential features;

[0145] The target bounding boxes are filtered to obtain preliminary prediction results for the original image;

[0146] The results from the differential feature branch are used to correct the category features, and then the scores of the target boxes with low scores in the previous prediction result are updated to obtain the final prediction result.

[0147] Specifically, the loss function for training the object detection model is as follows:

[0148]

[0149] Specifically, the formulas for feature correction and score update are as follows:

[0150]

[0151]

[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A target detection method for aerial images based on differential feature learning, characterized in that, This method uses an anchor-free detection algorithm as the baseline network framework, sets the size and number of anchor boxes, and uses hourglass as the backbone network to extract features. It uses three parallel branches to predict target size, category, and differential features. High-quality features are collected in real-time during network training and used to specify the learning of differential features in subsequent training. Finally, differential features are used to correct erroneous features, resulting in a more accurate input feature for the entire network. Specifically, this includes: A specific object detection network is trained using relevant data, and high-quality features extracted by the network are collected in real time during the training process. The specific object detection network uses the Hourglass network as the backbone to extract features, and employs three parallel branches to extract object size, category, and differential features. The differential feature prediction branch is executed only once. Convolution yields a differential feature; Due to category characteristics Different prediction errors will occur at these locations, requiring different loss regressions; firstly, the maximum class score obtained from the class prediction branch needs to be calculated. and the highest category score of the true label The locations are divided and different loss functions are obtained, as shown in the following equation: in, This indicates the classification of locations on the feature map that meet the corresponding conditions. This represents the actual labeled score information. This indicates the score information of the network prediction. and This represents the threshold set when collecting high-quality features. Indicates the true category of the target when the foreground is incorrect. b This indicates the batch size during training. w Indicates the width of the feature map. h The height of the feature map is represented by the following equation: Then, different loss functions are applied to different locations within the differential feature branches, as shown in the equation below: in, Indicates the loss of differential features. n This indicates the number of feature vectors that need to be subjected to differential feature regression. This represents the loss corresponding to foreground error in the difference feature loss. This represents the loss corresponding to background error in the differential feature loss. This represents the difference feature loss at the remaining positions excluding the first two cases in the difference feature loss, where SmoothL1Loss is used as the final loss function in all cases. Indicates the first c High-quality features of class targets, This represents the differential features learned by the network. This indicates the features that need to be corrected in the category prediction branch. This represents the weight corresponding to each position; Furthermore, during training, high-quality features are collected in each iteration; in the category prediction branch, all features that meet the conditions in the current iteration are selected first: in, This represents the mean of the features that meet the conditions in the current iteration of the network. and These represent the maximum class score at each location obtained by applying the max function to the heatmap output by the category prediction branch and the actually labeled heatmap, respectively. This represents the threshold for category collection; then, high-quality features are updated: in, Indicates the first High-quality features from the next iteration. Indicates the first High-quality features from the next iteration. It is a weighted weight for updating high-quality features; Predict the target size, category, and corresponding location differential features; sum the differential features and category features to obtain the final feature. To achieve the correction of categorical features, and then to implement The convolution operation yields a new classification heatmap. ; Furthermore, the head-tail sampling method is used to classify the head and tail categories of targets so that the network can learn the features of both head and tail targets in each iteration.

2. The aerial image target detection method based on differential feature learning according to claim 1, characterized in that, The method also includes: specifically dividing the dataset into two subsets, and randomly selecting the same number of images from both subsets as input; utilizing... and The algorithm's prediction results for the category prediction branch Filter the candidate boxes to obtain the candidate boxes. Heatmaps obtained by predicting branches using differential features For candidate boxes The corresponding scores will be updated in the final analysis.

3. The aerial image target detection method based on differential feature learning according to claim 2, characterized in that, In step S2, three parallel branches are used to extract target size features, category features, and differential features. Specifically, this includes using two parallel detection heads to extract target size features and category features, obtaining the category features. and target size features And input Convolution; additionally, differential features can be obtained directly using a single branch. ; The category prediction branch and the size prediction branch will combine the features from the previous layer. Send in Convolution yields prediction results and ,in, Each position represents a target. c Scores for each category Each position represents the height and width of the target at that position. b This indicates the batch size during training. w Indicates the width of the feature map. h The height of the feature map is represented; then, a score heatmap is obtained based on the ground truth for each target. The focus loss function and the absolute value loss function were used to regress the category and size of the target, respectively.

4. The aerial image target detection method based on differential feature learning according to claim 3, characterized in that, In step S6, the heatmap output by the difference feature branch is used. For candidate boxes Update the scores of those with excessively low scores: in, and Representing the target boxes The scores before and after the update. Indicates based on the target box central position and categories get The score for the corresponding position in the middle. This indicates that the score updates the weight.

5. A system for implementing the aerial image target detection method based on differential feature learning as described in any one of claims 1 to 4, characterized in that, The system includes: The dataset partitioning module is used to partition the dataset in a targeted manner; The feature extraction module is used to extract features required by downstream detection tasks using the Hourglass backbone network; The target category prediction module is used to predict the target category at each location by utilizing the features learned by the upstream network. The target size prediction module is used to predict the height and width of the target at each location by utilizing the features learned by the upstream network. The difference feature prediction module is used to predict the difference features at each location by utilizing the features learned by the upstream network. The high-quality feature collection module is used to collect high-quality features for each category obtained from the category prediction module; The target bounding box filtering module is used to filter the target bounding boxes from the outputs of the target category prediction module and the target size prediction module. The score update module is used to correct the category prediction features using the features output by the difference feature module, and then update the scores of target boxes with low scores.

6. The aerial image target detection system based on differential feature learning according to claim 5, characterized in that, The weight corresponding to each position is represented by the following equation: 。 7. The aerial image target detection system based on differential feature learning according to claim 6, characterized in that, The target box filtering module updates the score using the following formula: in, and Representing the target boxes The scores before and after the update. Indicates based on the target box central position and categories get The score for the corresponding position in the middle. This indicates that the score updates the weight.

Images