Aerial photography target detection method and system based on multi-source comparison self-supervised learning

Abstract

The invention discloses an aerial target detection method and system based on multi-source comparison self-supervised learning, and the method comprises the steps: firstly obtaining an aerial image of an unmanned aerial vehicle, carrying out the standardization processing, constructing a multi-layer feature pyramid, extracting a multi-scale feature map, carrying out the self-supervised pre-training, and generating an optimized feature map; and then, through candidate region generation and background separation, screening a target related anchor box, extracting a feature vector, calculating a category probability, carrying out regression positioning on a bounding box, carrying out small target optimization, carrying out confidence ranking, filtering and optimization on the bounding box, and finally outputting a target detection result. Through a multi-layer feature pyramid structure and small target optimization, the influence of height and pitch angle changes of the unmanned aerial vehicle is considered, multi-layer feature extraction and fusion paths are combined, the detection precision of small targets and dense targets is improved, meanwhile, a multi-source comparison self-supervised learning mechanism is adopted, the background and target distinguishing capability is enhanced, and the detection precision of the small targets and the dense targets is improved. And an optimization strategy and non-maximum suppression are combined, so that the balance between the detection precision and the efficiency is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to target detection means, and belongs to the technical field of target detection, in particular to a method and system for aerial target detection based on multi-source contrastive self-supervised learning. BACKGROUND

[0002] With the increasing application of unmanned aerial vehicle technology in the fields of security monitoring, agricultural management and traffic supervision, the aerial target detection technology of unmanned aerial vehicles directly affects the efficiency and intelligence level of task execution by identifying and locating targets in aerial images. However, due to the characteristics of unmanned aerial vehicle aerial photography such as target size diversity, complex background and obvious light changes, existing target detection methods still have many shortcomings in dealing with these problems.

[0003] Currently, target detection technology is mainly divided into two categories: one is a method based on a single-stage target detection model (YOLO series), which is characterized by fast detection speed, but performs poorly in small target and dense target detection accuracy; the other is a method based on a two-stage target detection model (Faster R-CNN), which has higher detection accuracy but is limited in real-time performance. In addition, with the development of deep learning technology, some research has begun to try to use a multi-scale feature pyramid structure to improve the adaptability of the model to multi-scale targets. However, traditional multi-scale methods have significant defects in the following aspects:

[0004] On the one hand, existing target detection models lack the ability to deal with small targets and dense targets in aerial images, which is mainly due to the fact that targets in aerial scenes are usually small or closely clustered, and the feature extraction module of traditional models cannot fully capture target details. In addition, when detecting targets, the change in flight height and pitch angle of the unmanned aerial vehicle is not considered, resulting in a significant decrease in detection accuracy. In addition, existing methods usually rely on a single resolution feature map for prediction, which cannot effectively integrate multi-scale information, further limiting the adaptability of the model to complex scenes.

[0005] On the other hand, the background in unmanned aerial vehicle aerial photography scenes is complex, and there are high interference elements such as vegetation and buildings that can cause the detection model to misjudge the background as a target. Existing technologies lack targeted means to solve the problem of background interference, and traditional models have limited ability to distinguish between background and target in unsupervised or weakly supervised learning scenarios, resulting in high false positive rates and poor accuracy. SUMMARY

[0006] The purpose of the present application is to overcome the above-mentioned defects and problems in the prior art, and to provide a method and system for aerial target detection based on multi-source contrastive self-supervised learning with high precision.

[0007] To achieve the above object, the technical solution of the present application is: a kind of aerial target detection method based on multi-source contrast self-supervised learning, comprising:

[0008] Obtain multi-source aerial image data of unmanned aerial vehicle, and generate standardized aerial image data;

[0009] Multi-layer feature pyramid structure is constructed, and the multi-layer feature pyramid structure includes a bottom-up feature extraction path and a top-down feature fusion path;Based on the bottom-up feature extraction path, the high-level feature map and the bottom-level feature map of the multi-scale target in the standardized aerial image data are extracted, and based on the top-down feature fusion path, the high-level feature map and the bottom-level feature map are fused to generate multi-scale feature map;

[0010] Based on multi-source contrast self-supervised learning mechanism, the multi-scale feature map is self-supervised pre-trained, and the positive sample pair and the negative sample pair of multi-source data are constructed;The pre-training feature representation is generated by optimizing the feature extraction pre-training weight through contrast learning;

[0011] Based on the pre-training feature representation, the multi-scale feature map is enhanced and fused, and the contrast learning is carried out on the feature maps of different scales to generate optimized feature map;

[0012] Based on the optimized feature map, candidate region is generated, and multi-scale original candidate region anchor frame is generated on each layer of multi-layer feature pyramid;The background separation ability of candidate region is optimized by background separation, and the candidate region anchor frame related to target is screened out to obtain the total set of candidate region anchor frame;

[0013] Based on the region interest alignment operation, the feature vector in each candidate region is extracted, and the target class probability is calculated;The optimized feature map is used to regress and locate the candidate region anchor frame to generate the boundary box with accurate positioning;The target class probability and the boundary box with accurate positioning are taken as the target detection result and output;

[0014] The boundary box with accurate positioning is divided into small target boundary box and large target boundary box, and the small target boundary box is fused with the large target boundary box after accurate positioning to obtain the target detection result considering small target and large target;

[0015] The confidence of each boundary box is defined based on the target detection result considering small target and large target, and the ordered candidate frame set is constructed by sorting;The redundant boundary box in the ordered candidate frame set is filtered, and the position of the retained boundary box is optimized and the class is weighted to obtain the final unmanned aerial vehicle aerial target detection result.

[0016] The multi-source aerial image data of unmanned aerial vehicle is obtained, and the standardized aerial image data is generated, specifically including:

[0017] The Gaussian filtering method is used for denoising the aerial image data collected by the unmanned aerial vehicle, and the pixel value of the aerial image data is smoothed, and the expression is as follows:

[0018] ;

[0019] Wherein: is a Gaussian weight, , is the coordinate offset of the image pixel, is the standard deviation;

[0020] The normalized aerial image data is obtained by normalizing the denoised aerial image data, and the pixel value range of the aerial image data is linearly mapped to the interval [0, 1];

[0021] The normalized aerial image data is cropped, and the size of the cropped aerial image region is , and the center coordinates of the aerial image region are :

[0022] ; ;

[0023] Wherein: , are the original width and height of the aerial image data respectively;

[0024] Based on the cropped aerial image data, the pixel coordinates are corrected by using the known camera intrinsic matrix and distortion coefficient for optical distortion correction, and the expression is as follows:

[0025] ;

[0026] ;

[0027] Wherein: is the corrected pixel coordinate, is the original pixel coordinate, , , , are all distortion coefficients, is the radial distance;

[0028] The optical distortion corrected aerial image data is histogram equalization processed to generate the standardized aerial image data.

[0029] The generated multi-scale feature map specifically includes:

[0030] A bottom-up feature extraction path is constructed to extract features of multi-scale targets in standardized aerial image data, and in the feature extraction process, the flight height and pitch angle of the unmanned aerial vehicle are considered to correct perspective, to obtain high-level feature maps and bottom-level feature maps, which are expressed as follows:

[0031] ;

[0032] ;

[0033] ;

[0034] ;

[0035] wherein: is the feature value of the bottom-level feature map of the layer at the pixel coordinate and the channel , is the pixel value of the standardized aerial image data at the coordinate and the channel , is a perspective transformation homography matrix, is a dynamic scale factor, is a focal length, is a pixel physical size, is a flight height of the unmanned aerial vehicle, is a pitch angle of the unmanned aerial vehicle, is a bias term, is an activation function, is an intrinsic matrix, is a rotation matrix established according to the pitch angle ;

[0036] A top-down feature fusion path is constructed to fuse the high-level feature maps and the bottom-level feature maps to generate a fused feature map, which is expressed as follows:

[0037] ;

[0038] wherein: is the fused feature map, is the feature value of the high-level feature map at the coordinate and the channel , , are both learnable fusion weight parameters, , are respectively the width and the height of the aerial image data after cropping;

[0039] A skip connection is established between the bottom-up feature extraction path and the top-down feature fusion path, the fused feature maps from different resolutions are spliced in the channel dimension to generate multi-scale feature maps and perform normalization processing, and the expression is as follows:

[0040] ;

[0041] Wherein: is a multi-scale feature map, is a channel direction feature splicing operation, is a channel index of the spliced feature map;

[0042] The normalized multi-scale feature map is down-sampled to generate a multi-scale feature map.

[0043] The pre-training feature representation is generated, specifically including:

[0044] Based on the multi-scale feature map, a positive sample pair and a negative sample pair of multi-source data are constructed;

[0045] Wherein: define the positive sample pair as a feature pair derived from the same target area but processed by different data enhancement methods, and define the negative sample pair as a feature pair derived from different target areas or background areas, and the expression is as follows:

[0046] ;

[0047] ;

[0048] Wherein: , is a multi-scale feature map derived from the same target area processed by different enhancement methods , , and , is a multi-scale feature map derived from different target areas or background areas;

[0049] The positive sample pair and the negative sample pair are compared and learned, and based on the contrastive loss function, the pre-training contrastive learning weight of the feature extraction network is optimized, and the expression is as follows:

[0050] ;

[0051] ; ;

[0052] Wherein: is a contrastive loss function, is a set of positive sample pairs, is a set of negative sample pairs, is the number of positive sample pairs, is the total number of positive and negative sample pairs, is the cosine similarity between features, is the temperature coefficient;

[0053] In the contrast learning process, batch normalization is performed on the multi-scale feature maps to obtain normalized multi-scale feature maps, and based on the optimized contrast learning weight, a pre-training feature representation is generated, which is expressed as follows:

[0054] ;

[0055] wherein: is the pre-training feature representation, is the feature mapping weight matrix, is the normalized multi-scale feature map is the bias term.

[0056] The generated optimized feature map specifically includes:

[0057] The multi-scale feature map and the pre-training feature representation are spliced in the channel dimension to obtain a preliminary fusion feature, which is expressed as follows:

[0058] ;

[0059] wherein: is the preliminary fusion feature, is the new channel index after splicing, is the multi-scale feature map, is the pre-training feature representation;

[0060] Based on the preliminary fusion feature, a learnable linear transformation matrix, a bias term and an activation function are constructed to obtain an optimized feature map, which is expressed as follows:

[0061] ;

[0062] wherein: is the optimized feature map, is the output channel index, is the number of channels of the fused feature, is the activation function, is the learnable linear transformation matrix, is the bias term.

[0063] The total set of target-related candidate region anchor boxes specifically includes:

[0064] Based on the optimized feature map, a multi-scale original candidate region anchor box is generated at the pixel coordinates of the optimized feature map on each layer of the multi-layer feature pyramid, which is expressed as follows:

[0065] ;

[0066] wherein: is the original candidate region anchor box generated at the layer, pixel coordinate and anchor box scale index , , is the anchor box center point coordinate mapped to the aerial image data coordinate system, , are the anchor box width and height of the layer corresponding anchor box scale index , respectively;

[0067] For each original candidate region anchor box , a background score is calculated by background separation, and its expression is as follows:

[0068] ;

[0069] wherein: is the background score, is the weight parameter of background separation, is the bias term, is the feature value of the optimized feature map at pixel coordinate and channel , is the number of channels of the optimized feature map;

[0070] Based on the background score and a preset threshold, the original candidate region anchor box is screened to obtain a candidate region anchor box, and candidate region anchor boxes from each layer of the multi-layer feature pyramid are collected to obtain a total set of target-related candidate region anchor boxes. The expression of the screening is as follows:

[0071] ;

[0072] ;

[0073] wherein: is the retained target-related candidate region set, is the background region set.

[0074] The target class probability and the precisely positioned bounding box are output as the target detection result, specifically including:

[0075] Based on the obtained total set of target-related candidate region anchor boxes and the optimized feature map, a feature vector in each candidate region is extracted by a region interest alignment operation, and its expression is as follows:

[0076] ;

[0077] wherein: is a feature vector, is a region interest alignment operation, is an optimized feature map, is a candidate region anchor frame;

[0078] Based on the feature vector and the multi-class cross-entropy loss function, the target class probability is calculated;

[0079] Based on the target feature in the optimized feature map, the candidate region anchor frame is regressed and positioned, the boundary box prediction offset is calculated, and the original candidate region anchor frame is adjusted based on the boundary box prediction offset to generate a precisely positioned boundary box, and the expression is as follows:

[0080] ;

[0081] wherein: is the center coordinate of the adjusted boundary box, is the width and height of the adjusted boundary box, , , , is the boundary box prediction offset;

[0082] The target class probability and the precisely positioned boundary box are output as the target detection result.

[0083] The target detection result considering both small targets and large targets is obtained, specifically including:

[0084] A small target area threshold is set to divide all precisely positioned boundary boxes into small target boundary boxes and large target boundary boxes; if the area of the boundary box is smaller than the small target area threshold, it is divided into a small target boundary box, otherwise, it is divided into a large target boundary box;

[0085] The region interest alignment is performed on the small target boundary box on the bottom feature map to obtain a fixed-size fine-grained feature block, and the attention enhancement is performed through the channel attention mechanism and the spatial attention mechanism in turn to obtain the attention-enhanced small target feature;

[0086] The attention-enhanced small target feature is input into the classification network and the regression sub-network, the class confidence is output through the classification network, and the fine-tuning offset of the boundary box is output through the regression sub-network;

[0087] Based on the fine-tuning offset, the original precisely positioned boundary box is precisely positioned to generate a small target precise boundary box set and filter the redundancy;

[0088] Fuse the set of small target bounding boxes with the set of large target bounding boxes; if the same position appears in both the small target bounding box and the large target bounding box during fusion, the small target bounding box will be output first;

[0089] All fused bounding boxes are sorted by confidence and redundancy is filtered again to output object detection results that take into account both small and large objects.

[0090] Generating the final UAV aerial photography target detection results specifically includes:

[0091] Based on the target detection results that take into account both small and large targets, the confidence of each bounding box is defined, and its expression is as follows:

[0092] ;

[0093] in: is the confidence of the bounding box, is the target category probability, Bounding box for precise positioning and the corresponding real box The intersection and union ratio of

[0094] All bounding boxes are sorted in descending order according to their confidence levels to construct an ordered candidate box set. Each element in the ordered candidate box set is represented as follows:

[0095] ;

[0096] in: is an ordered set of candidate boxes, For the bounding boxes, For the The confidence of the bounding box, is the confidence threshold;

[0097] The redundant bounding boxes in the ordered candidate box set are filtered based on the non-maximum suppression method. The expression is as follows:

[0098] ;

[0099] in: is the set of bounding boxes retained after non-maximum suppression, is the intersection-over-union ratio threshold for non-maximum suppression, Respectively and Bounding boxes;

[0100] The position information and classification results in the retained bounding box set are optimized and weighted to obtain the final drone aerial photography target detection result, which is expressed as follows:

[0101] ;

[0102] wherein: is a position parameter of the th bounding box, is a target class probability of the th bounding box.

[0103] Aerial target detection system based on multi-source contrastive self-supervised learning, the system is applied to the method, the system comprises:

[0104] An aerial image data generation unit is configured to obtain multi-source aerial image data of a UAV and generate standardized aerial image data.

[0105] A multi-scale feature generation unit is configured to construct a multi-layer feature pyramid structure, the multi-layer feature pyramid structure comprising a bottom-up feature extraction path and a top-down feature fusion path; based on the bottom-up feature extraction path, high-level feature maps and low-level feature maps of multi-scale targets in the standardized aerial image data are extracted, and based on the top-down feature fusion path, the high-level feature maps and the low-level feature maps are fused to generate multi-scale feature maps.

[0106] A pre-training feature generation unit is configured to perform self-supervised pre-training on the multi-scale feature maps based on a multi-source contrastive self-supervised learning mechanism, to construct positive sample pairs and negative sample pairs of multi-source data; pre-training feature representations are generated by optimizing feature extraction pre-training weights through contrastive learning.

[0107] An optimized feature map generation unit is configured to enhance and fuse the multi-scale feature maps based on the pre-training feature representations, and perform contrastive learning on feature maps of different scales to generate optimized feature maps.

[0108] A candidate region anchor box acquisition unit is configured to generate candidate regions based on the optimized feature maps, and generate multi-scale original candidate region anchor boxes on each layer of the multi-layer feature pyramid, to optimize the background distinguishing ability of the candidate regions through background separation, filter out candidate region anchor boxes related to targets, and obtain a total set of candidate region anchor boxes.

[0109] A target class probability and bounding box determination unit is configured to extract feature vectors in each candidate region based on a region interest alignment operation, and calculate target class probabilities; the candidate region anchor boxes are regressed and positioned based on the optimized feature maps to generate precisely positioned bounding boxes; and the target class probabilities and the precisely positioned bounding boxes are output as target detection results.

[0110] The small target optimization unit is used for dividing the accurately positioned bounding box into a small target bounding box and a large target bounding box, fusing the small target bounding box after accurate positioning with the large target bounding box, and obtaining a target detection result considering both small targets and large targets.

[0111] The target detection unit is used for defining the confidence of each bounding box based on the target detection result considering both small targets and large targets, and sorting and constructing an ordered candidate box set; redundant bounding boxes in the ordered candidate box set are filtered, and the retained bounding boxes are subjected to position optimization and class weighting, so as to obtain a final unmanned aerial vehicle aerial target detection result.

[0112] Compared with the prior art, the present application has the following advantages:

[0113] In the aerial target detection method and system based on multi-source contrast self-supervised learning, the unmanned aerial vehicle aerial image is first acquired and standardized, then a multi-layer feature pyramid is constructed, multi-scale feature maps are extracted and subjected to self-supervised pre-training, optimized feature maps are generated, target-related anchor boxes are screened by candidate region generation and background separation, feature vectors are extracted and class probabilities are calculated, bounding boxes are regressed and positioned, and the bounding box confidence is sorted, filtered and optimized for small target optimization, and finally the target detection result is output; in the application, the multi-layer feature pyramid structure and small target optimization are used, the influence of the height and pitch angle of the unmanned aerial vehicle is considered, the bottom-up and top-down feature extraction and fusion paths are combined, the detection accuracy and recall rate of small targets and dense targets are improved, the multi-source contrast self-supervised learning mechanism is used to enhance the background and target distinguishing ability, and the optimization strategy and non-maximum suppression are combined to realize high-precision target detection, balance the detection accuracy and efficiency. BRIEF DESCRIPTION OF DRAWINGS

[0114] Figure 1 It is the method flowchart of the present application.

[0115] Figure 2 It is the system structure diagram of the present application.

[0116] Figure 3 It is the equipment structure diagram of the present application.

[0117] In the figure: aerial image data generation unit 1, multi-scale feature generation unit 2, pre-training feature generation unit 3, optimized feature map generation unit 4, candidate region anchor box acquisition unit 5, target class probability and bounding box determination unit 6, small target optimization unit 7, target detection unit 8, processor 9, memory 10, computer program code 101. DETAILED DESCRIPTION

[0118] The present application is further described in detail below in combination with the accompanying drawings and specific embodiments.

[0119] Embodiment 1

[0120] Referring to Figure 1 A multi-source contrast-based self-supervised learning aerial target detection method, comprising:

[0121] Obtaining multi-source aerial image data of a UAV and generating standardized aerial image data;

[0122] Further, obtaining multi-source aerial image data of a UAV and generating standardized aerial image data, specifically comprising:

[0123] Based on the Gaussian filtering method, the aerial image data collected by the UAV is denoised, and the pixel value of the aerial image data is smoothed, and the expression is as follows:

[0124] ;

[0125] Wherein: is the Gaussian weight; , is the coordinate offset of the image pixel; is the standard deviation, used to control the filtering strength;

[0126] The denoised aerial image data is normalized, and the pixel value range of the aerial image data is linearly mapped to the [0, 1] interval;

[0127] The normalized aerial image data is cropped, and the size of the cropped aerial image region is , and the center coordinates of the aerial image region are :

[0128] ; ;

[0129] Wherein: , are the original width and height of the aerial image data respectively;

[0130] Based on the cropped aerial image data, the pixel coordinate correction is performed optical distortion correction using the known camera intrinsic matrix and distortion coefficient, and the expression is as follows:

[0131] ;

[0132] ;

[0133] Wherein: is the corrected pixel coordinate, is the original pixel coordinate, , , , are distortion coefficients, is a radial distance;

[0134] The aerial image data corrected by the optical distortion is subjected to histogram equalization processing to suppress overexposed and underexposed areas, and finally standardized aerial image data is generated.

[0135] A multi-layer feature pyramid structure is constructed, which includes a bottom-up feature extraction path and a top-down feature fusion path; based on the bottom-up feature extraction path, high-level feature maps and bottom-level feature maps of multi-scale targets in the standardized aerial image data are extracted, and based on the top-down feature fusion path, the high-level feature maps and the bottom-level feature maps are fused to generate multi-scale feature maps;

[0136] Further, the generation of multi-scale feature maps specifically includes:

[0137] In order to make the multi-scale feature extraction more in line with the actual situation of the unmanned aerial vehicle aerial scene, the bottom-level feature maps not only contain the spatial detail information of the image itself, but also need to record the scale and perspective changes caused by the external parameters related to the unmanned aerial vehicle flight, the flight height, the pitch angle, etc., therefore, in the feature extraction process, the flight height and the pitch angle are added to the perspective transformation modeling of the image by the following formula:

[0138] ;

[0139] wherein: is an intrinsic matrix, is a rotation matrix established according to the pitch angle When the unmanned aerial vehicle height is higher or the pitch angle is larger, the target in the image will show more significant scale changes and perspective distortion.

[0140] The scheme adds the unmanned aerial vehicle parameters, i.e. the height and the pitch angle , which is a "dynamic scale self-adaptation", i.e. the target size changes caused by the height and the perspective are normalized in the feature extraction stage, because in the target detection process, the influence of the height and the pitch angle on the scale accounts for more than 90%, so the yaw angle and other secondary parameters can be ignored, therefore only the height and the pitch angle are retained as dynamic scale factors to avoid redundancy.

[0141] The bottom-up feature extraction path is constructed to extract features of multi-scale targets in the standardized aerial image data, and in the feature extraction process, the flight height and the pitch angle of the unmanned aerial vehicle are considered for perspective correction to obtain high-level feature maps and bottom-level feature maps, whose expressions are as follows:

[0142] ;

[0143] ;

[0144] ;

[0145] wherein: is the bottom layer feature map at pixel coordinate and channel , is the pixel value of the normalized aerial image data at coordinate and channel , is the perspective transformation homography matrix, is the dynamic scale factor, is the focal length, is the pixel physical size, is the flight height of the UAV, is the pitch angle of the UAV, is the bias term, is the activation function;

[0146] is a 3x3 perspective transformation homography matrix, which is obtained by multiplying the camera intrinsic matrix and the rotation and translation matrix considering the flight height and the pitch angle , and is used for perspective correction of the original pixel coordinates, so that the target at different heights and angles can be presented in the image with approximately consistent scale after mapping by , which is convenient for subsequent feature extraction.

[0147] The high layer feature map sequentially inputs the normalized aerial image into several convolution modules, each of which is composed of convolution, normalization and activation; at the end of some modules, the feature map is down-sampled by convolution with a stride of two or maximum pooling; with the increase of network depth, the resolution of the feature map gradually decreases, the number of channels gradually increases, and the semantic information continuously enhances, and finally the low resolution and high semantic feature map obtained at the deepest layer (after all convolution and down-sampling) is the high layer feature map.

[0148] A top-down feature fusion path is constructed to fuse the high layer feature map and the bottom layer feature map to generate a fusion feature map, and its expression is as follows:

[0149] ;

[0150] wherein: is the fusion feature map, is the feature value of the high layer feature map at coordinate and channel ,​ 、 are learnable fusion weight parameters, 、 respectively are the width and height of the aerial image data after cropping;

[0151] A skip connection is established between the bottom-up feature extraction path and the top-down feature fusion path, the fused feature maps from different resolutions are spliced in the channel dimension to generate multi-scale feature maps and normalized, and the expression is as follows:

[0152] ;

[0153] wherein: is a multi-scale feature map; is a channel direction feature splicing operation, is the channel index of the spliced feature map; by concatenating different levels of fused feature maps, the multi-scale feature map integrates target features of each level in the unmanned aerial vehicle aerial scene, and optimizes the recognition ability of multi-scale targets;

[0154] The normalized multi-scale feature map is down-sampled to finally generate a multi-scale feature map.

[0155] Based on the multi-source contrastive self-supervised learning mechanism, the multi-scale feature map is pre-trained in a self-supervised manner, and positive sample pairs and negative sample pairs of multi-source data are constructed; the feature extraction pre-training weight is optimized through contrastive learning to generate a pre-training feature representation;

[0156] Further, the pre-training feature representation is generated, specifically including:

[0157] Based on the multi-scale feature map, positive sample pairs and negative sample pairs of multi-source data are constructed;

[0158] wherein: the positive sample pair is defined as a feature pair derived from the same target region but processed by different data enhancement methods, and the negative sample pair is defined as a feature pair derived from different target regions or background regions, and the expression is as follows:

[0159] ;

[0160] ;

[0161] wherein: 、 are multi-scale feature maps derived from the same target region processed by different enhancement methods 、 , 、 Multi-scale feature maps from different target regions or background regions;

[0162] Further, the steps of obtaining positive and negative sample pairs are as follows:

[0163] Suppose there are multi-source aerial image data, denoted as denotes the index set of data sources (such as different flight altitudes, different shooting angles); denoted as denotes the index set of target regions or background regions of interest; the target regions or background regions of interest are distinguished from artificial annotation GT according to loU threshold 0.5 / 0.1.

[0164] The multi-scale feature map is output as a candidate box by a candidate box generator; the candidate box generator preferably adopts YOLOv5+NMS with a threshold of 0.3;

[0165] The candidate box is compared with the IoU threshold of the target region and the background region of interest to compare the coincidence degree; if , the candidate box is marked as a target region of interest, i.e. ROI; if , the candidate box is marked as a background region;

[0166] The set of all candidate boxes is output as follows:

[0167] ;

[0168] Wherein: is the candidate box set of all target regions of interest, is the set of all candidate boxes, is the candidate box set of all background regions, is an empty set;

[0169] The region feature of each candidate box is extracted, and the expression is as follows:

[0170] ;

[0171] Wherein: is the global average pooling, denotes the cropping to the candidate box , is the feature mapping of the layer;

[0172] The following data augmentation operators are applied to the extracted region features to maintain geometric consistency:

[0173] ;

[0174] Wherein: is random cropping and scaling, For color dithering, is Gaussian blur, To flip horizontally, is random noise;

[0175] When two feature maps originate from the same real target area (not background) but may come from different data sources and are processed by different enhancement methods, they are considered to be a positive sample pair because they both correspond to the "same target" but with different shooting sources or data enhancement methods. The expression is as follows:

[0176] ;

[0177] in: is the candidate box, is the background area, 、 Respectively represent the candidate boxes Apply enhancement operators separately or The feature map obtained later;

[0178] When two feature maps come from different target areas or at least one of them is a background area, they are considered as negative sample pairs, and their expression is as follows:

[0179] ;

[0180] in: or or ; For the feature maps, For the feature maps;

[0181] The set of all positive sample pairs is denoted as , the set of all negative sample pairs is recorded as , and their expressions are as follows:

[0182] ;

[0183] ;

[0184] In the positive sample set The same real target All corresponding ROI boxes are target area sets , which is expressed as follows:

[0185] ;

[0186] in: Represents a candidate box The true label of the target ;

[0187] For any , optionally two enhancement operators , the positive sample pair can be obtained as follows:

[0188] ;

[0189] The negative sample pair set satisfies any of the following conditions, that is, the negative sample pair:

[0190] From different targets or at least one of them is a background box ;

[0191] Then the negative sample pair is as follows:

[0192] ;

[0193] After obtaining the positive and negative sample pairs, an index set is output for the contrast loss function to call; the index set is as follows:

[0194] ;

[0195] The positive and negative sample pairs are subjected to contrast learning, and based on the contrast loss function, the pre-training contrast learning weight of the feature extraction network is optimized, and the model's ability to distinguish aerial image targets and backgrounds is optimized, which is expressed as follows:

[0196] ;

[0197] ; ;

[0198] Wherein: is the contrast loss function, is the set of positive sample pairs, is the set of negative sample pairs, is the number of positive sample pairs, is the total number of positive and negative sample pairs, is the cosine similarity between features, is the temperature coefficient;

[0199] In the contrast learning process, batch normalization is performed on the multi-scale feature map to obtain the normalized multi-scale feature map, and based on the optimized contrast learning weight, the pre-training feature representation is generated, which is expressed as follows:

[0200] ;

[0201] Wherein: The pre-training feature representation is used for pre-training the feature representation, The feature mapping weight matrix is used for feature mapping, The normalized multi-scale feature map is used for normalization, The bias term is used for bias.

[0202] The multi-scale feature map is enhanced and fused based on the pre-training feature representation, and contrastive learning is performed on the feature maps of different scales to generate an optimized feature map;

[0203] Further, the generation of the optimized feature map specifically includes:

[0204] The multi-scale feature map and the pre-training feature representation are spliced in the channel dimension to obtain a preliminary fusion feature, and the expression is as follows:

[0205] ;

[0206] Wherein: The preliminary fusion feature is, The new channel index after splicing is, The multi-scale feature map is, The pre-training feature representation is;

[0207] A learnable linear transformation matrix, a bias term and an activation function are constructed based on the preliminary fusion feature to obtain an optimized feature map, and the expression is as follows:

[0208] ;

[0209] Wherein: The optimized feature map is, The output channel index is, The number of channels of the fused feature is, The activation function is, The learnable linear transformation matrix is, The bias term is.

[0210] Based on the optimized feature map, a candidate region is generated, and a multi-scale original candidate region anchor box is generated on each layer of the multi-layer feature pyramid. The background separation ability of the candidate region is optimized to filter out the candidate region anchor box related to the target, and a total set of candidate region anchor boxes is obtained;

[0211] Further, the total set of candidate region anchor boxes specifically includes:

[0212] Based on the optimized feature map, a multi-scale original candidate region anchor box is generated at the pixel coordinates of the optimized feature map on each layer of the multi-layer feature pyramid, and the expression is as follows:

[0213] ;

[0214] Wherein: To generate the original candidate region anchor frame in the feature pyramid layer, pixel coordinates and anchor frame scale index generated; 、 To map the anchor frame center point coordinates to the aerial image data coordinate system; 、 The anchor frame width and height of the corresponding anchor frame scale index of the first layer, respectively, generate multi-scale anchor frames in combination with the characteristics of the diverse target size in the aerial image of the unmanned aerial vehicle;

[0215] For each original candidate region anchor frame Calculate the background score by background separation, and the background score represents the background probability of the anchor frame corresponding region in the aerial scene of the unmanned aerial vehicle, and its expression is as follows:

[0216] ;

[0217] Where: is the background score, is the weight parameter of the background separation, is the bias term, is the feature value of the optimized feature map in the pixel coordinates and channel , is the number of channels of the optimized feature map;

[0218] Based on the background score and the preset threshold, the original candidate region anchor frame is screened to obtain the candidate region anchor frame, and the candidate region anchor frame sets from each layer of the multi-layer feature pyramid are summarized to obtain the total set of target-related candidate region anchor frames. The expression of the screening is as follows:

[0219] ;

[0220] ;

[0221] Where: is the retained target-related candidate region set, is the background region set. By screening, irrelevant background regions are filtered out and potential target regions are retained in the aerial image of the unmanned aerial vehicle.

[0222] Based on the region interest alignment operation, the feature vector in each candidate region is extracted, and the target class probability is calculated; based on the optimized feature map, the candidate region anchor frame is regressed and positioned to generate the precisely positioned bounding box; the target class probability and the precisely positioned bounding box are output as the target detection result;

[0223] Further, the output target category probability and the precisely positioned bounding box are specifically comprised of:

[0224] Based on the obtained total set of target-related candidate region anchor boxes and the optimized feature map, a feature vector in each candidate region is extracted through a region interest alignment operation, and the expression is as follows:

[0225] ;

[0226] Wherein: is the feature vector; is the region interest alignment operation, which can be realized by constructing a region interest alignment module of a two-stage target detection model, and is used to extract the feature vector of the candidate region anchor box from the optimized feature map; is the optimized feature map; is the candidate region anchor box;

[0227] Based on the feature vector and the multi-class cross-entropy loss function, the target category probability is calculated;

[0228] The expression of the multi-class cross-entropy loss function is as follows:

[0229] ;

[0230] Wherein: is the feature vector of the candidate region, , are the classification weight and bias of the first class respectively, is the one-hot true value of the first class, is the total number of classes;

[0231] Based on the target features in the optimized feature map, the candidate region anchor box is regressed and positioned, the bounding box prediction offset is calculated, and the original candidate region anchor box is adjusted based on the bounding box prediction offset to generate a precisely positioned bounding box, and the expression is as follows:

[0232] ;

[0233] Wherein: is the center coordinate of the adjusted bounding box, is the width and height of the adjusted bounding box, , , , is the bounding box prediction offset;

[0234] Specifically, the feature vector corresponding to the candidate region is extracted from the optimized feature map and input into the classification / regression subnetwork. For boundary box regression, the optimized feature map obtained by fusing the pre-trained feature representation and the multi-scale feature map is pooled (Pool), and then linearly mapped to the regression offset, which is expressed as follows:

[0235] ;

[0236] wherein: represents the transformation of the feature vector aligned with the region of interest into a fixed dimension, and are learnable regression weights and biases;

[0237] The target class probability and the precisely positioned boundary box are output as the target detection result.

[0238] The precisely positioned boundary box is divided into a small target boundary box and a large target boundary box. After fine positioning of the small target boundary box, it is fused with the large target boundary box to obtain a target detection result that takes into account both small targets and large targets.

[0239] Further, the target detection result that takes into account both small targets and large targets specifically includes:

[0240] By the area and size of each precisely positioned boundary box under the real ground scale, combined with the size transformation brought by the flight height and pitch angle of the unmanned aerial vehicle, and by setting a small target area threshold, all precisely positioned boundary boxes are divided into small target boundary boxes and large target boundary boxes. If the area of the boundary box is less than the small target area threshold, it is divided into a small target boundary box, otherwise it is divided into a large target boundary box.

[0241] The region of interest alignment is performed on the small target boundary box on the bottom feature map to obtain a fixed-size fine-grained feature block, and then the channel attention mechanism and the spatial attention mechanism are sequentially passed through to enhance the attention, obtaining attention-enhanced small target features. Channel attention can focus on the most discriminative feature channels, while spatial attention can highlight the key regions where small targets are located.

[0242] The attention-enhanced small target features are input into the classification network and the regression subnetwork. The classification network outputs the class confidence, and the regression subnetwork outputs the fine-tuning offset of the boundary box.

[0243] Based on the fine-tuning offset, the original precisely positioned boundary box is fine-positioned to generate a small target precise boundary box set and filter out redundancies. The preferred method for filtering out redundancies is soft non-maximum suppression (Soft-NMS).

[0244] The small target fine bounding box set is fused with the set of large target bounding boxes; if the same position appears in the small target bounding box and the large target bounding box at the same time during the fusion, the small target fine bounding box is taken as the priority output;

[0245] All the bounding boxes after the fusion are sorted according to the confidence and filtered again, and the target detection result considering both the small target and the large target is output.

[0246] The confidence of each bounding box is defined based on the target detection result, and an ordered candidate box set is constructed by sorting; the redundant bounding boxes in the ordered candidate box set are filtered, and the remaining bounding boxes are position optimized and class weighted to obtain the final unmanned aerial vehicle aerial target detection result.

[0247] Further, the final unmanned aerial vehicle aerial target detection result is generated, specifically including:

[0248] Based on the target category probability and the precisely positioned bounding box, the confidence of each bounding box is defined, and the expression is as follows:

[0249] ;

[0250] Wherein: is the confidence of the bounding box; is the target category probability; is the precisely positioned bounding box and the intersection over union of the corresponding real box , used to measure the overlap degree of the predicted box and the real box;

[0251] All the bounding boxes are sorted in descending order according to the confidence to construct an ordered candidate box set; each element in the ordered candidate box set is represented as follows:

[0252] ;

[0253] Wherein: is the ordered candidate box set; is the th bounding box; is the confidence of the th bounding box; is the confidence threshold, only the bounding boxes with a confidence higher than are retained;

[0254] The redundant bounding boxes in the ordered candidate box set are filtered based on the non-maximum suppression method, and the expression is as follows:

[0255] ;

[0256] Wherein: is the bounding box set retained after the non-maximum suppression; is a non-maximum suppression intersection over union threshold, ensuring that only high-confidence bounding boxes with small overlap are retained; are the first and the second bounding boxes, respectively;

[0257] The position information and classification results in the retained bounding box set are subjected to position optimization and class weighting to obtain a final unmanned aerial vehicle aerial target detection result, which is expressed as follows:

[0258]

[0259] wherein: is a position parameter of the first bounding box, is a target class probability of the second bounding box.

[0260] In this embodiment, in order to verify the feasibility and effectiveness of the method of the present application in actual scenarios, the embodiment selects a complex unmanned aerial vehicle aerial task scenario for experiment, specifically including the following:

[0261] A certain mountainous forest protection area is selected as the test area, the area has complex terrain, high vegetation coverage, and a large number of occlusions and background interference, the target of the study is to monitor illegal activities (illegal logging vehicles, illegal construction material stacking) in the area in real time, the unmanned aerial vehicle completes multiple aerial tasks during 8:00-10:00 in the morning and 14:00-16:00 in the afternoon under sunny and cloudy conditions, and a total of 5000 aerial images with a resolution of 3840x2160 are collected, the targets in the images are mainly logging vehicles, stacked wood and other small mechanical equipment, the target size is between 15x15 pixels and 100x100 pixels, the densely distributed area accounts for more than 40% of the total target, and the background also contains a large number of plant, rock and building interference. In this scenario, the method of the present application is applied, and the specific steps are as follows:

[0262] First, the collected aerial images are subjected to data preprocessing, including denoising, normalization, cropping and optical distortion correction, to eliminate the influence of light changes, noise and lens distortion on detection accuracy; after preprocessing, a standardized aerial image dataset is generated, a total of 5000 images.

[0263] Then a multi-layer feature pyramid structure is constructed to extract multi-scale target features from the standardized images, low-level spatial detail features are extracted through a bottom-up path and combined with a top-down path to fuse multi-scale features, and finally a multi-scale feature map suitable for small target and dense target detection is generated, and the model is initialized using pre-trained weights during the feature map construction process, greatly reducing the dependence on labeled data.​

[0264] On this basis, a multi-source contrastive self-supervised learning mechanism is used to optimize and train the multi-scale feature maps. By constructing positive sample pairs and negative sample pairs, the model significantly improves the ability to distinguish between the background and the target during the training process. We use enhanced images of the same target but different lightings as positive sample pairs, and use the target and background regions as negative sample pairs, respectively. Through optimization of the contrastive loss function, high-quality feature representations are finally generated, which enhances the generalization ability of the model in complex scenes.

[0265] In the candidate region generation stage, a multi-scale anchor box generation method based on the feature pyramid is used to screen the target-related regions, and a background separation module is used to filter the complex background regions of vegetation and rocks. After screening, about 150 candidate regions are retained on average for each image, which is 30% less than the irrelevant regions of the traditional method.

[0266] Subsequently, the model enters the second stage of two-stage target detection, in which the classification subnetwork is used to predict the target class of the candidate regions, and the regression subnetwork is used to accurately adjust the detection frame. The detected logging vehicle frame is optimized from the original 30x40 pixels to 35x42 pixels, and the overlap degree of the detection frame is significantly improved. Finally, all the detection frames and classification results are processed by non-maximum suppression to output the final target detection results in each image.

[0267] To verify the effectiveness of the present application, a comparative experiment was conducted with the traditional YOLOv5 model. The experiment used 3000 aerial images as the training set and the remaining 2000 images as the test set. The specific comparison data are shown in the following table.

[0268] ;

[0269] The experimental results show that the detection accuracy of small targets in the unmanned aerial image is improved by 13.8% compared with the traditional method, and the recall rate of dense targets is improved by 15.5%.

[0270] In addition, in order to further analyze the effect in complex background scenes, we randomly selected 200 test images with dense target distribution and calculated the number of false positive frames and the number of correctly detected targets. The specific comparison data are shown in the following table.

[0271] ;

[0272] From the data in the above two tables, it can be seen that the present application is significantly better than the traditional method in small target and dense target detection, with a false positive rate reduction of 66% and a correct detection target number increase of 28%. In addition, the present application method reduces the amount of labeled data while maintaining high detection performance, which shows the significant advantage of the self-supervised learning mechanism.

[0273] It can be seen from the above embodiments that the unmanned aerial vehicle aerial scene in the complex background and the dense distribution of multiple targets exhibits excellent detection performance. The experimental results prove that the application not only improves the accuracy and recall rate of target detection, but also performs well in reducing the labeling cost and coping with complex background interference, and has high practical application value.

[0274] Embodiment 2

[0275] Referring to Figure 2 Aerial target detection system based on multi-source contrastive self-supervised learning, the system is applied to the method of embodiment 1, the system comprises:

[0276] An aerial image data generation unit 1 is configured to obtain unmanned aerial vehicle multi-source aerial image data and generate standardized aerial image data.

[0277] A multi-scale feature generation unit 2 is configured to construct a multi-layer feature pyramid structure, the multi-layer feature pyramid structure comprises a bottom-up feature extraction path and a top-down feature fusion path; based on the bottom-up feature extraction path, high-level feature maps and low-level feature maps of multi-scale targets in the standardized aerial image data are extracted, and based on the top-down feature fusion path, the high-level feature maps and the low-level feature maps are fused to generate multi-scale feature maps.

[0278] A pre-training feature generation unit 3 is configured to perform self-supervised pre-training on the multi-scale feature maps based on a multi-source contrastive self-supervised learning mechanism, and construct positive sample pairs and negative sample pairs of multi-source data; the pre-training feature representation is generated by optimizing the feature extraction pre-training weight through contrastive learning.

[0279] An optimized feature map generation unit 4 is configured to enhance and fuse the multi-scale feature maps based on the pre-training feature representation, and perform contrastive learning on feature maps of different scales and sources based on a multi-source contrast mechanism to generate optimized feature maps.

[0280] A candidate region anchor frame acquisition unit 5 is configured to generate candidate regions based on the optimized feature maps, and generate multi-scale original candidate region anchor frames on each layer of the multi-layer feature pyramid, optimize the background distinguishing ability of the candidate regions through background separation, filter out candidate region anchor frames related to the target, and obtain a total set of target-related candidate region anchor frames.

[0281] A target class probability and bounding box determination unit 6 is configured to extract feature vectors in each candidate region based on a region interest alignment operation, and calculate a target class probability; the optimized feature maps are used to regress and locate the candidate region anchor frames to generate precisely located bounding boxes; and the target class probability and the precisely located bounding boxes are output as target detection results.

[0282] The small target optimization unit 7 is configured to divide the accurately positioned bounding box into a small target bounding box and a large target bounding box, and fuse the small target bounding box with the large target bounding box after accurate positioning of the small target bounding box, to obtain a target detection result taking into account both small targets and large targets.

[0283] The target detection unit 8 is configured to define a confidence of each bounding box based on the target detection result taking into account both small targets and large targets, and sort and construct an ordered candidate box set; filter redundant bounding boxes in the ordered candidate box set, and perform position optimization and class weighting on the retained bounding boxes, to obtain a final unmanned aerial vehicle aerial target detection result.

[0284] Further, the steps of implementing the functions of the above units are described in the corresponding description of the method in Embodiment 1.

[0285] Embodiment 3

[0286] Referring to Figure 3 An aerial target detection device based on multi-source contrastive self-supervised learning, the device comprising a processor 9 and a memory 10;

[0287] The memory 10 is configured to store computer program code 101 and transmit the computer program code 101 to the processor 9;

[0288] The processor 9 is configured to execute the aerial target detection method based on multi-source contrastive self-supervised learning according to the instructions in the computer program code 101.

[0289] In this embodiment, a computer readable storage medium is also included, and the computer readable storage medium stores computer executable instructions, when the computer executable instructions are executed on a computer, the method of aerial target detection based on multi-source contrastive self-supervised learning in Embodiment 1 is implemented.

[0290] Generally, the computer instructions used to implement the method of the present application can be carried by any combination of one or more computer readable storage media. Non-transitory computer readable storage media can include any computer readable medium except a signal propagating transitorily.

[0291] The computer readable storage medium may, for example, be tangible and non-transitory. A non-exhaustive list of examples of a computer readable storage medium include: electrical connection-based media, portable computer diskette, hard disk, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or Flash memory), optical fiber, portable compact disc read only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium can be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0292] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages, and specifically Python language and platform framework based on TensorFlow, PyTorch, etc. suitable for neural network computing. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0293] The above apparatus and non-transitory computer readable storage medium can refer to the specific description of the aerial target detection method based on multi-source contrast self-supervised learning and its beneficial effects, which will not be described here.

[0294] Although the embodiments of the present application have been shown and described above, it should be understood by those skilled in the art that the above embodiments are exemplary and cannot be interpreted as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A method for aerial target detection based on multi-source contrastive self-supervised learning, characterized in that: include: Acquire multi-source aerial image data from drones and generate standardized aerial image data; Constructing a multi-layer feature pyramid structure, wherein the multi-layer feature pyramid structure includes a bottom-up feature extraction path and a top-down feature fusion path; Based on the bottom-up feature extraction path, high-level feature maps and low-level feature maps of multi-scale targets in the standardized aerial image data are extracted, and based on the top-down feature fusion path, high-level feature maps and low-level feature maps are fused to generate multi-scale feature maps; Based on the multi-source contrastive self-supervised learning mechanism, self-supervised pre-training of multi-scale feature maps is performed to construct positive and negative sample pairs of multi-source data; pre-trained feature representations are generated by optimizing feature extraction pre-training weights through contrastive learning; Enhance and fuse multi-scale feature maps based on pre-trained feature representation, and perform comparative learning on feature maps of different scales to generate optimized feature maps; Generate candidate regions based on the optimized feature map, and generate multi-scale original candidate region anchor frames on each layer of the multi-layer feature pyramid. Optimize the background discrimination ability of the candidate regions through background separation, screen out target-related candidate region anchor frames, and obtain the total set of candidate region anchor frames. Based on the region of interest alignment operation, the feature vector in each candidate region is extracted and the target category probability is calculated; based on the optimized feature map, the anchor box of the candidate region is regressed and positioned to generate a precisely positioned bounding box; the target category probability and the precisely positioned bounding box are output as the target detection result; The precisely located bounding box is divided into a small target bounding box and a large target bounding box. After precise positioning of the small target bounding box, it is fused with the large target bounding box to obtain an object detection result that takes into account both small and large targets. Based on the target detection results that take into account both small and large targets, the confidence of each bounding box is defined, and the candidate boxes are sorted to form an ordered set. Redundant bounding boxes in the ordered candidate box set are filtered out, and the retained bounding boxes are optimized in position and weighted in category to obtain the final UAV aerial photography target detection results.

2. The method for aerial target detection based on multi-source contrastive self-supervised learning according to claim 1, characterized in that: The method of obtaining multi-source aerial image data from a UAV and generating standardized aerial image data specifically includes: The aerial image data collected by the UAV is denoised based on the Gaussian filter method, and the pixel values ​​of the aerial image data are smoothed. The expression is as follows: ； in: is the Gaussian weight, 、 is the coordinate offset of the image pixel, is the standard deviation; Normalize the denoised aerial image data and linearly map the pixel value range of the aerial image data to the interval [0,1]; The normalized aerial image data is cropped, and the size of the cropped aerial image area is , the center coordinates of the aerial image area for: ； ； in: 、 are the original width and height of the aerial image data respectively; Based on the cropped aerial image data, optical distortion correction is performed on the pixel coordinates using the known camera intrinsic parameter matrix and distortion coefficients. The expression is as follows: ； ； in: is the corrected pixel coordinate, is the original pixel coordinate, 、 、 、 are the distortion coefficients, is the radial distance; The aerial image data after optical distortion correction is subjected to histogram equalization processing to finally generate standardized aerial image data.

3. The method for aerial target detection based on multi-source contrastive self-supervised learning according to claim 1, characterized in that: Generating a multi-scale feature map specifically includes: A bottom-up feature extraction path is constructed to extract features of multi-scale targets in standardized aerial image data. During the feature extraction process, perspective correction is performed considering the flight altitude and pitch angle of the drone to obtain high-level feature maps and low-level feature maps. The expressions are as follows: ； ； ； ； in: For the The bottom layer feature map is in pixel coordinates and channels The eigenvalues ​​on To standardize the aerial image data in coordinates and channels The pixel value on is the perspective transformation homography matrix, is the dynamic scale factor, is the focal length, is the physical size of pixels, is the flight altitude of the UAV, is the pitch angle of the drone, is the bias term, is the activation function, is the internal parameter matrix, According to the pitch angle The created rotation matrix; Construct a top-down feature fusion path, fuse the high-level feature map with the low-level feature map, and generate a fused feature map, which is expressed as follows: ； in: is the fusion feature map, For high-level feature maps in coordinates and channels The eigenvalues ​​on 、 are all learnable fusion weight parameters, 、 are the width and height of the cropped aerial image data respectively; A skip connection is established between the bottom-up feature extraction path and the top-down feature fusion path. The fused feature maps from different resolutions are concatenated in the channel dimension to generate a multi-scale feature map and normalize it. The expression is as follows: ； in: is a multi-scale feature map, is the feature concatenation operation in the channel direction, is the channel index of the concatenated feature map; The normalized multi-scale feature map is downsampled to finally generate a multi-scale feature map.

4. The method for aerial target detection based on multi-source contrastive self-supervised learning according to claim 1, characterized in that: Generating the pre-trained feature representation specifically includes: Construct positive and negative sample pairs from multi-source data based on multi-scale feature maps; Among them: the positive sample pair is defined as a feature pair originating from the same target area but processed by different data enhancement methods, and the negative sample pair is defined as a feature pair originating from different target areas or background areas. The expressions are as follows: ； ； in: 、 is a multi-scale feature map derived from the same target area Through different enhancement methods 、 The processed multi-scale feature map, 、 It is a multi-scale feature map derived from different target areas or background areas; Perform contrastive learning on positive and negative sample pairs, and optimize the pre-trained contrastive learning weights of the feature extraction network based on the contrastive loss function, which is expressed as follows: ； ； ； in: is the contrast loss function, is the set of positive sample pairs, is the set of negative sample pairs, is the number of positive sample pairs, is the total number of positive and negative sample pairs, is the cosine similarity between features, is the temperature coefficient; In the contrastive learning process, batch normalization is performed on the multi-scale feature map to obtain the normalized multi-scale feature map, and based on the optimized contrastive learning weights, a pre-trained feature representation is generated, which is expressed as follows: ； in: is the pre-trained feature representation, is the feature map weight matrix, is the normalized multi-scale feature map is the bias term.

5. The method for aerial target detection based on multi-source contrastive self-supervised learning according to claim 1, characterized in that: Generating the optimized feature map specifically includes: The multi-scale feature map and the pre-trained feature representation are spliced ​​in the channel dimension to obtain the preliminary fusion feature, which is expressed as follows: ； in: For the initial fusion feature, is the new channel index after splicing, is a multi-scale feature map, is the pre-trained feature representation; Based on the preliminary fusion features, a learnable linear transformation matrix, bias term and activation function are constructed to obtain the optimized feature map, which is expressed as follows: ； in: To optimize the feature map, is the output channel index, is the number of channels of the fused feature, is the activation function, is a learnable linear transformation matrix, is the bias term.

6. The method for aerial target detection based on multi-source contrastive self-supervised learning according to claim 1, characterized in that: The obtaining of the total set of candidate region anchor frames specifically includes: Based on the optimized feature map, a multi-scale anchor box of the original candidate region is generated at the pixel coordinates of the optimized feature map on each layer of the multi-layer feature pyramid. Its expression is as follows: ； in: For the feature pyramid Layer, pixel coordinates And the anchor box ratio index The original candidate region anchor box generated below, 、 is the coordinate of the center point of the anchor frame mapped to the aerial image data coordinate system, 、 Respectively Layer corresponding anchor box ratio index The width and height of the anchor box; For each original candidate region anchor box The background score is calculated by background separation, and its expression is as follows: ； in: Score for background, is the weight parameter for background separation, is the bias term, To optimize the feature map in pixel coordinates and channels The eigenvalues ​​on To optimize the number of channels of the feature map; The original candidate region anchor frames are filtered based on the background score and the preset threshold to obtain candidate region anchor frames, and the candidate region anchor frame sets from each layer of the multi-layer feature pyramid are aggregated to obtain the total set of target-related candidate region anchor frames; the filtering expression is as follows: ； ； in: is the set of target-related candidate regions to be retained, A collection of background areas.

7. The method for aerial target detection based on multi-source contrastive self-supervised learning according to claim 1, characterized in that: Outputting the target category probability and the precisely located bounding box as the target detection result specifically includes: Based on the total set of target-related candidate region anchor frames and the optimized feature map, the feature vector within each candidate region is extracted through the region interest alignment operation, and its expression is as follows: ； in: is the eigenvector, For the region of interest alignment operation, To optimize the feature map, is the anchor box of the candidate region; Calculate the target category probability based on the feature vector and the multi-class cross entropy loss function; Based on the target features in the optimized feature map, the anchor box of the candidate region is regressed and positioned, the bounding box prediction offset is calculated, and the anchor box of the original candidate region is adjusted based on the bounding box prediction offset to generate a precisely positioned bounding box. The expression is as follows: ； in: are the adjusted bounding box center coordinates, is the width and height of the adjusted bounding box, 、 、 、 Predict offsets for bounding boxes; The target category probability and the precisely located bounding box are output as the target detection result.

8. The method for aerial target detection based on multi-source contrastive self-supervised learning according to claim 1, characterized in that: Obtaining target detection results that take into account both small and large targets specifically includes: Set a small target area threshold and divide all precisely located bounding boxes into small target bounding boxes and large target bounding boxes. If the area of ​​a bounding box is smaller than the small target area threshold, it is classified as a small target bounding box; otherwise, it is classified as a large target bounding box. Perform regional interest alignment on the underlying feature map for the small object bounding box to obtain a fixed-size fine-grained feature block, and then perform attention enhancement through the channel attention mechanism and the spatial attention mechanism in sequence to obtain the attention-enhanced small object feature; The attention-enhanced small object features are input into the classification network and the regression sub-network. The classification network outputs the category confidence, and the regression sub-network outputs the fine-tuned offset of the bounding box. Based on the fine-tuned offset, the original precisely positioned bounding box is precisely positioned to generate a set of small target precise bounding boxes and filter out redundancies. Fuse the set of small target bounding boxes with the set of large target bounding boxes; if the same position appears in both the small target bounding box and the large target bounding box during fusion, the small target bounding box will be output first; All fused bounding boxes are sorted by confidence and redundancy is filtered again to output object detection results that take into account both small and large objects.

9. The method for aerial target detection based on multi-source contrastive self-supervised learning according to claim 1, characterized in that: Generating the final UAV aerial photography target detection results specifically includes: Based on the target detection results that take into account both small and large targets, the confidence of each bounding box is defined, and its expression is as follows: ； in: is the confidence of the bounding box, is the target category probability, Bounding box for precise positioning and the corresponding real box The intersection and union ratio of All bounding boxes are sorted in descending order according to their confidence levels to construct an ordered candidate box set. Each element in the ordered candidate box set is represented as follows: ； in: is an ordered set of candidate boxes, For the bounding boxes, For the The confidence of the bounding box, is the confidence threshold; The redundant bounding boxes in the ordered candidate box set are filtered based on the non-maximum suppression method. The expression is as follows: ； in: is the set of bounding boxes retained after non-maximum suppression, is the intersection-over-union ratio threshold for non-maximum suppression, Respectively and Bounding boxes; The position information and classification results in the retained bounding box set are optimized and weighted to obtain the final drone aerial photography target detection result, which is expressed as follows: ； in: For the The position parameters of the bounding box, For the The object class probability of each bounding box.

10. An aerial target detection system based on multi-source contrastive self-supervised learning, characterized in that: The system is applied to the method according to any one of claims 1 to 9, and the system comprises: An aerial image data generating unit (1) is used to obtain multi-source aerial image data from a UAV and generate standardized aerial image data; A multi-scale feature generation unit (2) is used to construct a multi-layer feature pyramid structure, wherein the multi-layer feature pyramid structure includes a bottom-up feature extraction path and a top-down feature fusion path; based on the bottom-up feature extraction path, a high-level feature map and a low-level feature map of a multi-scale target in the standardized aerial image data are extracted, and based on the top-down feature fusion path, the high-level feature map and the low-level feature map are fused to generate a multi-scale feature map; A pre-trained feature generation unit (3) is used to perform self-supervised pre-training on multi-scale feature maps based on a multi-source contrastive self-supervised learning mechanism, construct positive sample pairs and negative sample pairs of multi-source data; optimize feature extraction pre-training weights through contrastive learning, and generate pre-trained feature representations; An optimized feature map generation unit (4) is used to enhance and fuse multi-scale feature maps based on pre-trained feature representation, and to perform comparative learning on feature maps of different scales to generate an optimized feature map; The candidate region anchor frame acquisition unit (5) is used to generate candidate regions based on the optimized feature map, and generate multi-scale original candidate region anchor frames on each layer of the multi-layer feature pyramid, optimize the background distinction ability of the candidate region through background separation, screen out target-related candidate region anchor frames, and obtain a total set of candidate region anchor frames; The target category probability and bounding box determination unit (6) is used to extract the feature vector in each candidate region based on the region interest alignment operation and calculate the target category probability; regress and locate the anchor frame of the candidate region based on the optimized feature map to generate a precisely located bounding box; and output the target category probability and the precisely located bounding box as the target detection result; A small target optimization unit (7) is used to divide the precisely located bounding box into a small target bounding box and a large target bounding box, and after precisely positioning the small target bounding box, fuse it with the large target bounding box to obtain a target detection result that takes into account both small and large targets; The target detection unit (8) is used to define the confidence of each bounding box based on the target detection results of both small targets and large targets, and to sort and construct an ordered candidate box set; filter the redundant bounding boxes in the ordered candidate box set, and optimize the position and weight the category of the retained bounding boxes to obtain the final UAV aerial photography target detection result.

Images