An unmanned aerial vehicle target detection method based on density and center prior hierarchical guidance

Abstract

The application relates to the technical field of computer vision, and discloses a UAV target detection method based on density and center prior hierarchical guidance, which comprises feature extraction, coding, query allocation, decoding and joint training modules. First, multi-scale visual features are extracted, and center and density prior features are generated in a shallow network; spatial and frequency domain modulation is performed on the multi-scale visual features; subsequently, a sampling offset field is calculated by using prior information, grid resampling and residual fusion are performed on deep features, and spatial alignment of multi-scale features is realized; then, a query quota is dynamically allocated according to regional density integration, candidate query points are adaptively extracted, content features and reference frames are corrected, an attention bias matrix is generated in combination with the spatial geometric features of the candidate node reference frame, and the network is updated in combination with a joint loss function. The application dynamically allocates a query quota by using density and center prior and generates an attention bias, overcomes feature mixing in dense areas, and improves the detection precision of dense small targets.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a method for UAV target detection based on density and center prior hierarchical guidance. Background Technology

[0002] In recent years, drones have been widely used in many fields such as traffic monitoring, agricultural surveys, and disaster relief due to their advantages such as high mobility and wide field of view. As a result, target detection technology for drone aerial images has become an important research direction in the field of computer vision. Compared with conventional ground natural images, drone aerial images have the characteristics of high viewing angle and wide coverage.

[0003] Existing target detection algorithms have achieved high accuracy when processing regular images, but they still face adaptability issues when directly transferred to drone aerial photography scenarios. Because targets in aerial images are physically small and affected by changes in flight altitude, the scale differences of similar targets are large. When extracting deep semantic features, existing networks need to perform multiple downsampling operations, which causes the already weak feature information of small targets to be submerged by background noise and also causes spatial position shifts. In subsequent multi-scale feature fusion, there is a spatial misalignment between deep and shallow features, affecting the network's accuracy in locating small targets.

[0004] Meanwhile, the spatial distribution of targets in aerial photography scenarios exhibits extreme non-uniformity. A high-resolution image contains large areas without target backgrounds, while local areas are densely packed with vehicles or crowds. Existing target detection networks based on query mechanisms or dense prediction initialize detection nodes or allocate computational resources uniformly throughout the entire image space. This lack of an adaptive mechanism based on target spatial density causes the model to consume unnecessary computation in areas without target backgrounds, while in areas with highly dense targets, the insufficient number of allocated detection nodes prevents full coverage of all targets, resulting in a high false negative rate.

[0005] Furthermore, when faced with densely packed adjacent targets, the attention mechanism or feature interaction process of existing models mainly relies on the semantic similarity between feature channels for calculation, lacking explicit constraints on the relative geometric position and local crowding degree between targets. This can lead to the features of adjacent overlapping targets being confused with each other, and the bounding box regression process being interfered with by the surrounding background or other targets, resulting in inaccurate localization or incorrect suppression in the post-processing stage. Therefore, how to combine spatial distribution features to guide the detection process is a technical problem that needs to be solved to improve the performance of UAV target detection. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a UAV target detection method based on density and center prior hierarchical guidance. This method solves the technical problems of misalignment of deep and shallow feature spaces when facing multi-scale changes, inefficient query node allocation when target distribution is extremely uneven, and confusion and missed detection of dense small targets in existing UAV target detection methods.

[0007] To achieve the above objectives, the present invention provides the following technical solution: A hierarchical UAV target detection method based on density and center prior guidance includes: Aerial images of the drone to be detected are acquired, and shallow and deep visual features are extracted from the multi-scale visual features of the drone to be detected. A center feature map and a density feature map are generated based on the shallow visual features. The multi-scale visual features are modulated using the center feature map and the density feature map. The scale-aligned center prior features, density prior features and query-assigned density prior features are output. Based on the aforementioned center prior features, density prior features, and shallow visual features, a sampling offset field is calculated. The deep visual features are then resampled and fused to output spatially aligned multi-scale high semantic features. Based on the aforementioned query allocation density prior features, a local query budget is determined, and the number of query candidates in each region is determined according to the regional density distribution. Local candidate query points are extracted based on the number of query candidates and the classification quality, center response, and density response corresponding to the candidate positions. Candidate positions that are not locally selected are globally supplemented based on classification quality to obtain a preset number of candidate query points. Based on the candidate query points, extract the corresponding content feature vectors, prior features, and reference boxes. Use the prior features to correct the content feature vectors and reference boxes to generate an initial candidate query set. Generate an attention bias matrix based on the geometric feature vectors of the reference boxes. Embed the attention bias matrix into a self-attention mechanism to update the query features. Perform cross-attention interaction between the updated query features and the spatially aligned multi-scale high semantic features to output the detection results. Based on the error loss between the center feature map and the density feature map and the corresponding spatial distribution ground value, combine classification and regression losses to construct a joint loss function to update the network.

[0008] Furthermore, the step of modulating the multi-scale visual features using the central feature map and the density feature map includes: The central feature map and the density feature map are scaled to obtain central prior features and density prior features corresponding to visual feature spatial dimensions at different scales. The multi-scale visual features are divided into an identity mapping branch, a local spatial branch, and a frequency domain branch according to a preset channel ratio. The local spatial branch is then mapped with spatial attention weights using the central prior features. The density prior features are converted into a density weight matrix. The spatial domain features of the frequency domain branch are modulated and then restored to spatial domain features after frequency domain transformation. The modulated local spatial branch and frequency domain branch are then fused with the identity mapping branch.

[0009] By aligning the central feature map and density feature map at scale and constructing a multi-branch structure, the local spatial features near the target center are enhanced by utilizing the central prior, and the target distribution information is modeled by using the density prior in the frequency domain branch, thereby improving the expressive power of multi-scale visual features in complex backgrounds.

[0010] Furthermore, the process of outputting scale-aligned center prior features, density prior features, and query-assigned density prior features includes: The shallow visual features are input into a shared prior feature extraction network, and then into the center prior prediction branch and the density prior prediction branch respectively; the outputs of the two branches are activated by activation functions to obtain the center feature map and the non-negative density feature map respectively. The density feature map is used as the prior density feature for query allocation; the center feature map is subjected to a scaling transformation that preserves the peak response of the target center to obtain the prior center feature at the corresponding scale; the density feature map is subjected to a scaling transformation that preserves the density distribution of the region to obtain the prior density feature at the corresponding scale.

[0011] By extracting center and density prior information in parallel from shallow high-resolution visual features and using different scale transformation methods to maintain the target center response and regional density distribution respectively, unified spatial prior information is provided for subsequent hierarchical feature formation, cross-scale feature alignment and query allocation.

[0012] Furthermore, the step of calculating the sampling offset field based on the center prior features, density prior features, and shallow visual features includes: Upsampling interpolation is performed on the deep visual features to obtain upsampled deep visual features; The center prior feature, the density prior feature, the upsampled deep visual feature, and the shallow visual feature corresponding to the current feature interaction node are spliced ​​in the channel dimension; the spliced ​​features are dimensionality-reduced using a convolutional mapping network, and a two-dimensional grid sampling offset field with horizontal and vertical offset channels is generated by constraining the hyperbolic tangent function.

[0013] By stitching together the center and density prior information with the channels of multi-scale visual features, and using the hyperbolic tangent function to constrain the offset amplitude, a deformation sampling grid is generated, providing spatial coordinate offset constraints for subsequent feature alignment.

[0014] Furthermore, the steps of resampling and fusing the deep visual features to output spatially aligned multi-scale high semantic features include: The standard two-dimensional spatial coordinate matrix is ​​numerically superimposed using the two-dimensional grid sampling offset field, and then bilinear interpolation is performed on the upsampled deep visual features to resample the grid and obtain spatially aligned deep visual features. Based on the spatially aligned deep visual features and the upsampled deep visual features before resampling, residual calibration features are constructed. Weighted fusion is performed using preset or learnable calibration parameters to obtain calibrated deep visual features. The calibrated deep visual features are then fused with the shallow visual features at multiple scales.

[0015] By resampling the grid to calibrate the spatial position of deep visual features, the spatial position deviation during cross-scale feature interaction is reduced, and residual calibration is used to preserve the original deep semantic information, thereby improving the spatial alignment accuracy of multi-scale features.

[0016] Furthermore, the steps of allocating the number of query candidates based on the prior features of query allocation density, and extracting candidate query points based on the number of query candidates include: Obtain a preset total number of queries parameter, determine a local query budget based on the overall density integral of the query allocation density prior feature, and limit the local query budget to within the total number of queries parameter; The query allocation density prior features are divided into independent grid regions, and the density integral value of each independent grid region and the total integral value of the grid region are calculated based on the query allocation density prior features. Calculate the floating-point query quota for each independent grid region based on the proportional relationship and the local query budget; use the Hamilton maximum remainder method to assign a base quantity based on the integer part of the floating-point query quota, and allocate the remaining quota in descending order of the decimal part to determine the number of query candidates for each grid region. The local joint quality is determined based on the classification quality, center response, and density response corresponding to the candidate positions. Based on the number of query candidates, candidate query points with higher local joint quality are selected in each grid area. After excluding the selected local candidate query points, the remaining candidate positions are globally sorted according to classification quality, and candidate query points are added until the total number of queries is reached.

[0017] By introducing a density integral-based local query budget and the Hamiltonian maximum remainder method, integer query quotas are adaptively allocated according to the regional density distribution. Combined with regional local selection and global quality supplementation, the query coverage of dense target regions is improved while keeping the total number of queries constant, while also taking into account sparse regions and high-quality global candidates.

[0018] Furthermore, the step of generating an initial candidate query set based on the candidate query points includes: Based on the candidate query point, coordinate transformation and feature sampling are performed on the center prior feature and density prior feature of the corresponding scale. The center response and density response are extracted respectively, and combined with the classification quality corresponding to the candidate query point to form the candidate prior feature. The candidate prior features are input into the prior content mapping network to align the feature dimensions, and then residual fusion is performed with the content feature vector to obtain the corrected content feature vector. The candidate prior features are input into the reference box correction network to generate reference box correction values, and residual fusion is performed with the corresponding reference boxes to obtain the corrected reference boxes. The corrected content feature vectors and reference boxes are then associated to generate the initial candidate query set.

[0019] By forming candidate prior features from classification quality, center response, and density response, the initial content feature vector and reference box are modified using the candidate prior features, providing the decoder with initial query conditions that combine visual semantics, target center, and local distribution information.

[0020] Furthermore, the step of generating an attention bias matrix based on the geometric feature vectors of the reference box includes: Calculate the normalized relative center position, logarithmic scale ratio feature, intersection-union ratio feature and relative orientation angle based on any two of the reference frames, and then concatenate them to form the geometric feature vector; The geometric feature vector is mapped to a scalar bias value through a non-linearly activated multilayer perceptron network to generate the attention bias matrix.

[0021] By fusing the relative positions, scale differences, and spatial overlap between the reference boxes corresponding to the query nodes, the degree of spatial geometric association between the query nodes is quantified, providing structured location prior information for attention computation.

[0022] Furthermore, the steps of embedding the attention bias matrix into a self-attention mechanism to update the query features, and performing cross-attention interaction between the updated query features and the spatially aligned multi-scale high semantic features include: Generate a query feature matrix, a key feature matrix, and a value feature matrix based on the corrected content feature vector; calculate the dot product of the query feature matrix and the transpose of the key feature matrix; and add the attention bias matrix element by element to the dot product result after dividing by the scaling factor. The superimposed dot product result is input into the normalized exponential activation function to generate an attention weight matrix, and multiplied with the value feature matrix to generate a self-attention updated query feature; the self-attention updated query feature is used as the query input, and multi-scale cross-attention interaction is performed with the spatially aligned multi-scale high semantic feature as the image feature input.

[0023] By incorporating the attention bias matrix into the attention dot product operation in a numerical superposition manner, the self-attention mechanism is guided to model the spatial positional relationship between targets corresponding to different query nodes, thereby enhancing the feature discrimination ability between densely adjacent targets.

[0024] Furthermore, the step of constructing a joint loss function to update the network based on the error loss between the center feature map and the density feature map and the corresponding spatial distribution ground value, combined with classification and regression losses, includes: The Gaussian heatmap truth matrix of the center point is synthesized based on the coordinates of the actual two-dimensional center point of the labeled real target; the standard deviation parameter of the Gaussian density kernel is determined based on the actual bounding box size, and an adaptive Gaussian density map truth matrix is ​​generated by unit mass normalization and element-wise summation. The error between the center feature map and the true value of the center point Gaussian heat map is calculated to obtain the center space error loss. The error between the density feature map and the true value of the adaptive Gaussian density map is calculated to obtain the density spatial error loss, and the density quality consistency loss is obtained based on the difference between the spatial integral of the density feature map and the true number of targets. The joint loss function is constructed by weighting the center space error loss, density space error loss, density quality consistency loss, classification loss, and regression loss according to preset weights.

[0025] By combining the center and density prior errors to construct a joint loss function, the network can be guided to learn more accurate spatial distribution and target density in complex aerial photography scenarios, thereby improving the overall detection accuracy of the model.

[0026] This invention provides a UAV target detection method based on density and center prior hierarchical guidance. It has the following beneficial effects: 1. This invention generates a center feature map and a density feature map by extracting shallow visual features, modulates the features of multi-scale visual features in local spatial branches and frequency domain branches, and performs grid resampling and residual calibration on deep visual features in combination with sampling offset fields. It uses shallow visual features and center and density prior information to guide deep semantic features to perform spatial position calibration, reduces the spatial position deviation between features of different scales, and improves the feature expression and spatial alignment ability of multi-scale high semantic features in complex aerial photography backgrounds.

[0027] 2. This invention utilizes the overall density integral of prior features of query allocation density to determine the local query budget, and adaptively allocates regional query quotas based on the density integral of each grid region using the Hamiltonian maximum remainder method. Within each region, candidate query points are selected based on the local joint quality composed of classification quality, center response, and density response, while supplementing the remaining query quantity using global classification quality. This strategy, combining local density-aware allocation with global quality supplementation, enables limited query resources to accurately cover dense target areas while maintaining a fixed total number of queries, and also takes into account sparse areas and high-quality global candidates, thereby improving the query coverage and detection recall rate of dense small targets from the perspective of UAVs.

[0028] 3. This invention utilizes classification quality, center response, and density response to construct candidate prior features, performs residual correction on the content feature vectors and reference boxes of selected candidate points, and generates an attention bias matrix based on the geometric relationship between the initial reference boxes to embed self-attention calculation. Simultaneously, it introduces Gaussian heatmaps, adaptive Gaussian density maps, and density-quality consistency constraints to construct a joint loss function, enabling the query initialization process to fully integrate target center and spatial distribution information, enhance the feature discrimination ability of densely adjacent targets by utilizing the spatial geometric relationship between targets, and effectively guide the network to learn the target distribution pattern during the training phase, thereby improving the detection accuracy and stability of the model in dense small target and complex occlusion scenarios. Attached Figure Description

[0029] Figure 1 This is the interaction timing diagram of the present invention; Figure 2 This is a schematic diagram of the method flow of the present invention; Figure 3 This is a flowchart of the multi-scale visual feature modulation and prior feature generation process of the present invention; Figure 4 This is a sequence diagram for the generation of query quota allocation and initialization candidate query set in this invention; Figure 5 This is a flowchart of the multimodal feature fusion and target detection decoding prediction process of the present invention. Detailed Implementation

[0030] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] See attached document Figure 1 This invention provides a UAV target detection system based on density and center prior hierarchical guidance. The system includes: a feature extraction module, a feature encoding module, a query allocation module, a feature decoding module, and a joint training module.

[0032] The feature extraction module is used to receive drone aerial images input from external devices, extract multi-scale visual features through the backbone network, and generate a center feature map and a density feature map based on shallow visual features.

[0033] The feature extraction module performs scale alignment on the center feature map and density feature map respectively, and uses the scale-aligned center prior features and density prior features to perform feature modulation operations on the multi-scale visual features with local spatial branching and frequency domain branching.

[0034] The feature extraction module outputs center prior features and density prior features at different scales, and uses the base-scale density feature map as the query allocation density prior feature for subsequent query budget and regional quota calculation.

[0035] The feature encoding module is connected to the feature extraction module. The feature encoding module receives modulated multi-scale visual features as well as the center prior features and density prior features corresponding to the current feature interaction node.

[0036] The feature encoding module uses the center prior features, density prior features, and multi-scale visual features of the two adjacent layers corresponding to the current feature interaction node to calculate the two-dimensional grid sampling offset field. Based on the coordinate information corresponding to the two-dimensional grid sampling offset field, the upsampled deep visual features are resampled.

[0037] The feature encoding module performs residual calibration on the resampled deep visual features and the upsampled deep visual features before resampling, and performs multi-scale feature fusion on the calibrated deep visual features and shallow visual features to output spatially aligned multi-scale high semantic features.

[0038] The query allocation module is connected to the feature extraction module and the feature encoding module. The query allocation module receives query allocation density prior features, scale-aligned center prior features and density prior features, and receives multi-scale high semantic features output by the feature encoding module, as well as candidate classification quality and candidate bounding box information generated based on the features.

[0039] The query allocation module determines the local query budget based on the overall density integral of the query allocation density prior features, divides the query allocation density prior features into multiple spatial grid regions and calculates the regional density integral, and performs a quantity allocation operation in combination with the local query budget to determine the number of query candidates corresponding to each grid region.

[0040] The query allocation module determines the local joint quality based on the classification quality, center response, and density response corresponding to the candidate position, and selects local candidate query points based on the number of query candidates in each grid region. After excluding the selected local candidates, it globally supplements the total number of queries based on the original classification quality. Then, it synchronously extracts the content features, prior features, and reference boxes corresponding to the selected candidates, uses the prior features to perform residual correction on the content features and reference boxes, and outputs the initial candidate query set.

[0041] The feature decoding module is connected to the feature encoding module and the query allocation module. The feature decoding module receives multi-scale high semantic features and an initial candidate query set.

[0042] The feature decoding module extracts the geometric coordinate difference features between the reference boxes corresponding to each query node in the initial candidate query set, and generates an attention bias matrix by mapping through a fully connected layer.

[0043] The feature decoding module embeds the attention bias matrix into the multi-head scaling dot product self-attention calculation equation of the detection decoder, performs feature decoding interaction, and performs multi-scale cross-attention interaction between the self-attention updated query features and the multi-scale high semantic features output by the feature encoding module, outputting the final target category probability and bounding box coordinate prediction results.

[0044] The joint training module is communicatively connected to both the feature extraction module and the feature decoding module, and it runs during the network parameter training phase of the detection system.

[0045] The joint training module obtains the target's true coordinate annotation data corresponding to the input image samples, and generates the true values ​​of the center point Gaussian heatmap and the adaptive Gaussian density map accordingly. It also calculates the center spatial error loss between the center feature map and the true value of the center point Gaussian heatmap, the density spatial error loss between the density feature map and the true value of the adaptive Gaussian density map, and the density quality consistency loss, respectively.

[0046] The joint training module weights and sums the center space error loss, density space error loss, density quality consistency loss, and target classification loss and bounding box regression loss corresponding to candidate prediction and final detection results to generate a joint loss function value, and then backpropagates the gradient signal to update the network parameters based on the joint loss function value.

[0047] See attached document Figure 2 This invention provides a UAV target detection method based on density and center prior hierarchical guidance, the method comprising the following steps: S1: Obtain the drone aerial image to be detected from the external device and input it into the feature extraction module to extract multi-scale visual features. Generate a center feature map and a density feature map in parallel in the shallow network. Perform scale alignment on the center feature map and the density feature map respectively. Use the scale-aligned center prior features and density prior features to perform local spatial and frequency domain modulation on the multi-scale visual features to obtain prior-guided multi-scale visual features. At the same time, retain the basic scale density feature map as the query allocation density prior feature. S2 calculates the sampling offset field using the center prior feature, density prior feature and multi-scale visual features of the two adjacent layers corresponding to the current feature interaction node. Based on the coordinate information corresponding to the sampling offset field, the upsampled deep visual features are resampled using a grid. The deep visual features before and after resampling are then calibrated with residuals. Finally, they are fused with the adjacent shallow visual features at multiple scales to output spatially aligned multi-scale high semantic features. S3: Determine the local query budget based on the overall density integral of the query allocation density prior features; divide the query allocation density prior features into multiple grid regions and allocate the number of query candidates according to the region density integral; determine the local joint quality based on the classification quality, center response, and density response corresponding to the candidate positions; extract local candidate query points in each region; and globally supplement the unselected candidates based on the original classification quality; simultaneously extract the content features, prior features, and reference boxes corresponding to the final candidates; and use the prior features to perform residual correction on the content features and reference boxes to generate an initial candidate query set. S4. Calculate the geometric relationship features between the initial reference boxes corresponding to each query node in the initial candidate query set. After mapping, generate the attention bias matrix. Embed the attention bias matrix into the self-attention calculation logic of the detection decoder for feature interaction. Then, perform multi-scale cross-attention interaction between the self-attention updated query features and multi-scale high semantic features to output the final target category and bounding box prediction results. S5. During the system network parameter training phase, the ground truth values ​​of the Gaussian heatmap and the adaptive Gaussian density map of the center point are generated by obtaining the real labeled data corresponding to the input image samples. The prior supervision error between the center feature map and the density feature map and the corresponding spatial distribution ground truth values ​​is calculated, and the density quality consistency loss is calculated. The joint loss function is constructed by combining the classification loss and bounding box regression loss corresponding to the candidate prediction results and the final detection results, and the gradient is backpropagated to update the network parameters.

[0048] See attached document Figure 3 In specific implementations, step S1 provided by the present invention may include the following steps: S110: The drone aerial image to be detected is acquired from an external device and input into the backbone network within the feature extraction module. The input drone aerial image is represented as a three-channel two-dimensional image tensor. The backbone network uses a multi-layer convolutional structure to extract multi-scale visual features. The basic feature matrix is ​​extracted from the shallow high-resolution feature nodes. First, channel mapping is performed through 1×1 convolution, and then local spatial information is extracted through 3×3 depth-separable convolution to obtain the prior representation features shared by the central prior prediction branch and the density prior prediction branch.

[0049] The center prior prediction branch and the density prior prediction branch are composed of parallel, independent feature mapping paths. The center prior prediction result is mapped to a numerical range of 0 to 1 by the Sigmoid nonlinear activation function, and the density prior prediction result is mapped to a non-negative value by the Softplus nonlinear activation function, respectively, to obtain a single-channel center feature map and a single-channel density feature map. The center feature map is used to characterize the response degree of the target center at each spatial location, and the density feature map is used to characterize the spatial distribution density of the target in different image regions.

[0050] S120, the center feature map and density feature map are predicted in parallel by shared prior representation features. There is no direct feature interaction or numerical fusion between the center prior prediction branch and the density prior prediction branch. The expressions for the shared prior representation and the center and density prior generation process are as follows: ; ; ; In the formula, To share prior representation features; This is a shallow visual feature tensor extracted from shallow high-resolution feature nodes. For use in channel mapping Standard convolution operation; For extracting local spatial information Depthwise separable convolution operations This is a single-channel center feature map, used to characterize the response level of the target center at each spatial location; It is a Sigmoid non-linear activation function used to map values ​​to the interval between 0 and 1; The unique feature of the center-prior prediction branch Depthwise separable convolution operations; The unique feature of the center-prior prediction branch Standard convolution operation; It is a single-channel density feature map used to characterize the spatial distribution density of the target in different image regions, and also serves as a prior feature for query allocation density; It is a non-negative nonlinear activation function used to ensure that the density feature output is a non-negative value; Specific to density prior prediction branch Depthwise separable convolution operations; Specific to density prior prediction branch Standard convolution operation.

[0051] Among them, the center prior prediction branch and the density prior prediction branch perform local feature extraction and channel mapping on the shared prior representation features, respectively, to ensure that the center response and density distribution are generated in parallel from the same shallow high-resolution semantic basis.

[0052] The density feature map also serves as a prior feature for query allocation density, and is directly supervised by the ground truth of the adaptive Gaussian density map at the base scale during the training phase. The spatial integral result is used for subsequent local query budget and regional query quota calculation.

[0053] S130, the central feature map and density feature map are input into the spatial prior scale alignment operation, and spatial size transformation is performed on each multi-scale visual feature to be modulated; wherein, adaptive max pooling is used on the central feature map to preserve the local peak response, and adaptive average pooling is used on the density feature map to characterize the regional density distribution of the corresponding scale; then, the visual feature to be modulated is divided into identity mapping branch, local spatial branch and frequency domain branch according to a preset channel ratio, and in one embodiment the channel ratio of the three is set to 1∶2∶1.

[0054] For the local spatial branch, spatial modulation weights are generated using the corresponding scale center prior features, and element-wise response modulation is performed on the local spatial features. For the frequency domain branch, density modulation weights are generated using the corresponding scale density prior features, and response modulation is performed on the spatial domain features. After being processed in the frequency domain by fast Fourier transform, the features are restored to the spatial domain. Thus, the central response and regional density information are used to guide local spatial modeling and frequency domain distribution modeling, respectively.

[0055] Subsequently, the local spatial branch modulated by the center prior, the frequency domain branch modulated by the density prior, and the identity mapping branch that maintains the original data flow direction are spliced ​​and fused in the channel dimension to obtain the prior guided visual features at the current scale; the remaining feature levels to be modulated are processed in the same way.

[0056] S140: For visual features at different scales, output the center prior features and density prior features corresponding to the spatial size respectively, and use the scale alignment prior for subsequent cross-scale feature alignment; the query allocation density prior features still use the basic scale density feature map, and do not perform residual update through deep visual features.

[0057] The scale alignment expression for the center feature map and the density feature map is as follows: ; ; In the formula, For the first Dynamic central prior feature matrix at each scale; This is the scale level index number of the multi-scale feature map; This is an adaptive max-pooling operation used to preserve local peak responses; It is a single-channel center feature map matrix at the basic scale; For the first Dynamic density prior feature matrix at each scale; This is an adaptive average pooling operation used to characterize the regional density distribution at the corresponding scale; It is the single-channel density feature map matrix at the basic scale.

[0058] The center prior scale alignment process focuses on maintaining the peak response of the target center position, while the density prior scale alignment process focuses on maintaining the target distribution information in the region, so that the same set of center and density priors can be reused at different feature levels.

[0059] In specific implementations, step S2 provided by this invention may include the following steps: S210, the feature encoding module obtains the deep visual features after upsampling and interpolation, obtains the adjacent shallow visual features that are consistent with the upsampled deep visual features in terms of spatial scale, and selects the scale alignment center prior features and density prior features corresponding to the current feature interaction node. After performing necessary feature mapping on the center prior features, density prior features, deep visual features processed by upsampling interpolation, and adjacent shallow visual features, tensor splicing operation is performed in the channel dimension to obtain fused features for predicting the sampling offset field.

[0060] A convolutional mapping network is used to perform feature mapping on the fused features, outputting two offset channels in the horizontal and vertical directions. The predicted offset is constrained within a bounded range using a hyperbolic tangent function. Furthermore, the scale-dependent maximum offset coefficient is used to control the allowable offset amplitude at different feature levels, resulting in a two-dimensional grid sampling offset field, as expressed below: ; In the formula, For the first Two-dimensional spatial sampling offset field predicted by each feature interaction node; This is the scale level index number of the multi-scale feature map; This represents the maximum offset coefficient corresponding to the current scale; It is the hyperbolic tangent activation function; This is a convolution mapping operation; This is a tensor concatenation operation for the feature channel dimension; These are the upsampled deep visual features; These are visual features of adjacent shallow layers; and These are the center prior features and density prior features at the current scale, respectively.

[0061] S220, the feature encoding module uses a two-dimensional grid sampling offset field to correct the position of the standard two-dimensional spatial coordinate matrix composed of basic grid points, and then performs a grid resampling operation based on bilinear interpolation algorithm on the upsampled and interpolated deep visual features according to the corrected sampling coordinates to obtain spatially aligned deep visual features; wherein, spatial resampling is only performed on the upsampled deep visual features, and adjacent shallow visual features retain their original spatial positions and are used as spatial references. The expression for the grid resampling operation is as follows: ; In the formula, These are deep visual features that have undergone spatial deformation alignment processing; This is a bilinear grid resampling operation; These are the upsampled deep visual features; The spatial sampling grid coordinate matrix is ​​a standard rule; This is the predicted two-dimensional spatial sampling offset field.

[0062] S230, the feature encoding module constructs a residual calibration term based on the spatially aligned deep visual features and the upsampled deep visual features before resampling, and uses a preset or learnable calibration coefficient to control the correction magnitude of the resampling result on the original deep semantic features; in one embodiment, the calibration coefficient is initialized with a small value to maintain the stability of the original deep semantic features in the early stage of network training.

[0063] After residual calibration is completed, the calibrated deep visual features are input into the multi-scale feature fusion structure and fused with the adjacent shallow visual features to output the spatially aligned high semantic features corresponding to the current feature interaction node. The same prior guided alignment and multi-scale fusion operation are performed on the other adjacent feature levels in sequence to obtain the spatially aligned multi-scale high semantic features.

[0064] The residual calibration expression for deep visual features is as follows: ; In the formula, These are the deep visual features after residual calibration; These are upsampled deep visual features before resampling; Spatially aligned deep visual features after resampling; This is the calibration coefficient used to control the spatial calibration amplitude.

[0065] See attached document Figure 4 In specific implementations, step S3 provided by the present invention may include the following steps: S310, the query allocation module obtains the query allocation density prior features output by the feature extraction module and calculates the density integral within the complete spatial region; since the query allocation density prior features are supervised by the unit mass density true value, the overall integral is used to characterize the target demand estimation of the current image and to determine the local query budget accordingly.

[0066] The query allocation density prior features are adaptively divided into multiple non-overlapping fixed regions according to the normalized image space; in one embodiment, the region is divided into 16 grid regions of 4×4, and the density response in each grid region is integrated to obtain the region density integral.

[0067] The region density integral is used to characterize the relative target demand of each grid region; the center prior does not participate in the region classification, and the region number of the candidate position is determined according to the normalized spatial coordinates in the original feature grid.

[0068] S320, the query allocation module obtains the preset total number of queries for the detection network. In one embodiment, the total number of queries for the detection network is set to 300. First, the query allocation density prior features are integrated globally and rounded to obtain the target demand estimate, which is limited to between 0 and 300, to obtain the local query budget. Then, the floating-point query quota of each region is calculated according to the proportion of the density integral of each grid region to the total integral of the region.

[0069] The Hamiltonian maximum remainder method is used to assign a basic candidate quantity based on the integer part of the floating-point query quota for each region, and the remaining local query quota is allocated in descending order of the decimal part to obtain the integer query candidate quantity for each region, so that the sum of the integer query candidate quantities for all regions equals the local query budget.

[0070] When the overall density integral is close to zero or the local query budget is zero, regional local selection is not performed, and all candidate queries are selected by the global classification quality. Under normal circumstances, only queries equal to the number of local query budgets are used for regional local selection, and the remaining number of queries is supplemented by high-quality global candidates, thereby keeping the final total number of queries fixed.

[0071] The expressions for local query budget and regional query quota are as follows: ; ; ; ; ; In the formula, To query the overall density integral value of the assigned density prior features; This refers to a traversal and summation operation on a complete two-dimensional grid of an image. These are the discrete grid indices for the horizontal and vertical coordinates of the feature map, respectively. Density feature map in discrete coordinates The characteristic value at the location; For local queries, specify the integer quantity of the budget. This is a numeric truncation function used to restrict a numeric value to a specified upper or lower limit range; This is the rounding function; This is the preset total number of queries parameter; For the first Floating-point query quota for each grid area; For the first The regional density integral value of each grid region; The index number of the target grid region after division; The sum of density integrals for all gridded regions; The traversal index number for all grid regions; This is a preset minimum constant used to prevent the denominator from being zero during division calculations; This is the set of integer query candidate counts corresponding to each grid region after allocation; Assignment operations for Hamilton's maximum remainder method; For the set of floating-point query quotas for all grid regions; The sum of the number of integer query candidates assigned to each grid region.

[0072] In step S330, the query allocation module unifies and expands the spatially aligned multi-scale high semantic features into candidate sequences through channel unification. In one embodiment with an input size of 640×640, the four scale feature maps are 160×160, 80×80, 40×40, and 20×20, respectively. After being mapped to 128 channels via 1×1 convolution and normalization, they are expanded and concatenated to form a candidate feature sequence containing 34,000 candidate locations. The classification prediction branch and the bounding box prediction branch output the classification prediction results and candidate reference boxes for each candidate location, respectively.

[0073] For each candidate location, the highest probability in the category prediction is taken as the classification quality. The center response is obtained from the center prior features at the corresponding scale for each candidate location, and the density response is obtained based on the quality-preserving density scale alignment result. The density scale alignment employs adaptive average pooling and quality compensation based on the spatial area ratio of the base scale to the target scale to avoid scale transformation weakening the region's density quality. The classification quality, center response, and log-compressed density response are combined into candidate prior features, and a quality correction value is obtained through a prior quality mapping network. This constitutes a local joint quality while preserving the original classification quality.

[0074] The grid region to which each candidate position belongs is determined based on the normalized spatial coordinates of each candidate position. Within the corresponding grid region, candidates are sorted according to their local joint quality, and the top-ranked candidates with a number equal to the integer query candidate number of the grid region are selected to obtain a local candidate index set. After masking the local candidate indexes, the remaining candidates are sorted globally according to their original classification quality. The remaining candidates with the difference in number are selected as global supplements, and the local candidate indexes are merged with the global supplementary candidate indexes to obtain a final candidate index set with a number equal to the preset total query number and no duplicate indexes.

[0075] Based on the final candidate index set, corresponding candidate visual features, candidate prior features, and candidate reference boxes are extracted synchronously from the candidate feature sequence, candidate prior feature sequence, and candidate reference box sequence, so that the three maintain a one-to-one correspondence in the candidate dimension; the candidate visual features are directly used as the visual semantic basis of the query content, and zero-value content is not used for initialization.

[0076] The allocation of regional query quotas and the selection of Top-K candidates belong to the discrete index selection process and do not directly bear the gradient propagation of detection loss. The density prior branch updates parameters through density spatial error loss and density quality consistency loss, and the prior features of the selected candidates participate in the initialization of query content and reference boxes through subsequent continuous mapping.

[0077] The expression for local joint quality is as follows: ; ; ; In the formula, For the first A candidate prior feature vector synthesized by concatenating candidate positions; The candidate position index number in the candidate sequence; This is the concatenation operator for the feature channel dimensions; For the first The original classification quality scalar score corresponding to each candidate position; For the first The central response value corresponding to each candidate position in the central feature map; For the first The density response values ​​corresponding to each candidate location in the density feature map; For the first The quality correction scalar value for each candidate position; For prior quality mapping networks; For the first Local joint quality after correction of candidate positions; Let be the inverse Sigmoid transformation function from probability space to log-odds space; This is a learnable quality correction factor used to control the correction magnitude.

[0078] S340, the selected candidate prior features are input into the prior embedding mapping network to obtain the prior embedding with the same dimension as the candidate visual features, and the query content correction amount and reference box correction amount are generated by the independent content correction mapping network and reference box correction mapping network respectively.

[0079] Candidate visual features are used as the main body of the query content, and the initial query content is obtained by superimposing prior content correction amounts with learnable or preset content calibration coefficients. For candidate reference boxes, reference box correction amounts generated by prior embedding are superimposed in the bounding box log odds parameter space, and the reference boxes are restored to the normalized bounding box coordinate space by the Sigmoid function to obtain the initial reference boxes.

[0080] The expressions for initializing the query content and reference boxes are as follows: ; ; ; In the formula, The prior embedded feature vector after aligning the dimensions; For prior embedding mapping networks; These are the candidate prior features corresponding to the finally selected candidates; This is the feature vector of the initialized query content; The original visual features corresponding to the finally selected candidates; Calibrate coefficients for learnable content; Correct the mapping network for the content; The initialized normalized candidate reference box coordinate vector; It is the Sigmoid non-linear activation function; It is the inverse Sigmoid transform function; This is the original reference box coordinate vector corresponding to the finally selected candidate; Calibration coefficients for a learnable reference box; Correct the mapping network for the reference box.

[0081] See attached document Figure 5 In specific implementations, step S4 provided by the present invention may include the following steps: S410, the feature decoding module obtains the initial candidate query set output by the query allocation module. For any two candidate query nodes, it obtains the corresponding initial reference boxes. It calculates the normalized relative center position feature based on the center coordinates of the two reference boxes, calculates the logarithmic scale ratio feature based on the width and height of the two reference boxes, and calculates the intersection-union ratio feature based on the overlapping area of ​​the two reference boxes. At the same time, it calculates the relative direction angle based on the relative displacement of the centers of the two reference boxes, and concatenates the features to form the geometric feature vector between the query node pairs.

[0082] S420, the feature decoding module inputs the geometric feature vector into a multilayer perceptron network with a nonlinear activation function. The multilayer perceptron network is composed of an input fully connected layer, a nonlinear activation function, and an output fully connected layer connected in series. The multilayer perceptron network maps the geometric feature vector corresponding to each query node to a scalar bias value, generating an attention bias matrix that adjusts the network's attention allocation weights. In one embodiment, as the decoding layer iteratively corrects the reference boxes, the geometric relationships between query nodes are recalculated based on the updated reference boxes, and the corresponding attention bias matrix is ​​generated, ensuring that the prior positional relationships are updated iteratively with the bounding boxes. The expression for the attention bias matrix is ​​as follows: ; ; ; ; In the formula, For the first The and the first Normalized relative position and log-scale ratio features between the reference boxes corresponding to each candidate query node; These are the index numbers of two different candidate query nodes; , , , The first The continuous center x-coordinate, center y-coordinate, width, and height of each candidate reference box; , , , The first The continuous center x-coordinate, center y-coordinate, width, and height of each candidate reference box; To prevent extremely small constants with a denominator of zero or a logarithmic input of zero; This represents the complete geometric feature vector between the concatenated query node pairs; For the first The and the first Cross-union ratio among candidate reference boxes; The relative orientation angle is determined by the relative displacement of the centers of the two reference frames; It is the arctangent function that includes quadrant information; This represents the scalar value of the inter-node attention bias output by the multilayer perceptron network. It is a multilayer perceptron mapping network with a nonlinear activation function.

[0083] S430, the feature decoding module constructs a self-attention computation framework within the detection decoder, taking the initial query content or the updated query features from the previous decoding layer as input, and generating query feature matrices through corresponding linear projections. Key feature matrix Sum characteristic matrix In the decoding layer that introduces prior knowledge of positional relationships, the scaled dot product of the query feature matrix and the transpose of the key feature matrix is ​​calculated, and the attention bias matrix generated from the geometric relationships of the reference boxes is applied. The summation is applied to the scaled dot product result as element-wise tensor additions. The result after stacking the attention bias matrix is ​​input into the normalized exponential activation function to generate the attention weight matrix, and then matrix multiplication is performed with the value feature matrix to obtain the query features after self-attention update. The self-attention interaction expression of the bias embedding is as follows: ; In the formula, The updated query features are the output of the self-attention computation function; This is the query feature matrix generated from the candidate queries; This is the key feature matrix generated from the candidate queries; This is the feature matrix of values ​​generated from the candidate queries; The normalized exponential activation function; is the transpose of the key feature matrix; The channel dimension size parameter of the key feature matrix; The scalar value of attention bias between nodes Constructed attention bias matrix.

[0084] S440, the query features generated by the feature decoding module with biased self-attention update are used as query input, and the spatially aligned multi-scale high semantic features output by the feature encoding module are used as image feature input. The information interaction between the candidate query and the image visual features is performed through the multi-scale cross-attention mechanism. The multi-scale cross-attention mechanism determines the reference sampling position and samples the corresponding image features according to the reference box position of the candidate query in the high semantic feature map at different scales. Specifically, the following steps are taken: the reference sampling position is determined based on the center position of the reference box corresponding to the candidate query in the high semantic feature map at different scales; the sampling offset and attention weight corresponding to each reference sampling position are predicted based on the query features; multiple non-integer sampling coordinates are determined based on the sampling offset; the visual features corresponding to each sampling coordinate are obtained through bilinear interpolation; the visual features at different scales and different sampling positions are weighted and fused based on the attention weight; and the semantic information of the sampled image is fused into the query features. The updated decoded query features are then obtained through a feedforward network. The target class probability and bounding box coordinate correction are output through the classification prediction branch and the bounding box regression prediction branch, respectively. The current reference box is updated using the bounding box coordinate correction. In subsequent decoding layers, feature interaction and reference box correction are performed in the same way until the final target class and bounding box coordinate prediction results are obtained.

[0085] In specific implementations, step S5 provided by the present invention may include the following steps: S510, the joint training module obtains the true bounding box annotation data containing the target's true location annotation information in the training sample set of UAV aerial images. When the training samples perform geometric data augmentation operations such as scaling, cropping, flipping or affine transformation, the Gaussian heat map truth matrix and the adaptive Gaussian density map truth matrix are generated based on the true bounding box coordinates after the corresponding geometric transformation, so as to ensure that the spatial prior supervision signal and the augmented image of the actual input network are consistent in spatial position. The real bounding box annotation data includes the coordinates of the two-dimensional center point of the real target and the width and height of the bounding box. The joint training module maps the real bounding box to a shallow high-resolution feature scale and constructs a two-dimensional Gaussian response at the center position of each real target. The standard deviation of the central Gaussian kernel in the horizontal and vertical directions is determined according to the width and height of the corresponding bounding box, respectively. In one embodiment, it is set to the maximum value between one-sixth of the corresponding mapped width and one, and the maximum value between one-sixth of the corresponding mapped height and one, and the maximum response 1 is obtained at the center position of the target.

[0086] When the central Gaussian responses of multiple real targets overlap spatially, they are merged using the element-wise maximum value method to obtain the true value of the Gaussian heatmap at the center point. For the true value of density, a two-dimensional Gaussian density kernel with a wider scale is constructed at the center position of each real target. In one embodiment, it is set to the maximum value between half and half of the corresponding mapping width and the maximum value between half and half of the corresponding mapping height. Unit mass normalization is performed on the density Gaussian kernel corresponding to each target within the effective spatial region.

[0087] The true value of the adaptive Gaussian density map is obtained by summing the elements of the Gaussian density kernel corresponding to all real targets, so that the integral over the entire spatial region corresponds to the number of real targets. The expressions for the true value of the Gaussian heatmap at the center point and the true value of the adaptive Gaussian density map are as follows: ; ; In the formula: The true value of the Gaussian heatmap at the center point in continuous space. The value at that location; These are the spatial pixel coordinates at the feature level; To perform an element-wise maximum value operation on the response values ​​of multiple real targets at the same point; This is the index number of the real target in the image; For the first The central two-dimensional Gaussian kernel corresponding to each real target is in coordinates The response value at the location; For the true value of the adaptive Gaussian density map at a point in space The value at that location; To perform an element-wise summation operation on the density response of all real targets; For the first The broad-scale Gaussian density kernel corresponding to each real target in coordinates The response value at the location; These are the integral pixel coordinate variables within the effective spatial region; The set of effective spatial regions based on the fundamental feature scale; In the effective space area The summation and integration operations within the unit mass are used to normalize the unit mass. To prevent extremely small constants with a denominator of zero.

[0088] S520, the joint training module supervises the base-scale center feature map using the ground truth of the center point Gaussian heatmap, and the center spatial error loss adopts the focus loss form oriented towards the center peak response; it also supervises the base-scale density feature map using the ground truth of the adaptive Gaussian density map, and the density spatial error loss adopts the Smooth-L1 form that balances the foreground and background. In one embodiment, the Smooth-L1 transition parameter... Set to 0.01, and the background item weight is set to 0.1; simultaneously, according to The difference between the spatial integral and the actual number of targets constitutes a density-quality consistency loss.

[0089] The expression for density-mass consistency loss is as follows: ; In the formula, This represents the value of density-mass consistency loss. This refers to a traversal and summation operation on a complete two-dimensional grid of an image. These are the discrete grid indices for the horizontal and vertical coordinates of the feature map, respectively. Basic scale density feature map in discrete coordinates The predicted value at that location; This represents the total number of truly labeled targets in the current training sample images.

[0090] For the candidate prediction branches used to generate candidate classification quality and candidate reference boxes, as well as the target category and bounding box prediction results output by the final detection decoder, a matching relationship is established between the prediction results and the real target based on the classification cost and bounding box position cost during the training phase, and the relevant prediction branches are supervised according to the classification loss and bounding box regression loss of the detection network; among them, the one-to-one correspondence between the prediction query and the real target is determined by the bipartite graph matching method.

[0091] The joint training module weights and sums the target classification loss, target regression loss, center space error loss, density space error loss, and density-quality consistency loss according to preset weights to construct a global multi-task joint loss function. Gradient calculation and backpropagation are then performed based on this joint loss function to update the trainable network parameters in the detection system. The expression for the global multi-task joint loss function is as follows: ; In the formula, This is the global multi-task joint loss function; Preset weights for the target classification loss; Classification loss for the target; Preset weights for the target regression loss; For the target regression loss; Preset weights for the center space error loss; For the center space error loss; Preset weights for density spatial error loss; This is for density spatial error loss; Preset weights for density-quality consistency loss; This represents the loss of density and mass consistency.

[0092] To aid in understanding the technical solution of this invention, the following provides an application example of a UAV target detection method based on density and center prior hierarchical guidance.

[0093] This application example uses drone aerial traffic and crowd monitoring during a large-scale urban marathon as the application scenario. The drone aerial images simultaneously contain relatively sparsely distributed race guidance vehicles and locally densely distributed participants who may obscure each other. Due to the influence of the aerial shooting height and shooting angle, some participants occupy a small pixel area in the drone aerial images, and background textures such as roads, urban buildings, zebra crossings, and shadows interfere with the target features.

[0094] The feature extraction module receives drone aerial images containing marathon event scenes input from external devices. The feature extraction module generates a center feature map and a density feature map in a shallow network. The center feature map forms a high response near the target center corresponding to vehicles and participants, and the density feature map forms a relatively large regional integral in densely populated areas and a relatively small regional integral in areas where the target distribution is relatively sparse.

[0095] The feature extraction module performs scale alignment on the center feature map and density feature map respectively, and uses the center prior to modulate the response of the local spatial branch and the density prior to modulate the spatial domain features of the frequency domain branch. Then, it obtains the target distribution related features through frequency domain processing. After multi-branch feature fusion, it outputs the prior-guided multi-scale visual features as well as center prior features and density prior features of different scales, where the basic scale density feature map is used as the query allocation density prior feature.

[0096] The feature encoding module uses scale-aligned center prior features, density prior features, and deep and shallow visual features from adjacent layers to calculate a two-dimensional grid sampling offset field. Based on the two-dimensional grid sampling offset field, the upsampled deep visual features are resampled using a grid based on bilinear interpolation to calibrate the spatial correspondence between deep semantic features and adjacent shallow visual features. Subsequently, the resampled deep visual features are used to correct the original upsampled deep semantic features through residual methods, and then multi-scale fusion is performed with adjacent shallow visual features to output spatially aligned multi-scale high semantic features.

[0097] The query allocation module receives a preset total query quantity parameter, which is set to 300 in this embodiment. First, it determines the local query budget based on the overall integral of the query allocation density prior features. Then, it divides the density prior features into 4×4 grid regions and allocates local query quotas equal to the local query budget based on the density integral ratio of each region and using the Hamiltonian maximum remainder method. Therefore, the number of local queries varies with the overall target demand of the current image, and the remaining query quantity is the difference between the preset total query quantity and the local query quantity, which is used for global quality supplementation.

[0098] The query allocation module constructs a local joint quality based on the classification quality, center response, and density response of each candidate location, and selects local candidate query points within each grid region according to the corresponding quota. Then, the selected local candidates are excluded, and the remaining candidates are supplemented to 300 queries based on the original classification quality. For the finally selected candidate locations, candidate visual features, candidate prior features, and candidate reference boxes are extracted simultaneously. The candidate prior features are then used to perform residual corrections on the query content and reference boxes, respectively, and an initial candidate query set is output.

[0099] The feature decoding module obtains the initial candidate query set. For any two candidate query nodes, it calculates the normalized relative center position feature, log scale ratio feature, intersection-union ratio feature and relative orientation angle according to the corresponding initial reference frame, and forms the geometric feature vector between the query node pair.

[0100] The feature decoding module maps geometric feature vectors into attention bias matrices through a multilayer perceptron and embeds the attention bias matrix into the multi-head scaling dot product self-attention calculation process of the detection decoder. As the reference box is iteratively corrected during the decoding process, the positional relationship prior is reconstructed based on the updated reference box, so that the attention calculation between candidate query nodes is guided by the spatial geometric relationship of the reference box, thereby enhancing the ability to model query relationships between densely adjacent targets.

[0101] After completing the biased self-attention interaction, the feature decoding module performs multi-scale cross-attention interaction on the updated query features and spatially aligned multi-scale high semantic features. It determines the reference sampling positions in the feature maps of different scales based on the reference box positions corresponding to each candidate query, and predicts the corresponding sampling offset and attention weights based on the query features. It obtains the visual features corresponding to the sampling positions through bilinear interpolation and performs weighted fusion. Subsequently, it obtains the target category probabilities and bounding box coordinate prediction results corresponding to the participants and vehicles through the classification prediction branch and the bounding box regression prediction branch.

[0102] In the network training phase of this embodiment, the joint training module generates ground truth values ​​of Gaussian heatmaps and adaptive Gaussian density maps at shallow high-resolution feature scales based on the bounding box annotations of real targets such as participants and vehicles in the training images, and uses them to supervise the center feature map and density feature map at the base scale, respectively.

[0103] For the center point Gaussian heatmap, the center of the real target is used as the center of the Gaussian response, and the standard deviations of the Gaussian kernels in the horizontal and vertical directions are determined according to the width and height of the real target bounding box at the shallow feature scale. In one embodiment, the standard deviation in the horizontal direction is one-sixth of the width of the bounding box and not less than 1, and the standard deviation in the vertical direction is one-sixth of the height of the bounding box and not less than 1.

[0104] For the adaptive Gaussian density map, the center of the real target is used as the center of the Gaussian density kernel, and the horizontal and vertical standard deviations of the Gaussian density kernel are determined according to the width and height of the corresponding real target bounding box at the shallow feature scale. In one embodiment, half of the width and height of the bounding box, which is not less than 2, is taken respectively, and unit quality normalization is performed on the Gaussian density kernel corresponding to each real target, so that the integral of the density kernel corresponding to a single target is 1.

[0105] The joint training module uses the ground truth Gaussian heatmap of the center point to supervise the base scale center feature map, and uses the ground truth of the adaptive Gaussian density map to supervise the base scale density feature map. It also calculates the density quality consistency loss based on the difference between the spatial integral of the density feature map and the actual number of targets.

[0106] For the candidate prediction branch used to generate candidate classification quality and candidate reference boxes, and the prediction query output by the detection decoder, a matching relationship is established based on the category prediction cost and the bounding box location cost, and the corresponding classification loss and bounding box regression loss are calculated.

[0107] The joint training module weights and sums the target classification loss, target regression loss, center space error loss, density space error loss, and density quality consistency loss to form a global multi-task joint loss function, and updates the trainable network parameters of the detection system based on the joint loss function.

[0108] This invention characterizes the target distribution in different spatial regions by querying and allocating density prior features. First, it determines the local query budget based on the overall density integral, and then dynamically adjusts the number of regional query candidates based on the density integral results of each grid region. This allows limited query resources to be adaptively allocated to densely populated areas according to the actual target distribution. At the same time, it uses global classification quality to supplement the remaining query resources. In addition, it uses classification quality, center response, and density response to form candidate prior features, corrects the candidate query content and reference boxes, and generates attention bias by combining the geometric relationship of the reference boxes. This improves the ability to distinguish between query features corresponding to dense crowds, mutually occluded targets, and small-scale targets. In drone aerial photography scenarios where targets are distributed unevenly at varying heights, such as marathon events, this invention can reduce the number of missed targets in dense areas due to insufficient query resources, reduce the impact of complex backgrounds and the mixing of features of adjacent targets on the detection results, improve the detection accuracy of dense targets and small-scale targets, and enhance the stability of the overall target detection results.

[0109] During network training, this invention supervises the basic scale center feature map by using the ground value of the center point Gaussian heatmap, supervises the basic scale density feature map by using the ground value of the adaptive Gaussian density map and density quality consistency constraints, and incorporates the candidate prediction branch and the classification loss and bounding box regression loss corresponding to the final decoder into the joint training process, so that the center position representation, region density representation, candidate query generation and the final target detection task can be optimized collaboratively in a unified training process. The various losses of this invention gradually decrease and stabilize with training iterations, enabling the network to form center priors and density priors that are adapted to the spatial distribution of targets in UAV aerial images. This improves the model's adaptability to uneven target distribution, locally dense targets, small-scale targets, and complex background scenes, and enhances the overall target detection accuracy of the system.

[0110] While specific embodiments of the present invention have been described above, those skilled in the art should understand that these specific embodiments are merely illustrative. Those skilled in the art can omit, substitute, and modify the details of the above methods and systems in various ways without departing from the principles and essence of the present invention. For example, combining the above method steps to perform substantially the same function and achieve substantially the same result using substantially the same method falls within the scope of the present invention.

Claims

1. A UAV target detection method based on density and center prior hierarchical guidance, characterized in that, include: Acquire aerial images of the drone to be detected and extract multi-scale visual features from the aerial images of the drone to be detected; Based on the multi-scale visual features, center prior features and density prior features are generated. The center prior features and density prior features are used to modulate and fuse the multi-scale visual features to output multi-scale high semantic features. The local query budget and the number of query candidates in each region are determined by the density integral based on the density prior features; local candidate query points are extracted by combining the center response, density response and classification quality of the candidate locations, and after global supplementation, an initial candidate query set containing a preset number of query features and reference boxes is generated. An attention bias matrix is ​​generated based on the geometric relationship between the reference boxes in the initial candidate query set, and the attention bias matrix is ​​embedded into the self-attention mechanism of the detection decoder to update the query features. The updated query features are subjected to cross-attention interaction with the multi-scale high semantic features to output the target detection results.

2. The UAV target detection method based on density and center prior hierarchical guidance according to claim 1, characterized in that, The steps of generating center prior features and density prior features based on the multi-scale visual features, and modulating and fusing the multi-scale visual features using the center prior features and density prior features include: The shallow visual features in the multi-scale visual features are extracted, and a center feature map and a density feature map are generated based on the shallow visual features. The center feature map and the density feature map are scaled respectively to obtain the center prior features and density prior features corresponding to the spatial size of visual features at different scales. The multi-scale visual features are divided into an identity mapping branch, a local spatial branch, and a frequency domain branch according to a preset channel ratio. The local spatial branch is then mapped with spatial attention weights using the central prior features. The density prior features are converted into a density weight matrix, and the spatial domain features of the frequency domain branch are modulated. After being converted to the frequency domain by a fast Fourier transform, the features are restored to the spatial domain features. The modulated local spatial branch and frequency domain branch are then fused with the identity mapping branch.

3. The UAV target detection method based on density and center prior hierarchical guidance according to claim 2, characterized in that, The steps of extracting shallow visual features from the multi-scale visual features and generating a center feature map and a density feature map based on the shallow visual features include: The shallow visual features are mapped through channel mapping and local spatial information extraction to obtain shared prior representation features, which are then input into the center prior prediction branch and the density prior prediction branch, respectively. The outputs of the two branches are activated by activation functions to obtain the center feature map and the non-negative density feature map, respectively. The center feature map is subjected to a scaling transformation that preserves the local peak response, and the density feature map is subjected to a scaling transformation that preserves the regional density distribution to obtain the center prior features and density prior features corresponding to visual features at different scales.

4. The UAV target detection method based on density and center prior hierarchical guidance according to claim 1, characterized in that, The steps of modulating and fusing the multi-scale visual features using the center prior features and density prior features to output multi-scale high semantic features include: Extract the shallow and deep visual features from the multi-scale visual features; perform upsampling interpolation on the deep visual features to obtain upsampled deep visual features; and perform a stitching operation on the center prior feature, the density prior feature, the upsampled deep visual features, and the shallow visual features corresponding to the current feature interaction node in the channel dimension. A convolutional mapping network is used to perform dimensionality reduction mapping on the spliced ​​features. A two-dimensional grid sampling offset field with horizontal and vertical offset channels is generated by constraining the hyperbolic tangent function. Based on this, the deep visual features are resampled and fused to output the multi-scale high semantic features.

5. The UAV target detection method based on density and center prior hierarchical guidance according to claim 4, characterized in that, The step of resampling and fusing the deep visual features accordingly to output the multi-scale high semantic features includes: The standard two-dimensional spatial coordinate matrix is ​​numerically superimposed using the two-dimensional grid sampling offset field. Based on this, the upsampled deep visual features are resampled using bilinear interpolation to obtain spatially aligned deep visual features. Residual calibration features are constructed based on the spatially aligned deep visual features and the upsampled deep visual features before resampling. Weighted fusion is performed using preset or learnable calibration coefficients to obtain calibrated deep visual features. The calibrated deep visual features are then fused with the shallow visual features at multiple scales to output the multi-scale high semantic features.

6. The UAV target detection method based on density and center prior hierarchical guidance according to claim 1, characterized in that, The steps for determining the local query budget and the number of query candidates in each region based on the density integral of the aforementioned density prior features include: Obtain a preset total query count parameter, calculate the overall density integral of the density prior feature, determine the local query budget based on the overall density integral, and limit the local query budget to within the total query count parameter; The density prior features are divided into multiple independent grid regions. The density integral value of each independent grid region is calculated. The floating-point query quota of each independent grid region is calculated based on the proportional relationship between the density integrals of each region and the local query budget. The Hamilton maximum remainder method is used to assign a base quantity based on the integer part of the floating-point query quota, and the remaining quota is allocated in descending order of the decimal part to determine the number of query candidates for each grid region.

7. The UAV target detection method based on density and center prior hierarchical guidance according to claim 6, characterized in that, The step of extracting local candidate query points by combining the center response, density response, and classification quality of candidate locations, and generating an initial candidate query set containing a preset number of query features and reference boxes after global supplementation includes: The classification quality is determined based on the classification prediction results corresponding to each candidate position. Feature sampling is performed on the center prior features and density prior features at the corresponding scale according to the candidate position to obtain the center response and density response, respectively. The classification quality, the center response and the log-compressed density response are combined as features and passed through a prior quality mapping network to obtain the local joint quality. The grid region to which the candidate position belongs is determined according to the spatial coordinates of the candidate position. Within each grid region, the candidates are sorted according to the local joint quality, and the local candidate query points with higher scores are extracted based on the corresponding number of query candidates. After excluding the extracted local candidate query points, the remaining candidate positions are globally sorted according to the classification quality, and candidate query points are added until the preset number specified by the total number of queries is reached; the corresponding content feature vector, prior features and reference boxes are extracted according to the final candidate query points, and the content feature vector and reference boxes are residually corrected using the prior features to generate the initial candidate query set.

8. The UAV target detection method based on density and center prior hierarchical guidance according to claim 1, characterized in that, The step of generating an attention bias matrix based on the geometric relationships between reference boxes within the initialized candidate query set includes: The normalized relative center position, logarithmic scale ratio, intersection-union ratio, and relative orientation angle are calculated for any two reference boxes and concatenated to form a geometric feature vector, which is used to characterize the geometric relationship between the reference boxes. The geometric feature vector is then mapped to a scalar bias value through a nonlinearly activated multilayer perceptron network to generate the attention bias matrix.

9. The UAV target detection method based on density and center prior hierarchical guidance according to claim 1, characterized in that, The attention bias matrix is ​​embedded into the self-attention mechanism of the detection decoder to update the query features; The steps of performing cross-attention interaction between the updated query features and the multi-scale high semantic features include: Generate a query feature matrix, a key feature matrix, and a value feature matrix based on the content feature vectors contained in the initial candidate query set; calculate the dot product of the query feature matrix and the transpose of the key feature matrix; and add the attention bias matrix element by element to the dot product result after dividing by the scaling factor. The superimposed dot product result is input into the normalized exponential activation function to generate the attention weight matrix, and multiplied with the value feature matrix to generate the self-attention updated query feature; the self-attention updated query feature is used as the query input, and multi-scale cross-attention interaction is performed with the multi-scale high semantic feature as the image feature input.

10. The UAV target detection method based on density and center prior hierarchical guidance according to claim 1, characterized in that, The method further includes a model training phase, wherein the model training phase updates the network by constructing a joint loss function based on the error loss between the center feature map and density feature map and the corresponding spatial distribution ground value, combined with classification and regression losses; the steps for constructing the joint loss function include: The Gaussian heatmap truth matrix of the center point is synthesized based on the coordinates of the actual two-dimensional center point of the labeled target; the standard deviation parameter of the Gaussian density kernel is determined based on the actual bounding box size, and an adaptive Gaussian density map truth matrix is ​​generated by unit mass normalization and element-wise summation; the error between the center feature map and the Gaussian heatmap truth value of the center point is calculated to obtain the center space error loss. The error between the density feature map and the true value of the adaptive Gaussian density map is calculated to obtain the density spatial error loss, and the density quality consistency loss is obtained based on the difference between the spatial integral of the density feature map and the true number of targets. The joint loss function used to update the network is constructed by weighting and summing the center space error loss, density space error loss, density quality consistency loss, classification loss, and regression loss according to preset weights.

Images