Heterogeneous unmanned security cooperation-oriented perspective invariant target detection method and system

Abstract

The application relates to a perspective-invariant target detection method and system for heterogeneous unmanned cooperative security. The method comprises the following steps: acquiring heterogeneous security images collected by an aerial unmanned platform and a ground sensing platform, and inputting the images into a target detection network after unified preprocessing. The network is constructed based on a real-time end-to-end detection model, semantic denoising is realized through a polarity perception scale internal feature interaction module, cross-scale alignment is completed through a heterogeneous space displacement up-sampling module, and perspective-invariant morphological features are modeled through a multi-branch re-parameterized convolution module. The application can effectively alleviate the scale difference, background interference and morphological distortion problems in air-ground heterogeneous sensing.

Description

Technical Field

[0001] This application relates to the field of image detection technology, and in particular to a view-invariant target detection method and system for heterogeneous unmanned collaborative security. Background Technology

[0002] Heterogeneous unmanned system collaborative perception is a key technology in scenarios such as industrial park inspection, urban emergency response, and public safety monitoring. A collaborative network composed of low-altitude rotary-wing UAVs, fixed ground cameras, inspection robots, or other ground-based sensing nodes can simultaneously acquire large-scale aerial view information and close-up ground detail information, thereby providing multi-source visual evidence for security target localization, abnormal behavior identification, and situational assessment.

[0003] However, in actual deployments, the camera pose, altitude, field of view, and imaging distance of different sensing platforms vary significantly, causing the same target to appear completely different from different perspectives. Taking human targets as an example, they typically appear as a vertical torso structure from a ground-level viewpoint, while from a high-altitude or oblique viewpoint, they may be compressed into a clumped or horizontal structure. Meanwhile, high-altitude targets often occupy only a few pixels, while ground-level views are prone to problems such as dense occlusion and cluttered backgrounds. These scale differences, viewpoint shifts, and morphological distortions cause feature distribution misalignment, significantly reducing the robustness of traditional single-view target detectors.

[0004] Existing real-time target detection methods mainly include one-stage detectors based on convolutional neural networks and end-to-end detectors based on Transformers. Although these methods have achieved good results in conventional natural images or single-view scenarios, they still have shortcomings in heterogeneous collaborative security scenarios involving air and ground: First, commonly used attention mechanisms rely on non-negative constraints or single similarity modeling, making it difficult to retain negative interaction information used to suppress background interference, resulting in the weak response of small targets at high altitudes being easily submerged by complex terrain backgrounds; Second, the receptive field structure of conventional convolution or self-attention operators is relatively fixed, making it difficult to adapt to the asymmetric morphological changes of targets between top-down, tilted, and level-down perspectives; Third, traditional upsampling and feature pyramid fusion usually use bilinear interpolation, nearest neighbor interpolation, or channel stitching, lacking dynamic compensation for spatial offsets and local topological changes, and easily losing target boundary details during cross-scale semantic flow. Summary of the Invention

[0005] Based on this, it is necessary to address the aforementioned technical problems by providing a view-invariant target detection method and system for heterogeneous unmanned collaborative security that can simultaneously perform semantic denoising, scale alignment, and morphological enhancement in a deep feature representation space, enabling heterogeneous unmanned systems to achieve high-precision, robust, and edge-deployable collaborative perception in complex dynamic security environments.

[0006] A view-invariant target detection method for heterogeneous unmanned collaborative security, the method comprising: Acquire raw security images or video frames collected by at least two types of heterogeneous sensing platforms, perform data preprocessing and data augmentation on the raw security images or video frames, and generate training data; the heterogeneous sensing platforms include aerial drone platforms and ground sensing platforms; A target detection network is constructed, which includes a backbone feature extraction network, a hybrid encoder, a cross-scale feature fusion network, a query selection module, and a decoding detection head. The training data is input into the backbone feature extraction network to generate feature maps at least three scales. The feature maps are input into the hybrid encoder, and a polarity-aware intra-scale feature interaction module is set at the intra-scale feature interaction position of the hybrid encoder. The polarity-aware intra-scale feature interaction module is used to perform positive and negative polarity decomposition and differential attention calculation on the current scale features, and output the polarity-denoised intra-scale features. The intra-scale features after polarity denoising are input into the cross-scale feature fusion network. At the upsampling position of the cross-scale feature fusion network, the heterogeneous spatial displacement upsampling module is used to sequentially perform scale expansion, multi-directional spatial displacement and channel fusion on the input deep semantic features, and output scale-aligned high-resolution features. A multi-branch reparameterized convolutional module is embedded in the backbone feature extraction network and / or cross-scale feature fusion network. During the training phase, the multi-branch reparameterized convolutional module uses multiple asymmetric convolutional branches to extract various topological features of the target. During the inference phase, the multiple asymmetric convolutional branches are merged into a single equivalent convolutional operator to extract various topological features of the target. The various topological features are input to the decoding detection head to obtain the detection result. The target detection network is trained using the data to be trained, and the trained target detection network is obtained. The heterogeneous security images to be detected are input into the trained target detection network, and the target category, target location and confidence score are output.

[0007] A view-invariant target detection system for heterogeneous unmanned collaborative security, the system comprising: The data acquisition and preprocessing module is used to acquire raw security images or video frames collected by at least two types of heterogeneous sensing platforms, perform data preprocessing and data augmentation on the raw security images or video frames, and generate training data; the heterogeneous sensing platforms include aerial drone platforms and ground sensing platforms; The backbone feature extraction module is used to construct the target detection network, which includes a backbone feature extraction network, a hybrid encoder, a cross-scale feature fusion network, a query selection module, and a decoding detection head. The training data is input into the backbone feature extraction network to generate feature maps at least three scales. The polarity-aware intra-scale feature interaction module is used to input the feature map into the hybrid encoder and set the polarity-aware intra-scale feature interaction module at the intra-scale feature interaction position of the hybrid encoder. The polarity-aware intra-scale feature interaction module is used to perform positive and negative polarity decomposition and differential attention calculation on the current scale features and output the polarity-denoised intra-scale features. The heterogeneous spatial offset upsampling module is used to input the polarity-denoised intra-scale features into the cross-scale feature fusion network, and at the position where the cross-scale feature fusion network performs upsampling, the heterogeneous spatial offset upsampling module is used to sequentially perform scale expansion, multi-directional spatial displacement and channel fusion on the input deep semantic features, and output scale-aligned high-resolution features. A multi-branch reparameterized convolution module is used to embed a multi-branch reparameterized convolution module in a backbone feature extraction network and / or a cross-scale feature fusion network. During the training phase, the multi-branch reparameterized convolution module uses multiple asymmetric convolution branches to extract various topological features of the target. During the inference phase, the multiple asymmetric convolution branches are merged into a single equivalent convolution operator to extract various topological features of the target. The various topological features are input to the decoding detection head to obtain the detection result. The detection output module is used to train the target detection network using the data to be trained, and obtain the trained target detection network; it inputs the heterogeneous security image to be detected into the trained target detection network and outputs the target category, target location and confidence score.

[0008] The aforementioned view-invariant target detection method and system for heterogeneous unmanned collaborative security addresses the issue of traditional attention mechanisms relying solely on non-negative constraints and failing to retain negative background suppression interaction information by setting up a polarity-aware feature interaction module at the feature interaction position within the hybrid encoder. Employing positive and negative polarity decomposition and differential attention computation, the query and key features are decomposed into positive and negative polarity components. Differential operations between same-polarity semantic enhancement terms and opposite-polarity interference suppression terms solve the problem. This achieves effective suppression of complex terrain, building shadows, and densely occluded backgrounds, significantly enhancing the semantic response of small high-altitude targets. Furthermore, a heterogeneous spatial displacement upsampling module is used at the upsampling position of the cross-scale feature fusion network. This module performs scale expansion, multi-directional spatial displacement, and channel fusion, explicitly modeling pixel-level topological shifts caused by viewpoint transformation through channel splitting and multi-directional cyclic displacement. This solves the problem of traditional linear upsampling lacking dynamic compensation for spatial shifts and easily losing target boundary details without significantly increasing the number of parameters. This paper achieves high-resolution feature reconstruction while preserving the boundary details of small targets at high altitudes. By embedding multi-branch reparameterized convolutional modules into the backbone feature extraction network and / or cross-scale feature fusion network, it employs a technique of extracting multiple topological features through multiple asymmetric convolutional branches during the training phase and merging them into a single convolutional operator during the inference phase. Simultaneously, it learns horizontal and vertical morphological topology, improving the recognition ability of high-risk behaviors such as falling and running, as well as targets with cross-view morphological distortions, with almost no increase in edge-side inference overhead. Based on this, this application does not rely on explicit 3D reconstruction, complex sensor calibration, or adversarial domain adaptation training, solving the technical problems of high deployment complexity and poor adaptability of heterogeneous unmanned systems. It can be directly adapted to heterogeneous collaborative security systems composed of UAVs, ground cameras, mobile robots, and edge computing nodes. Finally, in large-scale heterogeneous security detection benchmarks and dynamic behavior detection scenarios, it achieves a significant improvement in detection accuracy and a substantial reduction in performance gaps between different viewpoints with lower parameter and computational loads, demonstrating good real-time application value. Attached Figure Description

[0009] Figure 1 This is a flowchart illustrating a view-invariant target detection method for heterogeneous unmanned collaborative security in one embodiment; Figure 2 This is a schematic diagram of the target detection network in one embodiment; Figure 3 This is a schematic diagram of the feature interaction module within the polarity perception scale in one embodiment; Figure 4 This is a schematic diagram of the heterogeneous spatial displacement upsampling module in another embodiment; Figure 5 This is a schematic diagram of the structure of a space-channel displacement hybrid unit in one embodiment; Figure 6 This is a schematic diagram illustrating the conversion of a multi-branch reparameterized convolutional module from a training structure to an inference structure in one embodiment. Figure 7 This is a schematic diagram of the structure of a view-invariant target detection system for heterogeneous unmanned collaborative security in one embodiment. Detailed Implementation

[0010] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0011] In one embodiment, such as Figure 1 As shown, a view-invariant target detection method for heterogeneous unmanned collaborative security is provided, including the following steps: Step 1: Acquire raw security images or video frames collected by at least two types of heterogeneous sensing platforms, perform data preprocessing and data augmentation on the raw security images or video frames, and generate training data; the heterogeneous sensing platforms include aerial drone platforms and ground sensing platforms.

[0012] Aerial drone platforms collect images from top-down or oblique perspectives, while ground-based sensing platforms collect images from a level-on perspective. Data collected from different platforms undergoes a unified category mapping: pedestrian-related categories such as "pedestrian" and "person" are mapped to "person" categories; vehicle-related categories such as "car," "bus," "truck," and "van" are mapped to "vehicle" categories; and security behaviors are mapped to at least one risk level among normal, suspicious, and high-risk. Specifically, behaviors such as walking, standing, sitting, reading, drinking, and calling are mapped to the normal category; behaviors such as running, pushing, handshaking, and hugging are mapped to the suspicious category; and behaviors such as falling down or lying are mapped to the high-risk category.

[0013] The input image is uniformly scaled, for example, to 640×640 pixels, while preserving the bounding box, object category, and behavior label. During the training phase, techniques such as random flipping, random cropping, color perturbation, and multi-scale training can be used to enhance the model's adaptability to different environments.

[0014] Let the input security image be After uniform-scale preprocessing, the following results were obtained:

[0015] in, and These represent the uniform height and width of the input detection network, respectively. Simultaneously, category labels from different data sources are uniformly mapped to a preset category set. ,in, This indicates the total number of target categories. For example, labels representing people in different datasets can be uniformly mapped to the "personnel" category, and labels representing different types of vehicles can be uniformly mapped to the "vehicle" category, or further subdivided into categories such as cars, trucks, and construction vehicles according to actual applications.

[0016] For behavioral risk levels, the original risk labels or scenario rule labels are mapped to:

[0017] in, Indicates the number of risk levels. For example, The risk levels can be categorized as low, medium, and high, respectively. Through the preprocessing described above, multi-source data from UAVs and ground cameras can have standardized input dimensions, bounding box formats, category labels, and risk level labels, providing standardized data for subsequent model training and inference.

[0018] Step 2: Construct the target detection network, which includes a backbone feature extraction network, a hybrid encoder, a cross-scale feature fusion network, a query selection module, and a decoding detection head.

[0019] The structure of the object detection network is as follows Figure 2 As shown, the backbone feature extraction network is a residual network, used to output three scale feature maps, including at least shallow-scale feature S3, mid-scale feature S4, and deep-scale feature S5; the hybrid encoder is used to perform global context modeling on the feature maps; the cross-scale feature fusion network is used to perform top-down and / or bottom-up fusion of features at different scales; the query selection module is used to select a detection query based on the intersection-union ratio (IU) perception score of the candidate features; and the decoding detection head is used to output detection boxes and class probabilities based on the detection query.

[0020] Preprocessed image Inputting the backbone network yields feature maps at different scales:

[0021] in, This represents the backbone feature extraction network. These represent shallow, mid-level, and deep-level features, respectively. The following steps will explain in detail the hybrid encoder and the cross-scale feature fusion network.

[0022] Because the resolution and number of channels of the three scale features are different, before performing cross-scale fusion, it is first necessary to... Convolution aligns the channels of three-scale features:

[0023]

[0024]

[0025] in, These represent the three-scale features after channel alignment.

[0026] Step 3: Input the training data into the backbone feature extraction network to generate feature maps at least three scales; input the feature maps into the hybrid encoder, and set a polarity-aware intra-scale feature interaction module at the intra-scale feature interaction position of the hybrid encoder. Use the polarity-aware intra-scale feature interaction module to perform positive and negative polarity decomposition and differential attention calculation on the current scale features, and output the polarity-denoised intra-scale features.

[0027] like Figure 3 As shown, the polarity-aware intra-scale feature interaction module is located at the intra-scale feature interaction position of the hybrid encoder, primarily acting on deep features. Addressing the issue that high-altitude small target signals are easily obscured by complex backgrounds, this module differs from conventional linear attention, which relies solely on non-negative attention weights. Instead, it decomposes the query and key into positive and negative polarity components, calculating semantic enhancement and interference suppression terms using learnable coefficients. Positive relationships between targets enhance the target response, while negative relationships between targets and the background suppress noise responses, resulting in cleaner intra-scale features.

[0028] Specifically, feature interactions within the polarity-perceived scale only act on Assume its characteristics are:

[0029] in, Indicates the number of channels. and These represent the height and width of the feature map, respectively. First, the feature map is flattened into a spatial feature sequence. :

[0030] in, Indicates the number of spatial locations. This represents the feature dimension for each spatial location. Then, query features, key features, and value features are generated through a learnable mapping:

[0031] in, , and These represent the learnable mapping matrices for queries, keys, and values, respectively. , and These are used to calculate the correlation between features and aggregate the output features, respectively. To distinguish between the effective response of the target region and the inhibitory response of the background region, the query features and key features are decomposed into positive and negative polarities:

[0032]

[0033] in, and Represents positive polarity feature components, used to characterize positive responses related to the target semantics; and This represents the negative polarity feature component, used to characterize background, noise, or inhibitory responses contrary to the target semantics. After polarity decomposition, the like-polarity interaction weights are calculated:

[0034] in, Indicates the interaction weights of the same polarity. It is a learnable polarity coefficient matrix. Same-polarity interactions are used to enhance the association between features of the same polarity, resulting in stronger semantic consistency within target regions or between similar target regions. Simultaneously, opposite-polarity interaction weights are calculated:

[0035] in, Indicates the interaction weights of opposite polarities. It is a learnable polarity coefficient matrix. Opposite-polarity interactions are used to model the difference between the target and background regions, thereby identifying and suppressing interfering information that contradicts the target semantics. Finally, the Polarity-Aware Intra-Scale Feature Interaction Module (POIFI) calculates the output features through same-polarity semantic enhancement terms and opposite-polarity interference suppression terms:

[0036] This step enhances the semantic response of small target areas from the drone's high-altitude perspective, while suppressing the impact of interfering areas such as road textures, building shadows, and vegetation backgrounds on the detection results.

[0037] Step 4: Input the polarity-denoised intra-scale features into the cross-scale feature fusion network, and use the heterogeneous spatial displacement upsampling module to perform scale expansion, multi-directional spatial displacement and channel fusion on the input deep semantic features at the upsampling position of the cross-scale feature fusion network, and output the scale-aligned high-resolution features.

[0038] like Figure 4 As shown, the heterogeneous spatial displacement upsampling module includes a nearest neighbor upsampling layer, a 3×3 depth-separable convolutional layer, a spatial-channel displacement hybrid unit, and a 1×1 pointwise convolutional layer connected in sequence; as shown... Figure 5 As shown, the spatial-channel displacement hybrid unit reconstructs spatial information through channel splitting and multi-directional cyclic displacement. Addressing the scale difference between the high-altitude and ground-level perspectives, this module replaces ordinary linear upsampling. Input features are first subjected to double nearest-neighbor upsampling and a 3x3 depthwise separable convolution to obtain initial high-resolution features. Subsequently, the features are divided into four groups along the channel dimension, and cyclic displacements are performed in the up, down, left, and right directions for each group. The four groups of features are then concatenated and fused using 1x1 pointwise convolutions, thereby redistributing neighborhood information to the vicinity of the target boundary in a low-parameter manner. This structure enhances the boundary of small high-altitude targets and cross-scale semantic consistency without significantly increasing the number of parameters.

[0039] Step 5: Embed a multi-branch reparameterized convolution module in the backbone feature extraction network and / or cross-scale feature fusion network. During the training phase, the multi-branch reparameterized convolution module uses multiple asymmetric convolution branches to extract various topological features of the target. During the inference phase, the multiple asymmetric convolution branches are merged into a single equivalent convolution operator to extract various topological features of the target. The various topological features are input to the decoding detection head to obtain the detection results.

[0040] like Figure 6 As shown, the multi-branch reparameterized convolution module has a multi-branch structure during the training phase and is equivalent to a single 3×3 convolution operator during the inference phase. This module is embedded in the backbone feature extraction network and / or the cross-scale feature fusion network. During training, the module sets up 3×3, 3×1, 1×3, and 1×1 convolution branches in parallel to learn local general structures, vertical trunk, horizontal edges, and channel recalibration information, respectively. The 3x3 convolution branch extracts the local general structure, the 3x1 convolution branch extracts the vertical trunk and vertical boundaries, the 1x3 convolution branch extracts the horizontal edges and inverted shape, and the 1x1 convolution branch is used to supplement channel recalibration information. Each branch is followed by batch normalization, and the outputs are concatenated and fused using pointwise convolutions. Before inference, the convolution kernels and biases of each branch are merged into an equivalent 3x3 convolution, ensuring that single-branch inference is maintained during the deployment phase.

[0041] During the training phase, the multi-branch reparameterized convolution module extracts features from the input feature X using 3×3, 3×1, 1×3, and 1×1 convolution branches simultaneously. The calculation process is as follows:

[0042] in, This represents the convolution operation. Represents the standard convolution kernel. This represents a vertical morphological convolution kernel, used to capture the vertical topology of elements such as a person's torso from a ground-view perspective. The lateral morphology convolution kernel is used to capture lateral edge features from an aerial viewpoint or in a fallen pose. The 1×1 convolution branch is used for channel mapping and local semantic compensation.

[0043] During the inference phase, the convolutional kernels of each branch are equivalently merged into a single kernel by padding with zeros and adding them together. Convolution kernel: Convolution kernels are equivalently merged into a single kernel by padding with zeros and adding them together. Convolution kernel:

[0044]

[0045] in, , , , These represent the weights of each convolutional branch during the training phase. , , , These represent the biases of each convolutional branch during the training phase. and These represent the equivalent convolution kernel weights and equivalent biases, respectively. Indicates non Zero-padding of convolution kernels Convolution kernel. Therefore, the output of the inference stage can be expressed as:

[0046] in, This represents the output features during the inference phase. Through the aforementioned reparameterization method, the detection network can learn target structural features from multiple perspectives, directions, and morphologies during the training phase, while only needing to perform a single 3×3 convolution operation during the inference phase. This improves the target morphology distortion modeling capability without increasing the computational burden of inference.

[0047] Step 6: Train the target detection network using the data to be trained to obtain the trained target detection network; input the heterogeneous security image to be detected into the trained target detection network, and output the target category, target location and confidence score.

[0048] This embodiment uses the AdamW optimizer to train the detection network. The initial learning rate can be set to 1e-4, the weight decay can be set to 1e-4, the number of training epochs can be set to 300, and the batch size can be set to 16. After training, the multi-branch reparameterized convolutional modules are structurally merged, and the merged network is deployed to an edge computing platform or a central server. During inference, the target image can obtain the bounding box, class probability, and confidence score through a single forward propagation of the network.

[0049] The specific process of forward propagation is as follows: obtaining Next, deep features By performing feature interaction processing within the polarity-aware scale, semantically purified deep features are obtained:

[0050] in, This represents the Polarity-Aware Intra-Scale Feature Interaction Module (POIFI). This step is used to enhance the target semantic response in deep features and suppress interference from complex backgrounds.

[0051] In the top-down fusion stage, the semantically purified deep features are first... Upsampling (HSSU) is performed using a heterogeneous spatial displacement upsampling module to match its spatial dimensions with mid-level features. Maintain consistency, then stitch and fuse with the mid-level features:

[0052] in, This indicates the Heterogeneous Spatial Displacement Upsampling Module (HSSU). The expression represents the fusion convolution operation (MBRConv), and Concat(·) represents the feature concatenation operation. This process is used to pass deep semantic information to mid-level features, while simultaneously using mid-level features to supplement the local structural information of the target.

[0053] Next, the merged middle-layer features Upsampling is performed again using a heterogeneous spatial displacement upsampling module to match its spatial dimensions with shallow features. Maintain consistency and merge with shallow features:

[0054] in, This represents a high-resolution fusion feature that integrates deep semantics, mid-level structure, and shallow spatial details. This feature is primarily used to enhance the detection capabilities of small-scale targets, distant targets, and targets with blurred edges.

[0055] In the bottom-up fusion stage, to further enhance the consistency between features at different scales, high-resolution fused features are... Perform downsampling to integrate its spatial dimensions with the mid-layer features. Maintain consistency and then merge again:

[0056] Where Down(·) represents the downsampling operation, This represents the mid-layer fusion characteristics after bottom-up information backhaul. Furthermore, it will... Perform downsampling to adjust its spatial dimensions relative to deep features. Maintain consistency and integrate:

[0057] in, This represents the final deep fusion feature. Through this process, shallow spatial positioning information can be fed back into deep semantic features, thereby improving the detection head's comprehensive ability to distinguish the target's location and category.

[0058] Ultimately, three fused scale features were obtained:

[0059] in, Used for detecting small-scale targets. Used for detecting mesoscale targets. Used for detecting large-scale or semantically complex targets. This involves fusing feature sets. Input the detector head, output the target class probability and bounding box position:

[0060] in, Indicates the detection head, This represents the final prediction detection result, including the predicted bounding box. Category prediction probability (Probability of Class c), Risk Level Prediction Probability .

[0061] In this embodiment, the model can simultaneously optimize bounding box regression, target category classification, and behavior risk level classification during training. The total loss function is expressed as:

[0062] in, This represents the target bounding box regression loss. This represents the target category classification loss. This indicates the loss categorized by behavioral risk level. and These are the weighting coefficients for category loss and risk level loss, respectively.

[0063] The bounding box regression loss can be an IoU-type loss, for example:

[0064] in, Indicates the prediction box. This represents the ground truth bounding box after preprocessing. The target category classification loss can be expressed as:

[0065] in, Indicates the first The true label of the class, The model predicts the number of... The probability of a class.

[0066] The loss based on behavioral risk level can be expressed as:

[0067] in, Indicates the first A true label for each risk level. The model predicts the number of... The probability of each risk level. This is determined by minimizing the total loss function: Complete model parameters The optimization process involves fusing the multi-branch convolutional structures in the multi-branch reparameterized convolutional module into a single-path convolutional structure after training. The fused model is then deployed to drones, edge computing devices, or security monitoring servers for real-time inference.

[0068] Through the above process, this embodiment can simultaneously utilize the wide-area perception capabilities of the UAV platform and the close-range detail perception capabilities of ground nodes to achieve cross-view semantic cleanup, cross-platform scale alignment, and robust modeling of target morphology distortions within a unified detection network. Compared with traditional target detection methods, the method described in this embodiment can effectively suppress interference from complex backgrounds, enhance the feature representation capabilities of small-scale and distorted targets, improve the detection accuracy of personnel, vehicles, and high-risk behavioral targets in collaborative security scenarios, and maintain high real-time inference efficiency while ensuring detection accuracy, since multi-branch reparameterized convolution modules can be equivalently merged into a single convolution operator during the inference phase.

[0069] In one embodiment, a polarity-aware intra-scale feature interaction module is used to perform positive and negative polarity decomposition and differential attention calculation on the current scale features, outputting polarity-denoised intra-scale features, including: Map input scale features to query features Key features Sum value characteristics The query features and key features are decomposed into positive and negative polarity components using the ELU function. ,in, , , , , and Indicates the positive polarity component. and It represents the negative polarity component.

[0070] Specifically, and Used to characterize positive responses that are related to the target semantics. and It is used to characterize background, noise, or inhibitory responses that are contrary to the target semantics. Through polar decomposition, the model can distinguish between the effective response of the target region and the inhibitory response of the background region, laying the foundation for subsequent differential attention calculation.

[0071] In one embodiment, the output feature is calculated based on the positive and negative polarity components through a difference operation between same-polarity and opposite-polarity interaction terms.

[0072] in, Represents a matrix of coefficients with the same polarity. Represents the heteropolarity coefficient matrix. Indicates value characteristics, Represents the key feature dimension. This represents element-wise multiplication. The superscript indicates the transpose operation.

[0073] Specifically, like-polarity interactions are used to enhance the association between features of the same polarity, resulting in stronger semantic consistency within the target region or between similar target regions; dissimilar-polarity interactions are used to model the difference between the target region and the background region, thereby identifying and suppressing interfering information that contradicts the target's semantics. Through the above difference operations, the model activates semantic enhancement terms in the core target region and strengthens interference suppression terms in complex background regions, thereby improving the separability of targets in complex cross-view backgrounds. The final output features are:

[0074] Through this process, the model can enhance the semantic response of small target areas from the high-altitude perspective of the UAV, while suppressing the influence of interference areas such as road textures, building shadows, and vegetation backgrounds on the detection results.

[0075] In one embodiment, the heterogeneous spatial displacement upsampling module includes a nearest neighbor upsampling layer, a 3×3 depthwise separable convolutional layer, a spatial-channel displacement hybrid unit, and a 1×1 pointwise convolutional layer connected in sequence; the heterogeneous spatial displacement upsampling module is used to sequentially perform scale expansion, multi-directional spatial displacement, and channel fusion on the input deep semantic features, including: Deep semantic features of the input Initial upsampled features are obtained by scaling and local smoothing through nearest neighbor upsampling and 3×3 depthwise separable convolution. ,in , Indicates the number of channels. Indicates altitude, Indicates width; Initial upsampled features The feature is divided into four sub-features along the channel dimension, and then shifted downward, upward, rightward, and leftward respectively. The four sub-features after displacement are concatenated and then fused using a 1×1 pointwise convolution to output high-resolution features. .

[0076] Specifically, double nearest neighbor upsampling is used to quickly expand the feature map size, and 3×3 depthwise separable convolutions are used to perform local context encoding on the initial upsampled features without significantly increasing computational cost, eliminating the block artifacts caused by upsampling. Spatial-channel displacement hybrid units are used to simulate the spatial offset of target edges and local structures under different viewpoints, redistributing neighborhood information to the vicinity of the target boundary. 1×1 pointwise convolutions are used to fuse feature information after displacement in different directions, generating the final high-resolution fused features.

[0077] In one embodiment, the initial upsampled features are divided into four sub-features along the channel dimension, and then shifted downwards, upwards, to the right, and to the left, respectively, including: Will Represented as ,in ; Perform the following for each of the four sub-features: , , , ,in, Indicates the displacement step size. Indicates displacement along the height direction. Indicates displacement along the width direction. This indicates a feature concatenation operation. Indicates a cyclic displacement operation; High-resolution features are .

[0078] Specifically, Indicates the characteristics after downward displacement. This indicates the characteristics after upward displacement. This indicates the characteristics after a rightward displacement. This represents the features after a leftward displacement. Through multi-directional cyclic displacement, the model can explicitly model pixel-level topological shifts caused by viewpoint transitions, effectively mitigating the scale difference between the UAV's viewpoint and the ground's viewpoint, and preserving the boundary details of small high-altitude targets in the high-resolution feature map.

[0079] In one embodiment, the multi-branch reparameterized convolutional module includes at least one 3×3 standard convolutional branch, one 3×1 vertical convolutional branch, one 1×3 horizontal convolutional branch, and one 1×1 point convolutional branch during the training phase. Each convolutional branch is followed by a batch normalization layer. The outputs of each branch are concatenated and then fused through a 1×1 convolution to obtain morphological enhancement features.

[0080] In one embodiment, during the inference phase, the convolution kernels of each convolution branch in the multi-branch reparameterized convolution module are fused with batch normalization parameters. Then, the non-3×3 convolution kernels are zero-padded and expanded according to the 3×3 convolution kernel size. Finally, the weights of all expanded convolution kernels are summed to obtain the equivalent convolution kernel weights, and the biases of all branches are summed to obtain the equivalent biases. This makes the multi-branch reparameterized convolution module equivalent to a single 3×3 convolution operator during the inference phase.

[0081] In one embodiment, an aerial drone platform collects images from a top-down or oblique perspective, while a ground-based sensing platform collects images from a level-on perspective. The data collected by different platforms are then mapped to a unified category, mapping pedestrian-related categories to personnel categories, vehicle-related categories to vehicle categories, and security behaviors to at least one risk level among normal, suspicious, and high-risk.

[0082] Specifically, the above process eliminates the differences in labeling systems between different data sources, enabling the model to learn cross-perspective target features in a unified semantic space. By classifying security behaviors into different risk levels, the model can not only detect the category and location of targets, but also identify abnormal behaviors, meeting the actual needs of security scenarios.

[0083] In one embodiment, the backbone feature extraction network is a residual network used to output three scale feature maps, including at least shallow-scale features S3, mid-scale features S4, and deep-scale features S5; the hybrid encoder is used to perform global context modeling on the feature maps; the cross-scale feature fusion network is used to perform top-down and / or bottom-up fusion of features at different scales; the query selection module is used to select a detection query based on the intersection-union ratio (IU) of the candidate features; and the decoding detection head is used to output detection boxes and class probabilities based on the detection query.

[0084] Specifically, residual networks possess excellent feature extraction capabilities and gradient propagation properties, making them suitable as the backbone of real-time detection networks. Hybrid encoders, through global context modeling, can capture long-range dependencies, improving the understanding of large-scale targets and complex scenes. Cross-scale feature fusion networks, through top-down and bottom-up bidirectional fusion, can effectively combine deep semantic information with shallow spatial detail information, improving the detection accuracy of targets at different scales. The query selection module can filter out the most representative detection queries, reducing computation and improving detection efficiency. The decoding detection head, based on a Transformer decoder, achieves end-to-end detection, avoiding the anchor box design and non-maximum suppression operations found in traditional detectors.

[0085] In a specific embodiment, to verify the effectiveness of this application, a large-scale heterogeneous security detection benchmark was constructed. Drone aerial photography data and ground-based dense crowd data were re-mapped into categories to form a composite test scenario. The training set consisted of approximately 18,500 images, the validation set approximately 4,200 images, and the test set approximately 5,800 images. Furthermore, dynamic security behavior sequence data was used to verify the stability of cross-view behavior recognition.

[0086] On the composite heterogeneous detection benchmark, the number of parameters in this application is approximately 21.5M, the computational cost is approximately 68.2 GFLOPs, the mAP50 index reaches 76.1%, and the mAP50:95 reaches 57.2%. Compared with the basic real-time end-to-end detection network's mAP50 of 73.5%, this invention improves by approximately 2.6 percentage points, while the number of parameters only increases by approximately 1.3M.

[0087] In UAV-view data, this application can improve the detection performance of small target categories such as people, vehicles and motorcycles; in ground-dense crowd data, this application can alleviate the confusion of occlusion boundaries; in dynamic behavior recognition data, this application has higher recognition accuracy for high-risk falling behavior.

[0088] To verify cross-view consistency, tests were conducted under simultaneous dual UAV viewpoints, one at a 45-degree oblique angle and the other at a 90-degree near-top view. The basic detection network showed a mAP50 difference of approximately 4.7 percentage points between the two viewpoints, while this application reduces the difference to approximately 1.4 percentage points, demonstrating that the present invention can improve view-invariant sensing capabilities across heterogeneous platforms.

[0089] Ablation experiments show that adding the polarity-aware scale-interaction module alone can bring about a gain of about 1.2 percentage points in mAP50, adding the heterogeneous spatial displacement upsampling module alone can bring about a gain of about 0.8 percentage points, and adding the multi-branch reparameterized convolution module alone can bring about a gain of about 0.4 percentage points. The highest accuracy is achieved when all three are used together, indicating that the three modules have complementary effects in semantic denoising, scale alignment, and morphological modeling.

[0090] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0091] In one embodiment, such as Figure 7 As shown, a view-invariant target detection system for heterogeneous unmanned collaborative security is provided, including: The data acquisition and preprocessing module 702 is used to acquire raw security images or video frames collected by at least two types of heterogeneous sensing platforms, perform data preprocessing and data augmentation on the raw security images or video frames, and generate training data; the heterogeneous sensing platforms include aerial drone platforms and ground sensing platforms; The backbone feature extraction module 704 is used to construct the target detection network, which includes a backbone feature extraction network, a hybrid encoder, a cross-scale feature fusion network, a query selection module, and a decoding detection head; the training data is input into the backbone feature extraction network to generate feature maps at least three scales. The polarity-aware intra-scale feature interaction module 706 is used to input the feature map into the hybrid encoder and set the polarity-aware intra-scale feature interaction module at the intra-scale feature interaction position of the hybrid encoder. The polarity-aware intra-scale feature interaction module is used to perform positive and negative polarity decomposition and differential attention calculation on the current scale features and output the polarity-denoised intra-scale features. The heterogeneous spatial offset upsampling module 708 is used to input the polarity-denoised intra-scale features into the cross-scale feature fusion network, and at the position where the cross-scale feature fusion network performs upsampling, the heterogeneous spatial offset upsampling module is used to sequentially perform scale expansion, multi-directional spatial displacement and channel fusion on the input deep semantic features, and output scale-aligned high-resolution features. The multi-branch reparameterized convolutional module 710 is used to embed a multi-branch reparameterized convolutional module in a backbone feature extraction network and / or a cross-scale feature fusion network. During the training phase, the multi-branch reparameterized convolutional module uses multiple asymmetric convolutional branches to extract multiple topological features of the target. During the inference phase, the multiple asymmetric convolutional branches are merged into a single equivalent convolutional operator to extract multiple topological features of the target. Multiple topological features are input to the decoding detection head to obtain the detection result. The detection output module 712 is used to train the target detection network using the data to be trained, and obtain the trained target detection network; input the heterogeneous security image to be detected into the trained target detection network, and output the target category, target location and confidence level.

[0092] For specific limitations regarding the view-invariant target detection system for heterogeneous unmanned collaborative security, please refer to the limitations of the view-invariant target detection method for heterogeneous unmanned collaborative security described above, which will not be repeated here. The various modules in the aforementioned view-invariant target detection system for heterogeneous unmanned collaborative security can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0093] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0094] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A view-invariant target detection method for heterogeneous unmanned collaborative security, characterized in that, The method includes: The system acquires raw security images or video frames collected by at least two types of heterogeneous sensing platforms, performs data preprocessing and data augmentation on the raw security images or video frames, and generates training data; the heterogeneous sensing platforms include aerial unmanned aerial vehicle platforms and ground sensing platforms. A target detection network is constructed, which includes a backbone feature extraction network, a hybrid encoder, a cross-scale feature fusion network, a query selection module, and a decoding detection head. The training data is input into the backbone feature extraction network to generate feature maps at least three scales; the feature maps are input into the hybrid encoder, and a polarity-aware intra-scale feature interaction module is set at the intra-scale feature interaction position of the hybrid encoder. The polarity-aware intra-scale feature interaction module is used to perform positive and negative polarity decomposition and differential attention calculation on the current scale features, and output the polarity-denoised intra-scale features. The polarity-denoised intra-scale features are input into the cross-scale feature fusion network, and at the position where the cross-scale feature fusion network performs upsampling, the heterogeneous spatial displacement upsampling module is used to sequentially perform scale expansion, multi-directional spatial displacement and channel fusion on the input deep semantic features, and output scale-aligned high-resolution features. A multi-branch reparameterized convolutional module is embedded in the backbone feature extraction network and / or the cross-scale feature fusion network. During the training phase, the multi-branch reparameterized convolutional module uses multiple asymmetric convolutional branches to extract various topological features of the target. During the inference phase, the multiple asymmetric convolutional branches are merged into a single equivalent convolutional operator to extract various topological features of the target. The various topological features are input to the decoding detection head to obtain the detection result. The target detection network is trained using the training data to obtain the trained target detection network; the heterogeneous security image to be detected is input into the trained target detection network, and the target category, target location and confidence level are output.

2. The method according to claim 1, characterized in that, The polarity-aware intra-scale feature interaction module performs positive and negative polarity decomposition and differential attention calculation on the current scale features, outputting polarity-denoised intra-scale features, including: Map input scale features to query features Key features Sum value characteristics The query features and key features are decomposed into positive and negative polarity components using the ELU function. ,in, , , , , and Indicates the positive polarity component. and It represents the negative polarity component.

3. The method according to claim 2, characterized in that, The method further includes: The output feature is calculated based on the difference operation of the positive polarity component and the negative polarity component through the same polarity interaction term and the opposite polarity interaction term. in, Represents a matrix of coefficients with the same polarity. Represents the heteropolarity coefficient matrix. Indicates value characteristics, Represents the key feature dimension. This represents element-wise multiplication. The superscript indicates the transpose operation.

4. The method according to claim 1, characterized in that, The heterogeneous spatial displacement upsampling module includes a nearest neighbor upsampling layer, a 3×3 depth separable convolutional layer, a spatial-channel displacement hybrid unit, and a 1×1 pointwise convolutional layer connected in sequence. The heterogeneous spatial displacement upsampling module is used to sequentially perform scale expansion, multi-directional spatial displacement, and channel fusion on the input deep semantic features, including: Deep semantic features of the input Initial upsampled features are obtained by scaling and local smoothing through nearest neighbor upsampling and 3×3 depthwise separable convolution. ,in , Indicates the number of channels. Indicates altitude, Indicates width; The initial upsampled features The feature is divided into four sub-features along the channel dimension, and then shifted downward, upward, rightward, and leftward respectively. The four sub-features after displacement are concatenated and then fused using a 1×1 pointwise convolution to output high-resolution features. .

5. The method according to claim 4, characterized in that, The initial upsampled features are divided into four sub-features along the channel dimension, and then shifted downwards, upwards, to the right, and to the left, respectively, including: Will Represented as ,in ; Perform the following for each of the four sub-features: , , , ,in, Indicates the displacement step size. Indicates displacement along the height direction. Indicates displacement along the width direction. This indicates a feature concatenation operation. Indicates a cyclic displacement operation; The high-resolution feature is .

6. The method according to claim 1, characterized in that, The multi-branch reparameterized convolutional module includes at least one 3×3 standard convolutional branch, one 3×1 vertical convolutional branch, one 1×3 horizontal convolutional branch, and one 1×1 point convolutional branch during the training phase. Each convolutional branch is followed by a batch normalization layer. The outputs of each branch are concatenated and then fused through a 1×1 convolution to obtain morphological enhancement features.

7. The method according to claim 6, characterized in that, During the inference phase, the convolution kernels of each convolution branch in the multi-branch reparameterized convolution module are fused with the batch normalization parameters. Then, the non-3×3 convolution kernels are zero-padded and expanded according to the 3×3 convolution kernel size. Finally, the weights of all expanded convolution kernels are summed to obtain the equivalent convolution kernel weights, and the biases of all branches are summed to obtain the equivalent biases. This makes the multi-branch reparameterized convolution module equivalent to a single 3×3 convolution operator during the inference phase.

8. The method according to claim 1, characterized in that, The aerial drone platform collects images from a top-down or oblique perspective, while the ground-based sensing platform collects images from a level perspective. The data collected by different platforms are uniformly categorized, with pedestrian-related categories mapped to personnel categories, vehicle-related categories mapped to vehicle categories, and security behaviors mapped to at least one risk level among normal, suspicious, and high-risk.

9. The method according to claim 1, characterized in that, The backbone feature extraction network is a residual network, used to output three scale feature maps, including at least shallow scale feature S3, mid-scale feature S4, and deep scale feature S5; the hybrid encoder is used to perform global context modeling on the feature maps; the cross-scale feature fusion network is used to fuse features of different scales in a top-down and / or bottom-up manner. The query selection module is used to select a detection query based on the intersection-union ratio (IU) perceived score of the candidate features; the decoding detection head is used to output a detection box and a category probability based on the detection query.

10. A view-invariant target detection system for heterogeneous unmanned collaborative security, characterized in that, The system includes: The data acquisition and preprocessing module is used to acquire raw security images or video frames collected by at least two types of heterogeneous sensing platforms, perform data preprocessing and data augmentation on the raw security images or video frames, and generate training data; the heterogeneous sensing platforms include aerial drone platforms and ground sensing platforms; The backbone feature extraction module is used to construct the target detection network, which includes a backbone feature extraction network, a hybrid encoder, a cross-scale feature fusion network, a query selection module, and a decoding detection head; the training data is input into the backbone feature extraction network to generate feature maps at least three scales; The polarity-aware intra-scale feature interaction module is used to input the feature map into the hybrid encoder, and to set the polarity-aware intra-scale feature interaction module at the intra-scale feature interaction position of the hybrid encoder. The polarity-aware intra-scale feature interaction module is used to perform positive and negative polarity decomposition and differential attention calculation on the current scale features, and output the polarity-denoised intra-scale features. The heterogeneous spatial offset upsampling module is used to input the polar denoised intra-scale features into the cross-scale feature fusion network, and at the position where the cross-scale feature fusion network performs upsampling, the heterogeneous spatial displacement upsampling module is used to sequentially perform scale expansion, multi-directional spatial displacement and channel fusion on the input deep semantic features, and output scale-aligned high-resolution features. A multi-branch reparameterized convolutional module is used to embed a multi-branch reparameterized convolutional module in the backbone feature extraction network and / or the cross-scale feature fusion network. During the training phase, the multi-branch reparameterized convolutional module employs multiple asymmetric convolutional branches to extract various topological features of the target. During the inference phase, the multiple asymmetric convolutional branches are merged into a single equivalent convolutional operator to extract various topological features of the target. These various topological features are input to the decoding detection head to obtain the detection result. The detection output module is used to train the target detection network using the training data to obtain the trained target detection network; input the heterogeneous security image to be detected into the trained target detection network, and output the target category, target location and confidence level.

Images