Unmanned aerial vehicle image target detection method and system based on cross-modal group relationship modeling

Abstract

The application discloses a kind of unmanned aerial vehicle image target detection method and system based on cross-modal group relationship modeling, belong to image processing and artificial intelligence technical field.The visible light image and auxiliary modal image under the same scene are input into double-flow feature extraction network, extract basic level features and perform hierarchical semantic convergence.And utilize double road mode whole sequence unit to carry out statistical calibration and spatial correction to cross-modal aggregated features.Further utilize dynamic group relationship deduction unit to construct dynamic hyperedge prototype, carry out high-order modeling to target group relationship and output high-order relationship enhancement features.Finally, high-order relationship enhancement features are injected into the multi-scale detection feature map of basic detection network, and the target class and position information are output by multiple detection heads.The application can suppress modal differences and complex background interference while improving small target detection accuracy in dense occlusion and low-light scenes, and taking into account model complexity, inference efficiency and engineering deployment feasibility.

Description

Technical Field

[0001] This invention belongs to the field of image processing and artificial intelligence technology, specifically relating to a method and system for UAV image target detection based on cross-modal group relationship modeling. Background Technology

[0002] With the widespread application of unmanned aerial vehicle (UAV) platforms in disaster search and rescue, road inspection, border and coastal defense monitoring, traffic enforcement, and nighttime security, the use of airborne imaging equipment for automated target detection has become an important technical means to improve the level of intelligent perception. Compared with single visible light imaging, multimodal imaging systems formed by visible light and auxiliary modes such as thermal infrared and near-infrared can provide more stable complementary information under low-light, strong obstruction, and complex weather conditions, and therefore have significant application value in UAV target detection.

[0003] However, targets in UAV images are typically small in scale, densely distributed, exhibit significant pose variations, and have complex background textures. Especially from a high-altitude perspective, targets such as vehicles and pedestrians occupy only a limited number of pixels. Existing detection networks are prone to losing detailed features during continuous downsampling, leading to missed detections, false detections, and inaccurate localization of small targets.

[0004] Meanwhile, images of different modalities exhibit significant differences in texture structure, energy distribution, and imaging mechanisms. Furthermore, due to the influence of acquisition perspective, time synchronization errors, and sensor noise, local misalignments may exist between different modalities. Directly stitching multimodal features together or using simple attention mechanisms for fusion can easily introduce statistical mismatch and spatial offset noise, reducing the stability of subsequent detection. Most existing multimodal target detection methods employ pairwise correlation modeling or fully connected attention mechanisms for feature interaction. While these methods can establish certain cross-modal correspondences, they struggle to explicitly describe high-order relationships between group structures such as convoys, crowds, and occluded target clusters. Simultaneously, as the number of feature nodes increases, the computational time and memory overhead rise rapidly, hindering deployment on resource-constrained UAV platforms.

[0005] In summary, current multimodal UAV target detection faces at least the following key technical challenges: First, the cross-modal ordering problem, namely, how to complete the statistical distribution calibration and spatial alignment of different modal features under lightweight conditions, reducing the interference introduced by modal differences; Second, the group relationship modeling and efficient deployment problem, namely, how to explicitly model the group relationships between dense targets, occluded targets, and long-distance semantically related targets without excessively increasing the number of parameters and computational load, thereby improving the detection accuracy and localization stability of small targets in complex scenarios. Summary of the Invention

[0006] To address the aforementioned issues, this invention provides a UAV image target detection method and system based on cross-modal group relationship modeling, which improves the detection accuracy and positioning stability of small targets in complex scenarios.

[0007] The first aspect discloses a method for target detection in UAV images based on cross-modal group relationship modeling, the method comprising:

[0008] Acquire visible light images and auxiliary modal images of the same scene;

[0009] The visible light image and the auxiliary modal image are input into a dual-stream feature extraction network to extract at least three levels of basic hierarchical features. Hierarchical semantic convergence is then performed on the basic hierarchical features of the visible light image and the auxiliary modal image to obtain the corresponding cross-level aggregated features.

[0010] Dual-path modal sorting is performed on the cross-level aggregated features corresponding to the visible light image and the auxiliary modal image to obtain cross-modal features with consistent statistical distribution and spatial alignment.

[0011] Based on the cross-modal features corresponding to the visible light image and the auxiliary modal image, a dynamic group relationship structure is constructed. Through dynamic hyperedge prototype generation, node-to-hyperedge information aggregation, and hyperedge-to-node information diffusion processing, higher-order relationship enhancement features are obtained.

[0012] The higher-order relation enhancement features are injected into the H3, H4, and H5 multi-scale detection feature maps of the basic detection network and fused with the corresponding scale features. The three detection heads are used to perform category prediction and bounding box regression on the fused H3, H4, and H5 multi-scale detection feature maps respectively, and the target detection results are output.

[0013] The second aspect discloses a UAV image target detection system based on cross-modal group relationship modeling, the system comprising:

[0014] The image acquisition unit is used to acquire visible light images and auxiliary modal images of the same scene;

[0015] A cross-level feature aggregation unit is used to input the visible light image and the auxiliary modal image into a dual-stream feature extraction network, extract at least three levels of basic hierarchical features, and perform hierarchical semantic aggregation on the basic hierarchical features of the visible light image and the auxiliary modal image respectively to obtain the corresponding cross-level aggregated features;

[0016] The dual-path modal reordering unit is used to perform dual-path modal reordering processing on the cross-level aggregated features corresponding to the visible light image and the auxiliary modal image, and respectively obtain the corresponding cross-modal features with consistent statistical distribution and spatial alignment.

[0017] The dynamic group relationship inference unit is used to construct dynamic group relationship structures based on the cross-modal features corresponding to the visible light image and the auxiliary modal image, and obtain higher-order relationship enhancement features through dynamic hyperedge prototype generation, node-to-hyperedge information aggregation and hyperedge-to-node information diffusion processing.

[0018] The multi-scale injection detection unit is used to inject the higher-order relation enhancement features into the H3, H4 and H5 multi-scale detection feature maps of the basic detection network, and fuse them with the corresponding scale features. The three detection heads are used to perform category prediction and bounding box regression on the fused H3, H4 and H5 multi-scale detection feature maps respectively, and output the target detection results.

[0019] As can be seen from the above technical solutions, the present invention has the following beneficial effects:

[0020] This invention combines hierarchical semantic convergence with dual-path modal ordering to perform statistical calibration and spatial location correction of cross-modal features before high-order inference, effectively mitigating modal differences and local misalignment issues and improving the stability of subsequent feature fusion. Furthermore, it explicitly models dense target clusters, long-distance semantically associated targets, and occluded group relationships through dynamic group relationship inference, improving the accuracy and localization stability of small target detection in complex scenes while controlling computational complexity. Finally, by injecting high-order relationship enhancement features into the H3, H4, and H5 multi-scale detection feature maps, it balances small target localization, complex background suppression, and deployment efficiency, demonstrating good accuracy, real-time performance, and generalization ability on public datasets. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the overall architecture of the hierarchical cross-modal group relationship detection network HCGDet provided by the present invention.

[0022] Figure 2 The flowchart of a UAV image target detection method based on cross-modal group relationship modeling provided by the present invention is shown.

[0023] Figure 3 This is a schematic diagram of the structure of the dual-mode sequencing unit (DMRU) of the present invention; wherein, (a) is a schematic diagram of the dual-mode semantic enhancement substructure, and (b) is a schematic diagram of the dual-mode position encoding substructure.

[0024] Figure 4 This is a schematic diagram of the structure of the Dynamic Group Relationship Inference Unit (DGRU) of the present invention; wherein, (a) is a schematic diagram of the overall dynamic group relationship inference process; (b) is a schematic diagram of the cross-modal adaptive hyperedge generator substructure; and (c) is a schematic diagram of the cross-modal adaptive hypergraph convolution process.

[0025] Figure 5This is a bubble chart comparing the performance of the present invention with existing technologies in terms of detection accuracy, model complexity, and inference speed.

[0026] Figure 6 This is a schematic diagram of the framework of a UAV image target detection system based on cross-modal group relationship modeling provided by the present invention. Detailed Implementation

[0027] To make the objectives, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Several embodiments of the present invention are shown in the drawings. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of the present invention will be thorough and complete.

[0028] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] It should be noted that when an element is referred to as being "fixed to" another element, it can be directly on the other element or there may be an intervening element. When an element is considered to be "connected to" another element, it can be directly connected to the other element or there may be an intervening element. The terms "vertical," "horizontal," "left," "right," "up," "down," and similar expressions used herein are for illustrative purposes only and are not intended to indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as limiting the invention.

[0030] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal communication between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances. The term "and / or" as used herein includes any and all combinations of one or more of the related listed items.

[0031] To address the aforementioned technical problems, this invention provides a method and system for UAV image target detection based on cross-modal group relationship modeling. Its core lies in constructing a hierarchical cross-modal group relationship detection network (HCGDet). Figure 1 As shown, the detection network can be divided into four stages: S1 hierarchical semantic convergence, S2 dual-path modal ordering, S3 dynamic group relationship inference, and S4 cross-modal hypergraph feature injection. It works in conjunction with the backbone network, neck network, and detection head of the basic detection network.

[0032] The basic detection network can adopt a YOLOv11 type backbone network, neck network, and detection head structure. Figure 1 The standard convolution, C3k2, SPPF, C2PSA, Concat, upsampling, and detection head are all existing modules or operations within the basic detection framework. Specifically, standard convolution is used for local feature extraction and channel adjustment; C3k2 is used to enhance hierarchical feature representation with a low parameter count; SPPF is used to converge multi-scale spatial context; C2PSA is used to perform channel and spatial recalibration on high-level features; Concat represents feature concatenation along the channel dimension; upsampling is used to restore spatial resolution; and the detection head is used to output class predictions and bounding box regression results at different scales.

[0033] Furthermore, such as Figure 1 As shown in the upper structure, the neck path of the basic detection network performs upsampling and Concat to fuse features from different levels step by step. After fusion, semantic compression and feature integration are completed through C3k2 or standard convolution, thus forming H3, H4 and H5 multi-scale detection feature maps for detection. Among them, H3 is a high-resolution detection feature map, mainly used to improve the detection capability of small and dense targets; H4 is a medium-resolution detection feature map, used to balance localization accuracy and semantic expression; H5 is a low-resolution high-semantic detection feature map, which is more suitable for representing larger targets or targets with more complete semantics.

[0034] In one embodiment, the present invention provides a method for UAV image target detection based on cross-modal group relationship modeling, such as... Figure 2 As shown, the specific steps include:

[0035] S101. Acquire visible light images and auxiliary modal images of the same scene;

[0036] The auxiliary modal image can be a thermal infrared image, a near-infrared image, or other spectral images that are complementary to the visible light image. Furthermore, after acquiring the visible light image and the auxiliary modal image of the scene to be detected, the two modal images need to undergo size normalization, synchronous flipping, synchronous cropping, and synchronous stitching enhancement to ensure that the geometric transformation remains consistent between the two modalities. After preprocessing, the two modal images are input into their respective two-stream feature extraction branches.

[0037] S102. Input the visible light image and the auxiliary modal image into a dual-stream feature extraction network to extract at least three levels of basic hierarchical features, and perform hierarchical semantic convergence on the basic hierarchical features of the visible light image and the auxiliary modal image respectively to obtain the corresponding cross-level aggregated features.

[0038] Specifically, a dual-stream feature extraction network is used to extract the basic features of the visible light branch and the auxiliary mode branch, respectively. For example... Figure 1 As shown, both modal branches can employ a top-down convolutional structure of standard convolution, standard convolution, C3k2, standard convolution, and C3k2 to form three levels of feature outputs: B1, B2, and B3. B1 represents shallow, high-resolution features, preserving rich details of edges, textures, and small objects. B2 represents mid-level transitional features, balancing local structural and semantic information. B3 represents deep semantic features, possessing a larger receptive field and stronger object category representation capabilities. Within the main deep path, SPPF and C2PSA can be applied sequentially to the fused high-level features to expand the receptive field and enhance responses in key regions. Finally, B1, B2, and B3 together form the input foundation for subsequent hierarchical semantic convergence and cross-modal interaction.

[0039] In the hierarchical collaborative fusion framework HCF, basic hierarchical features are subjected to unified scale mapping and cross-hierarchical fusion. Specifically, such as... Figure 1As shown in S1, basic layer features B1, B2, and B3 of the auxiliary modality and the visible light modality are collected respectively. These features from different layers are then aligned to a unified semantic scale using max pooling or adaptive max pooling. Concat concatenation and convolutional compression are then performed separately to obtain aggregated features that balance local details with high-level semantics. This process avoids semantic loss caused by performing modal interactions only at a single scale and provides a richer contextual basis for subsequent cross-modal integrators.

[0040] This step allows us to obtain cross-level aggregated features corresponding to the visible light image and the auxiliary modality image, respectively.

[0041] S103. Perform dual-path modal sorting on the cross-level aggregated features corresponding to the visible light image and the auxiliary modal image to obtain cross-modal features with consistent statistical distribution and spatial alignment.

[0042] The above steps specifically include:

[0043] S1031. Obtain the corresponding cross-modal channel guiding weights based on the cross-level aggregation features of the visible light image and the auxiliary modal image;

[0044] It should be noted that this step utilizes a dual-modal rectification unit (DMRU) to perform statistical calibration and spatial correction on cross-level aggregated features of different modalities. For example... Figure 3 As shown, firstly, global average pooling is performed on the cross-level aggregated features of the two modalities to obtain channel description vectors, and modality-specific statistical weights are learned through mutually independent transformation branches. Subsequently, the modality-specific statistical weights are concatenated and input into the joint calibration layer to obtain cross-modal channel guidance weights. In this way, the visible light texture distribution and the auxiliary modality energy distribution can be characterized separately in different branches, and then an effective mapping relationship is established in the joint calibration.

[0045] Specifically, the following steps are included:

[0046] Global average pooling is performed on the cross-level aggregated features of the visible light image and the auxiliary modality image respectively to obtain the channel description vector;

[0047] By utilizing the transformation branches corresponding to each mode, mode-specific statistical weights are learned to obtain visible light statistical weights and auxiliary mode statistical weights;

[0048] The visible light statistical weights and auxiliary modal statistical weights are spliced ​​together and jointly calibrated to obtain the cross-modal channel guiding weights;

[0049] Specifically, let the cross-level aggregation characteristics of the visible light mode and the infrared mode be respectively... , Indicates the number of feature channels. This represents the height and width of the feature map. For any modality... First, the channel description vector is obtained through global average pooling. The mode-specific statistical weights are then obtained through two independent transformation branches of this mode. The mutually independent transformation branches include global average pooling, first linear mapping, SiLU activation and second linear mapping, as shown in Equation (1):

[0050] (1)

[0051] in, and For learnable mappings that belong only to this modality, For the first linear mapping, For the second linear mapping, Activated by SiLU. This structure corresponds to... Figure 3 In (a), the dashed boxes on the left and right sides, representing "Visible Mode Channel Guidance" and "Infrared Mode Channel Guidance," ultimately yield the visible light statistical weights. and auxiliary modal statistical weights .

[0052] In one embodiment, the step of concatenating the visible light statistical weights and the auxiliary modal statistical weights and performing joint calibration to obtain the corresponding cross-modal channel guiding weights includes:

[0053] The visible light statistical weights and auxiliary modal statistical weights are concatenated by channel, mapped through a joint calibration layer, and normalized using Sigmoid to obtain the corresponding cross-modal channel guiding weights. The joint calibration layer consists of a linear layer, SiLU activation, and a linear layer, used to... 3D concatenation vector mapping back Dimensional channel guiding weight, among which, Indicates the number of feature channels.

[0054] Specifically, and After splicing the channels, input the joint calibration layer. The cross-modal channel guidance weights are obtained, as shown in formulas (2) and (3):

[0055] (2)

[0056] (3)

[0057] in, It can be composed of linear layers, SiLU activation, and linear layers, and is used to... 3D concatenation vector mapping back Channel-guided weighting The Sigmoid function broadcasts the resulting cross-modal channel weights to It then participates in subsequent feature modulation.

[0058] S1032. Extract multi-receptive-field context descriptions from the fused features through depthwise separable convolution, wherein the fused features are obtained by fusing cross-level aggregated features of visible light images and auxiliary modal images;

[0059] S1033. Using a cross-modulation method, the cross-level aggregation features of the second mode are enhanced according to the multi-receptive field context description and the cross-modal channel guidance weight of the first mode, respectively obtaining visible light image enhancement features and auxiliary mode image enhancement features. The first mode and the second mode are both visible light and auxiliary modes, and the first mode and the second mode are different.

[0060] It should be noted that here, "first mode" and "second mode" refer to one of the visible light and auxiliary modes. When the first mode is visible light, the second mode is the auxiliary mode; when the first mode is the auxiliary mode, the second mode is visible light. The step of enhancing the cross-level aggregation features of the second mode based on the multi-receptive field context description and the cross-modal channel guiding weights of the first mode can be understood as enhancing the cross-level aggregation features of the auxiliary mode based on the multi-receptive field context description and the cross-modal channel guiding weights of the visible light, and enhancing the cross-level aggregation features of the visible light based on the multi-receptive field context description and the cross-modal channel guiding weights of the auxiliary mode.

[0061] To further enhance the channel calibration effect, this invention concatenates the cross-level aggregated features of two modes and feeds them into a multi-level depth-separable convolutional link to extract the context description of the multi-receptive field pyramid. Then, a cross-modulation method is used to enhance the features of another mode with the guiding weight of one mode, so that thermally significant regions, near-infrared significant regions or high-texture edges can compensate for each other, thereby reducing the detection degradation caused by single-mode distortion.

[0062] Specifically, Figure 3 In (a), "cross-modal semantic enhancement" corresponds to a multi-level deep separable convolutional link. First, the cross-level aggregated features of the two modalities are concatenated by channel and then... Convolutional compression into fused features Then, it undergoes multiple depthwise separable convolutions to expand the receptive field, and finally average pooling is used to obtain the multi-receptive field context description. During cross-modulation, the visible light features are enhanced using the calibration weights of the auxiliary modes, and the auxiliary mode features are enhanced using the calibration weights of the visible light modes, as shown in formulas (4) and (5):

[0063] (4)

[0064] (5)

[0065] in, This represents visible light image enhancement features. This represents auxiliary modality image enhancement features.

[0066] S1034. Obtain the corresponding spatial correction maps based on the visible light image enhancement features and the auxiliary modal image enhancement features, respectively;

[0067] In one embodiment, the specific steps include:

[0068] The visible light image enhancement features and the auxiliary modality image enhancement features are respectively generated using parallel branches with different convolutional kernel sizes to generate corresponding local spatial attention maps;

[0069] A global spatial attention map is obtained by convolution and modal fusion of the local spatial attention map;

[0070] The local spatial attention map and the global spatial attention map are fused to obtain the spatial correction map.

[0071] After completing the channel statistical calibration, the Dual-Mode Organizing Unit (DMRU) continues to perform spatial position correction. Specifically, parallel branches with different convolutional kernel sizes are set for each modal enhancement feature after channel enhancement to generate a local spatial attention map. At the same time, a global spatial attention map is obtained by fusing with the modality through 1×1 convolution. The local spatial attention map and the global spatial attention map are then fused to obtain a spatial correction map, and the position of another modality feature is corrected in a residual manner, thereby reducing the response drift caused by slight registration deviations, occlusion misalignments, and imaging noise.

[0072] Figure 3 In (b), the two dashed boxes representing "multi-scale location encoding" correspond to the local spatial attention map generation part. Channel enhancement features for each modality. , respectively Convolutional branches and Convolutional branches extract local spatial responses at different scales, and then... Dimensionality reduction convolution Fusion .

[0073] Figure 3 (b) The top two "linear layers" and the middle summing node correspond to the global spatial attention graph generation part. Specifically, and respectively Convolution yields single-channel global descriptions, which are then summed and followed by a sigmoid function to obtain the global spatial attention map. Specifically, as shown in formula (6):

[0074] (6)

[0075] in, express convolution.

[0076] Then and The spatial correction image is obtained by summing the results and passing them through the Sigmoid function. .

[0077] S1035. Based on the spatial correction map of the first mode, the position of the enhanced features of the second mode is corrected in a residual manner to obtain the cross-modal features of the visible light image and the cross-modal features of the auxiliary mode image, respectively.

[0078] In one embodiment, the above steps specifically include:

[0079] The cross-modal features of the visible light image are obtained by multiplying the enhanced features of the visible light image by the spatial correction map of the auxiliary modality image and then adding them to the cross-level aggregated features of the visible light image.

[0080] The cross-modal features of the auxiliary modality image are obtained by multiplying the spatial correction map of the visible light image by the enhanced features of the auxiliary modality image and then adding them to the cross-level aggregated features of the auxiliary modality image.

[0081] Specifically, the cross-level aggregation features of the visible light image and the cross-level aggregation features of the auxiliary modality image are used as the original input features, respectively. Spatial correction is completed using the cross-residual method. That is, the visible light output is obtained by multiplying the visible light channel enhancement feature by the auxiliary modality spatial correction map and then adding it to the original visible light input feature. The auxiliary modality output is obtained by multiplying the auxiliary modality channel enhancement feature by the visible light spatial correction map and then adding it to the original auxiliary modality input feature, as shown in formulas (7) and (8).

[0082] (7)

[0083] (8)

[0084] in, This represents the auxiliary modal space correction diagram. This represents the visible light spatial correction diagram.

[0085] The above method utilizes the position response of another mode for correction while preserving the original local details.

[0086] S104. Based on the cross-modal features corresponding to the visible light image and the auxiliary modal image, a dynamic group relationship structure is constructed respectively. Through dynamic hyperedge prototype generation, node-to-hyperedge information aggregation, and hyperedge-to-node information diffusion processing, a higher-order relationship enhancement feature is obtained.

[0087] Specifically, the Dynamic Group Reasoning Unit (DGRU) is used to model high-order relationships of cross-modal features that have completed their ordering. For example... Figure 4 As shown, dynamic group relationship inference units can be set for auxiliary modes and visible light modes respectively, thereby establishing a cross-modal consistent group topology representation while maintaining mode specificity. Specifically, DGRU employs a dynamic hyperedge prototype rather than a fixed neighborhood graphing strategy.

[0088] In one embodiment, the above steps specifically include:

[0089] S1041. Perform average pooling and max pooling on the cross-modal features of the visible light image and the cross-modal features of the auxiliary modal image respectively to construct a joint context vector;

[0090] S1042. Use the joint context vector to perform offset updates on the global learnable prototype to generate a dynamic hyperedge prototype;

[0091] S1043, Map the corresponding cross-modal feature deformation to A set of nodes, where... Represents an integer greater than 1;

[0092] S1044. The node-hyperedge association matrix is ​​obtained by calculating the similarity between the node query vector and the dynamic hyperedge prototype.

[0093] S1045. Based on the node-hyperedge association matrix, perform information aggregation from nodes to hyperedges and information diffusion from hyperedges to nodes to obtain the corresponding higher-order relationship enhancement features.

[0094] S1046. Based on the enhanced features of the higher-order relationship, the gate weight is predicted, wherein the gate weight is used to characterize the scene having obvious group structure characteristics. The larger the gate weight is when the scene has obvious group structure, the smaller the gate weight is when there are many discrete small targets in the scene.

[0095] S1047. The higher-order relation enhancement features and the input features are adaptively fused through the gating weights to obtain optimized higher-order relation enhancement features, wherein the input features are one of the cross-modal features corresponding to the visible light image and the auxiliary modal image.

[0096] First, average pooling and max pooling are performed on the cross-modal features of the current modality and the cross-modal features of the complementary modality, respectively, to simultaneously obtain background distribution information and significant peak information. These statistics are concatenated to form a joint context vector, and the globally learnable prototype is updated through an offset prediction layer, thereby generating a dynamic hyperedge prototype adapted to the current scene. It should be noted that when the current modality is the visible light modality, the complementary modality is the auxiliary modality. When the current modality is the auxiliary modality, the complementary modality is the visible light modality.

[0097] Figure 4 The “deformation mapping” in (a) is used to transform the features of the input space. Rearrange and project as node features ,in ;exist Figure 4 (b) is also used to project dynamic prototypes and node queries onto the multi-head similarity space, for example... , Therefore, the purpose of this deformation mapping is to perform the tensor morphological transformation and linear feature projection required for hypergraph computation, rather than to perform geometric deformation on the original image.

[0098] Regarding the implementation of "updating the globally learnable prototype through the offset prediction layer", the current modality is first... and complementary modes Average pooling and max pooling are performed separately to obtain statistics representing the background distribution and significant peaks; then the four sets of statistics are concatenated and passed through a projection layer. Forming a joint context vector Finally, the offset prediction layer... Output and with the globally learnable prototype Adding them together yields the dynamic hyperedge prototype. Specifically, as shown in formula (9):

[0099] (9)

[0100] Taking single-frame or single-sample features as an example, if the current modal features and complementary modal features satisfy... Then global average pooling and global max pooling will output respectively. The four sets of statistics are concatenated by channel to obtain the context input vector, as shown in formula (10):

[0101] (10)

[0102] Projection layer Used to from The shape mapping can be represented by compressing the dimension into the contextual latent space as follows: Output joint context vector ,in, For the context vector dimension, the preferred option is... Misalignment prediction layer For regressing all dynamic prototype offsets from the joint context vector, the shape mapping can be represented as: Then reshape to obtain If by batch If achieved, the above shapes are as follows: and .

[0103] Subsequently, the current modal features are flattened into a set of nodes, and a node-hyperedge association matrix is ​​obtained by calculating the similarity between the node query vector and the dynamic hyperedge prototype. Then, information aggregation from nodes to hyperedges is performed according to the association matrix to form group descriptions. These group descriptions are then propagated back from the hyperedges to the nodes, achieving information diffusion from hyperedges to nodes. Through this process, the system can explicitly model group relationships between convoys, dense crowds, overlapping occluded target clusters, and spatially dispersed but semantically consistent targets.

[0104] Figure 4 (b) Represents a node query vector, consisting of a set of nodes. Learnable query mapping The result is shown in formula (11):

[0105] (11)

[0106] in addition, This represents a dynamic hyperedge prototype, used as a small number of learnable and scene-adaptive group relation centers. Node-hyperedge association matrix. pass and The scaling dot product between them is calculated as shown in formula (12):

[0107] (12)

[0108] in, To scale the dimensions, express transpose, Each line represents a pair of spatial nodes. The membership strength of a dynamic hyperedge.

[0109] In obtaining Then, DGRU performs high-order message passing in two phases, such as Figure 4 As shown in (c):

[0110] The first stage is Aggregation, specifically as shown in formula (13):

[0111] (13)

[0112] in, express The transpose of the function aggregates node information belonging to the same dynamic hyperedge into a group description. ;

[0113] The second stage is Diffusion, obtaining the diffused nodes Specifically, as shown in formula (14):

[0114] (14)

[0115] After normalization, SiLU activation, and inverse deformation mapping, the information diffusion from the hyperedge to the node propagates the group description back to the node and restores it as spatial features, resulting in higher-order relation enhancement features. .

[0116] in, The learnable projection matrix representing the node-to-hyperedge aggregation stage can typically be taken as... Used in execution Previous pair node features Perform channel mapping; The learnable projection matrix representing the hyperedge-to-node diffusion stage can typically be taken as... Used in execution Previous group description Perform channel mapping. Both can be implemented as linear layers or... Convolution, its function is to make and Information transmission in both directions possesses independent, learnable transformation capabilities. Consequently, the context of groups such as convoys, crowds, and occluded target clusters is transmitted back to each spatial node.

[0117] Furthermore, the aforementioned information diffusion results are not isolated outputs, but rather serve as inputs for subsequent gated fusion and multi-scale injection. Specifically, higher-order relation enhancement features... Carrying high-order group relation semantics, the final output of DGRU is obtained through gated fusion. , This represents the optimized higher-order relation enhancement feature, i.e., the cross-modal hypergraph feature.

[0118] Since DGRU only needs to handle the interaction between nodes and a small number of dynamic hyperedge prototypes, it can significantly reduce computational and memory overhead compared to traditional fully connected pairwise correlation modeling methods. Meanwhile, to avoid excessive smoothing of small target boundaries due to the propagation of higher-order relations, this invention sets a gated fusion branch at the DGRU output. The gate weights are adaptively predicted by the higher-order relation enhancement features, and the higher-order relation enhancement features are then weighted and fused with the original input features accordingly. When the scene has a clear group structure, the proportion of higher-order relation information is increased; when there are many discrete small targets in the scene, more local details are preserved.

[0119] For the specific implementation of the gated fusion branch, features are enhanced by higher-order relations. Self-predicted position-by-position, channel-by-channel gating weights Specifically, as shown in formula (15):

[0120] (15) Among them, and Both can be lightweight convolution, Activated for SiLU This is the Sigmoid function. The final output is specifically represented as follows:

[0121] ;

[0122] When the group structure is obvious Enlarging to enhance higher-order relational information, when discrete small targets or boundary details are more important. Reduce to preserve the original input features.

[0123] For ease of understanding Figure 3 and Figure 4 The symbols and dimensions are explained below:

[0124] Indicates the number of feature channels. and These represent the height and width of the feature map, respectively. This represents the number of nodes obtained by flattening the spatial feature map and , Indicates the prototype or number of dynamic hyperedges. This represents the number of heads in the multi-head similarity calculation. The feature dimension of each head typically satisfies Or it could be the channel dimension after projection.

[0125] In DGRU, Indicates the current modal input features. Represents complementary modal input features; Indicates by The set of nodes obtained by deformation mapping; Represents the node query vector; or Represents a dynamic hyperedge prototype; Represents the node-hyperedge incidence matrix; This represents a description of a hyperedge or group after node aggregation. Dynamic Hyperedge Prototype The dimension in the channel space is After deformation mapping or linear projection, the dimension is , which represents the same prototype in different computational spaces. Figure 4 Input 1 in (b) corresponds to the complementary mode. Input 2 corresponds to the current mode. ; Both in Figure 4 (a) can be a spatial feature, in Figure 4 (b) becomes a node feature after deformation mapping.

[0126] In the picture This indicates splicing by channel dimension. This indicates element-wise multiplication. This indicates element-wise addition. This represents the Sigmoid normalization operation. Figure 4 The term "deformation mapping" in this context refers to tensor dimension rearrangement and linear projection, for example... , as well as The transformation does not involve physical deformation of the image's geometry.

[0127] S105. Inject the higher-order relation enhancement features into the H3, H4 and H5 multi-scale detection feature maps of the basic detection network, and fuse them with the corresponding scale features. Use the three detection heads to perform category prediction and bounding box regression on the fused H3, H4 and H5 multi-scale detection feature maps respectively, and output the target detection results.

[0128] Specifically, the high-order relation enhancement features output by DGRU are injected as cross-modal hypergraph features. For example... Figure 1 As shown in the path of S4 and its upper feature pyramid, the mid-scale high-order relation enhancement features output by S3 can be used as cross-modal hypergraph feature anchors. By upsampling and / or downsampling, they can be aligned with the H3, H4 and H5 multi-scale detection feature maps in the neck network in terms of spatial resolution. The aligned features are concatenated with the basic branch features at the corresponding scale by Concat, and further fused by standard convolution and / or C3k2 modules so that the high-order group relation information can participate in the construction of the H3, H4 and H5 multi-scale detection feature maps.

[0129] It should be noted that the DGRU output Firstly, it serves as an anchor point for higher-order relations at the mesoscale; if the spatial resolutions of H3, H4, and H5 in the object detection network are different, then... Perform upsampling or downsampling to align it with H3, H4, and H5 respectively. The aligned cross-modal hypergraph features are then concatenated, convolved, or gated with the corresponding scale-based detection features, enabling high-order group relationship information to simultaneously participate in small target localization, medium-scale target recognition, and high-semantic target discrimination.

[0130] After cross-modal hypergraph feature injection, the three multi-scale detection feature maps H3, H4, and H5 are input into three detection heads for class prediction and bounding box regression. By simultaneously setting detection heads for H3, H4, and H5, joint detection of targets at different scales can be achieved. The final output is the class label, confidence score, and location information of one or more targets in the UAV image.

[0131] To further verify the beneficial effects of the method disclosed in this invention, a series of experiments were conducted. All experiments were performed on a unified hardware platform and in a uniform software environment to ensure fairness in the comparison.

[0132] In this invention, mAP50 represents the average accuracy when the Intersection over Union (IoU) threshold is 0.5, and mAP50:95 represents the average accuracy when the IoU threshold ranges from 0.5 to 0.95 with a step size of 0.05; the number of parameters (M) represents the scale of learnable parameters of the model, the computational cost (G) represents the number of floating-point operations required for a single forward inference, FPS represents the number of frames processed per second, and ms represents the inference latency per frame. These metrics are used to measure detection accuracy, model complexity, and real-time deployment performance, respectively.

[0133] Table 1 is used to verify the gain effect of the newly added cross-modal group relationship modeling module of the present invention relative to the basic detection framework; Table 2 is used to compare the number of parameters, computational cost and mAP50 index of the present invention and some representative methods on the DroneVehicle public dataset; Table 3 is used to compare the differences between DMRU and different alignment mechanisms in terms of positioning accuracy and deployment efficiency.

[0134] Validation of the effectiveness of each key module (ablation experiment):

[0135] To verify the effectiveness of each innovative module of this invention, this application starts with a dual-stream lightweight detection baseline based on YOLOv11, while maintaining... Figure 1 Without changing the basic modules such as standard convolution, C3k2, SPPF, C2PSA, Concat, upsampling, and detection head, HCF, DMRU, and DGRU were gradually introduced into the DroneVehicle public dataset, and compared under the same training settings. The results are shown in Table 1.

[0136] Table 1

[0137]

[0138] The experimental results in Table 1 show that:

[0139] (1) After introducing HCF on the basis of the baseline model, mAP50 increased from 82.9% to 84.6%, and mAP50:95 increased from 62.5% to 63.9%, indicating that hierarchical semantic aggregation can effectively enhance the complementarity of multi-scale features.

[0140] (2) After further introducing DMRU on the basis of HCF, mAP50 increased to 85.4% and mAP50:95 increased to 64.5%, indicating that channel statistical calibration and spatial position correction can significantly reduce the noise caused by modal mismatch.

[0141] (3) After introducing DGRU on the basis of HCF, mAP50 reached 85.7% and mAP50:95 reached 64.8%, indicating that dynamic group relationship modeling has obvious advantages for dense targets and occlusion scenes.

[0142] (4) After integrating all modules, the complete model achieves 86.4% mAP50 and 65.9% mAP50:95, with 3.17M parameters and 9.11G computation, proving that there is good synergistic gain between the modules.

[0143] In addition, such as Figure 5 As shown, the performance comparison results between the present invention and the prior art in terms of detection accuracy, model complexity and inference speed show that the method of the present invention achieves better detection results while maintaining a lower number of parameters and computational load, thus further proving the effectiveness and engineering application value of the method in this paper.

[0144] Performance comparison experiment with existing technologies:

[0145] To demonstrate the superiority of this invention over the prior art, this application selects only some representative multimodal or YOLO series related methods from the DroneVehicle public dataset for comparative demonstration. The selected methods include YOLOv11n-B3, YOLOv13n-B3, HyperYOLO-B3, C2DFF-Net, and the N-type and S-type embodiments of this invention. The results are shown in Table 2.

[0146] In Table 2, YOLOv11n-B3, YOLOv13n-B3, HyperYOLO-B3, and C2DFF-Net are representative method names from published literature, belonging to the model or network naming of the comparative method; the N-type embodiment of the present invention represents a lightweight configuration, and the S-type embodiment of the present invention represents a standard configuration that further extends in terms of channel width and network capacity.

[0147] Table 2

[0148]

[0149] As can be seen from Table 2, the N-type embodiment of the present invention achieves an mAP50 of 86.4% with only 3.2M parameters and 9.1G of computation, which is better than the listed YOLO series baselines and HyperYOLO-B3, and exceeds the 85.7% mAP50 of C2DFF-Net, indicating that the present invention still has strong detection capabilities under lightweight conditions.

[0150] Furthermore, the S-type embodiment of the present invention achieves an mAP50 of 87.5% with 11.7M parameters and 32.4G computation, demonstrating that detection accuracy can be further improved after expanding the model capacity. Therefore, the N-type embodiment is more suitable for resource-constrained deployment scenarios, while the S-type embodiment is more suitable for application scenarios with higher accuracy requirements.

[0151] Comparison experiment of alignment method and deployment performance with existing methods:

[0152] To further demonstrate the efficiency advantages of this invention, DMRU was compared with several representative cross-modal alignment mechanisms within the same framework, and the model derivation performance was evaluated. The results are shown in Table 3.

[0153] Table 3

[0154]

[0155] Table 3 shows that the DMRU of this invention achieves 65.9% mAP50:95 and 308 FPS on the DroneVehicle public dataset, which has a high-precision positioning capability comparable to the Transformer-based alignment method, but with a higher inference speed. At the same time, after the model is exported as ONNX and TensorRT, the latency can be further reduced from 12.9 ms to 10.3 ms and 3.41 ms, respectively, indicating that this invention has good engineering deployment adaptability.

[0156] Among them, PyTorch FP32 indicates that inference is performed with 32-bit floating-point precision under the original training framework; ONNXFP32 indicates that the model is exported as an open neural network exchange format and inference is performed with 32-bit floating-point precision; TensorRTFP16 indicates that the model is deployed on the TensorRT inference engine and inference is accelerated with 16-bit floating-point precision.

[0157] In summary, a series of ablation experiments, comparative experiments, and deployment test results strongly demonstrate the beneficial effects claimed by this invention. Through synergistic innovation in key aspects such as feature ordering, group relation modeling, and multi-scale injection, this invention successfully solves the technical problem of balancing accuracy, robustness, and efficiency in UAV multimodal target detection.

[0158] This application also provides a detection system corresponding to the method embodiments described above. Since the system embodiments are basically similar to the method embodiments, the description is relatively simple. For details of the relevant technical features and their effects, please refer to the corresponding descriptions of the method embodiments provided above. This invention discloses a UAV image target detection system based on cross-modal group relationship modeling, such as... Figure 6 As shown, this system includes:

[0159] The image acquisition unit is used to acquire visible light images and auxiliary modal images of the same scene;

[0160] A cross-level feature aggregation unit is used to input the visible light image and the auxiliary modal image into a dual-stream feature extraction network, extract at least three levels of basic hierarchical features, and perform hierarchical semantic aggregation on the basic hierarchical features of the visible light image and the auxiliary modal image respectively to obtain the corresponding cross-level aggregated features;

[0161] The dual-path modal reordering unit is used to perform dual-path modal reordering processing on the cross-level aggregated features corresponding to the visible light image and the auxiliary modal image, and respectively obtain the corresponding cross-modal features with consistent statistical distribution and spatial alignment.

[0162] The dynamic group relationship inference unit is used to construct dynamic group relationship structures based on the cross-modal features corresponding to the visible light image and the auxiliary modal image, and obtain higher-order relationship enhancement features through dynamic hyperedge prototype generation, node-to-hyperedge information aggregation and hyperedge-to-node information diffusion processing.

[0163] The multi-scale injection detection unit is used to inject the higher-order relation enhancement features into the H3, H4 and H5 multi-scale detection feature maps of the basic detection network, and fuse them with the corresponding scale features. The three detection heads are used to perform category prediction and bounding box regression on the fused H3, H4 and H5 multi-scale detection feature maps respectively, and output the target detection results.

[0164] This application also provides an electronic device, which includes a processor and a memory. The memory stores at least one instruction or at least one program, which is loaded and executed by the processor. This is a UAV image target detection method based on cross-modal group relationship modeling provided in the above-described method embodiments.

[0165] Furthermore, the electronic device may participate in or include the apparatus or system provided in the embodiments of this application. The electronic device may include one or more processors (processors may include, but are not limited to, processing devices such as microprocessors (MCUs) or programmable logic devices (FPGAs), memory for storing data, and transmission devices for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera.

[0166] It should be noted that the aforementioned one or more processors and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits can be implemented wholly or partially as software, hardware, firmware, or any other combination. Furthermore, the data processing circuits can be a single, independent processing module, or wholly or partially integrated into any other element within a device (or mobile device). As involved in the embodiments of this application, the data processing circuit serves as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0167] The memory can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method described in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby realizing the above-mentioned data processing method. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to electronic devices via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0168] The transmission device is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the device's communication provider. In one example, the transmission device includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0169] The display can be, for example, a touchscreen liquid crystal display (LCD), which allows users to interact with the user interface of an electronic device (or mobile device).

[0170] This application embodiment also provides a computer storage medium storing at least one instruction or at least one program, which is loaded and executed by a processor to implement the UAV image target detection method based on cross-modal group relationship modeling provided in the above method embodiment.

[0171] Optionally, in this embodiment, the aforementioned computer storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the aforementioned storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0172] This application also provides a computer program product or computer program, which includes computer instructions stored in a computer storage medium. The processor of an electronic device reads the computer instructions from the computer storage medium and executes the computer instructions, causing the electronic device to perform the UAV image target detection method based on cross-modal group relationship modeling provided in the above-described method embodiment.

[0173] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0174] It should be understood that the above description of the preferred embodiments is quite detailed and should not be construed as a limitation on the scope of protection of the present invention. Those skilled in the art can make substitutions or modifications under the guidance of the present invention without departing from the scope of protection of the claims of the present invention, and all such substitutions or modifications fall within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for UAV image target detection based on cross-modal group relationship modeling, characterized in that, The method includes: Acquire visible light images and auxiliary modal images of the same scene; The visible light image and the auxiliary modal image are input into a dual-stream feature extraction network to extract at least three levels of basic hierarchical features. Hierarchical semantic convergence is then performed on the basic hierarchical features of the visible light image and the auxiliary modal image to obtain the corresponding cross-level aggregated features. Dual-path modal sorting is performed on the cross-level aggregated features corresponding to the visible light image and the auxiliary modal image to obtain cross-modal features with consistent statistical distribution and spatial alignment. Based on the cross-modal features corresponding to the visible light image and the auxiliary modal image, a dynamic group relationship structure is constructed. Through dynamic hyperedge prototype generation, node-to-hyperedge information aggregation, and hyperedge-to-node information diffusion processing, higher-order relationship enhancement features are obtained. The higher-order relation enhancement features are injected into the H3, H4, and H5 multi-scale detection feature maps of the basic detection network and fused with the corresponding scale features. The three detection heads are used to perform category prediction and bounding box regression on the fused H3, H4, and H5 multi-scale detection feature maps respectively, and the target detection results are output.

2. The UAV image target detection method based on cross-modal group relationship modeling according to claim 1, characterized in that, The step of performing dual-path modal reordering on the cross-level aggregated features corresponding to the visible light image and the auxiliary modal image to obtain cross-modal features with consistent statistical distribution and spatial alignment includes: The corresponding cross-modal channel guiding weights are obtained based on the cross-modal channel features of the visible light image and the auxiliary modal image; The fusion features are extracted using depthwise separable convolution to describe the multi-receptive field context. The fusion features are obtained by fusing cross-level aggregated features from visible light images and auxiliary modal images. Using a cross-modulation method, the cross-level aggregation features of the second mode are enhanced based on the multi-receptive field context description and the cross-modal channel guidance weight of the first mode, resulting in visible light image enhancement features and auxiliary mode image enhancement features, respectively. The first mode and the second mode are both visible light and auxiliary modes, and the first mode and the second mode are different. The corresponding spatial correction maps are obtained based on the visible light image enhancement features and the auxiliary modal image enhancement features, respectively. Based on the spatial correction map of the first modality, the position of the enhanced features of the second modality is corrected in a residual manner to obtain the cross-modal features of the visible light image and the cross-modal features of the auxiliary modality image, respectively.

3. The UAV image target detection method based on cross-modal group relationship modeling according to claim 2, characterized in that, The step of obtaining the corresponding cross-modal channel guidance weights based on the cross-level aggregation features of the visible light image and the auxiliary modal image includes: Global average pooling is performed on the cross-level aggregated features of the visible light image and the auxiliary modality image respectively to obtain the channel description vector; By utilizing the transformation branches corresponding to each mode, mode-specific statistical weights are learned to obtain visible light statistical weights and auxiliary mode statistical weights; The visible light statistical weights and auxiliary modal statistical weights are spliced ​​together and jointly calibrated to obtain the cross-modal channel guiding weights.

4. The UAV image target detection method based on cross-modal group relationship modeling according to claim 3, characterized in that, The step of concatenating the visible light statistical weights and the auxiliary modal statistical weights and performing joint calibration to obtain the corresponding cross-modal channel guiding weights includes: The visible light statistical weights and auxiliary modal statistical weights are concatenated by channel, mapped through a joint calibration layer, and normalized using Sigmoid to obtain the corresponding cross-modal channel guiding weights. The joint calibration layer consists of a linear layer, SiLU activation, and a linear layer, used to... 3D concatenation vector mapping back Channel-guided weights, among which, Indicates the number of feature channels.

5. The UAV image target detection method based on cross-modal group relationship modeling according to claim 2, characterized in that, The method employs cross-modulation to enhance the cross-level aggregation features of the second modality based on the multi-receptive field context description and the cross-modal channel guidance weights of the first modality, resulting in visible light image enhancement features and auxiliary modality image enhancement features, including: （4） （5） in, This represents visible light image enhancement features. This represents auxiliary modality image enhancement features. This indicates a description of the context of multiple receptive fields. This represents the cross-modal channel guided weights of the auxiliary modal image. This represents the cross-modal channel guided weights of a visible light image. For the Sigmoid function, This indicates element-wise multiplication.

6. The UAV image target detection method based on cross-modal group relationship modeling according to claim 2, characterized in that, The step of obtaining the corresponding spatial correction maps based on the visible light image enhancement features and the auxiliary modal image enhancement features includes: The visible light image enhancement features and the auxiliary modality image enhancement features are respectively generated using parallel branches with different convolution kernel sizes to generate corresponding local spatial attention maps; A global spatial attention map is obtained by convolution and modal fusion of the local spatial attention map; The local spatial attention map and the global spatial attention map are fused to obtain the spatial correction map.

7. The UAV image target detection method based on cross-modal group relationship modeling according to claim 2, characterized in that, The step of performing positional correction on the second modality enhancement features using a residual method based on the spatial correction map of the first modality to obtain the cross-modal features of the visible light image and the cross-modal features of the auxiliary modality image, respectively, includes: The cross-modal features of the visible light image are obtained by multiplying the enhanced features of the visible light image by the spatial correction map of the auxiliary modality image and then adding them to the cross-level aggregated features of the visible light image. The cross-modal features of the auxiliary modality image are obtained by multiplying the spatial correction map of the visible light image by the enhanced features of the auxiliary modality image and then adding them to the cross-level aggregated features of the auxiliary modality image.

8. The UAV image target detection method based on cross-modal group relationship modeling according to claim 1, characterized in that, The dynamic group relationship structure is constructed based on the cross-modal features corresponding to the visible light image and the auxiliary modal image, respectively. Through dynamic hyperedge prototype generation, node-to-hyperedge information aggregation, and hyperedge-to-node information diffusion processing, higher-order relationship enhancement features are obtained, including: Average pooling and max pooling are performed on the cross-modal features of the visible light image and the cross-modal features of the auxiliary modal image, respectively, to construct a joint context vector; The global learnable prototype is offset and updated using the joint context vector to generate a dynamic hyperedge prototype; Map the corresponding cross-modal feature deformation as A set of nodes, where... Represents an integer greater than 1; The node-hyperedge association matrix is ​​obtained by calculating the similarity between the node query vector and the dynamic hyperedge prototype; Based on the node-hyperedge association matrix, information aggregation from nodes to hyperedges and information diffusion from hyperedges to nodes are performed to obtain the corresponding higher-order relationship enhancement features.

9. The UAV image target detection method based on cross-modal group relationship modeling according to claim 8, characterized in that, The method further includes: The gating weight is predicted based on the higher-order relation enhancement features. The gating weight is used to characterize the scene having obvious group structure characteristics. The larger the gating weight is when the scene has obvious group structure, the smaller the gating weight is when there are many discrete small targets in the scene. The higher-order relation enhancement features are adaptively fused with the input features through the gating weights to obtain optimized higher-order relation enhancement features, wherein the input features are one of the cross-modal features corresponding to the visible light image and the auxiliary modal image.

10. A UAV image target detection system based on cross-modal group relationship modeling, characterized in that, The system includes: The image acquisition unit is used to acquire visible light images and auxiliary modal images of the same scene; A cross-level feature aggregation unit is used to input the visible light image and the auxiliary modal image into a dual-stream feature extraction network, extract at least three levels of basic hierarchical features, and perform hierarchical semantic aggregation on the basic hierarchical features of the visible light image and the auxiliary modal image respectively to obtain the corresponding cross-level aggregated features; The dual-path modal reordering unit is used to perform dual-path modal reordering processing on the cross-level aggregated features corresponding to the visible light image and the auxiliary modal image, and respectively obtain the corresponding cross-modal features with consistent statistical distribution and spatial alignment. The dynamic group relationship inference unit is used to construct dynamic group relationship structures based on the cross-modal features corresponding to the visible light image and the auxiliary modal image, and obtain higher-order relationship enhancement features through dynamic hyperedge prototype generation, node-to-hyperedge information aggregation and hyperedge-to-node information diffusion processing. The multi-scale injection detection unit is used to inject the higher-order relation enhancement features into the H3, H4 and H5 multi-scale detection feature maps of the basic detection network, and fuse them with the corresponding scale features. The three detection heads are used to perform category prediction and bounding box regression on the fused H3, H4 and H5 multi-scale detection feature maps respectively, and output the target detection results.

Images