Unmanned aerial vehicle based target detection method and system

Abstract

The application discloses a target detection method and system based on a UAV, which comprises inputting a visible light image-thermal infrared image into a target detection model, the target detection model comprising an alignment generation module, a context enhancement module and a detection head; the alignment generation module is used to extract visual features of a double mode, global semantic features of the double mode and local spatial features of the double mode, semantic-space mixed prompts are generated based on the global semantic features of the double mode and the local spatial features of the double mode, and after iterative denoising under the guidance of the semantic-space mixed prompts, latent variables obtained by encoding the visual features of the visible light mode are decoded to obtain visible light features aligned with the thermal infrared mode; the context enhancement module is used to fuse the visual features of the thermal infrared mode and the visible light features, and under the guidance of category semantics, semantic perception and enhancement are performed on initial fusion features to obtain reinforced fusion features; and the detection head is used to process the reinforced fusion features to obtain a detection result.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and deep learning technology, specifically to a target detection method and system based on unmanned aerial vehicles (UAVs). Background Technology

[0002] Unmanned aerial vehicle (UAV)-based target detection is widely used in scenarios such as traffic monitoring, disaster relief, and environmental monitoring, and the demand for high-precision identification and positioning under all-weather conditions is increasing. Target detection relying solely on visible light (RGB) images often performs poorly in low-light, nighttime, or severely obstructed conditions. In contrast, thermal infrared (T) images capture the radiant heat of a target, allowing for object detection without external illumination. Therefore, multimodal detection methods fusing visible light and thermal infrared images have attracted widespread attention. However, differences in imaging mechanisms, field of view, and time synchronization often lead to spatial and semantic misalignment between the two modalities, limiting effective multimodal fusion.

[0003] In recent years, researchers have proposed various alignment strategies to address the spatial and semantic misalignment between modalities and achieve effective multimodal feature fusion. For semantic misalignment, the paper "He et al., Multispectral object detection via cross-modal conflict-aware learning. Proceedings of the 31st ACM International Conference on Multimedia, 1465-1474" proposes using cross-modal context alignment to enhance semantic consistency. For spatial misalignment, the paper "Chen et al., Weakly misalignment-free adaptive feature alignment for uavs-based multimodal object detection. Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, 26836-26845" proposes implicit alignment based on feature difference estimation of spatial offset. However, these mainstream alignment methods typically estimate cross-modal geometric transformations or feature offsets. These methods are highly sensitive to field-of-view differences and imaging noise, making it difficult to simultaneously alleviate spatial and semantic misalignment, thus limiting detection performance.

[0004] Furthermore, in drone scenarios, targets are typically small with limited available details, and models often face the problem of semantic ambiguity in target regions. Therefore, in complex scenes, relying solely on the target's own features is insufficient for accurate identification. Incorporating surrounding scene context information allows models to more effectively capture the relationship between the target and its environment, thereby improving detection accuracy. For example, patent application CN120953998A proposes an infrared and visible light image fusion method with enhanced scene guidance and cues. This method also introduces scene cues to assist cross-modal processing, but its essence is to use an attention mechanism to map features of different modalities to a shared embedding space to achieve implicit alignment. This feature mapping-based method often struggles to establish accurate pixel-level correspondences when facing significant parallax and non-rigid distortions common in drone scenarios, resulting in limited alignment performance. Patent application CN120765479A proposes a text-guided semantic perception-based infrared and visible light image fusion method. It uses a semantic modulation module to scale and shift image features at the channel level using text features to enhance the semantic consistency of the fusion result. However, this modulation operation only operates on the statistical distribution level of features and lacks the ability to adjust the spatial geometry of the image. Therefore, it cannot effectively solve the inherent modal spatial misalignment problem in UAV multimodal detection. Summary of the Invention

[0005] The technical problem to be solved by this invention is how to achieve accurate detection of small targets in drone scenarios.

[0006] The present invention solves the above-mentioned technical problems through the following technical means:

[0007] A target detection method based on unmanned aerial vehicles (UAVs) is proposed, the method comprising: A pair of visible light images and thermal infrared images are input into a target detection model, which includes an alignment generation module, a context enhancement module, and a detection head. The alignment generation module extracts the visual features, global semantic features, and local spatial features of the bimodal mode, respectively. Based on the global semantic features and local spatial features of the bimodal mode, a semantic-spatial hybrid cue is generated. The latent variables obtained by encoding the visual features of the visible light mode are iteratively denoised under the guidance of the semantic-spatial hybrid cue and then decoded to obtain the visible light features that are semantically and spatially aligned with the thermal infrared mode. The visual features of the thermal infrared mode are fused with the visible light features using a context enhancement module. Then, under the guidance of category semantic features, the initial fused features are semantically perceived and context-enhanced to obtain enhanced fused features. The enhanced fusion features are processed using the detection head to obtain the target detection results.

[0008] Furthermore, the alignment generation module includes a dual-stream backbone network, an image encoder, a cue building network, and a U-Net network, wherein: A dual-stream backbone network is used to extract features from visible light images and thermal infrared images respectively to obtain dual-modal visual features, which include visual features of the visible light modality and visual features of the thermal infrared modality. The visible light image and the thermal infrared image are encoded separately using an image encoder to obtain the global semantic features and local spatial features of the dual-mode. The global semantic features of the dual-mode include the global semantic features of the visible light mode and the global semantic features of the thermal infrared mode. The local spatial features of the dual-mode include the local spatial features of the visible light mode and the local spatial features of the thermal infrared mode. The semantic cue embedding is obtained by performing semantic attention calculation on the global semantic features of the thermal infrared mode and the global semantic features of the visible light mode using a cue construction network; and by performing semantic attention calculation on the local spatial features of the visible light mode and the local spatial features of the thermal infrared mode to obtain the spatial cue embedding; and the semantic-spatial hybrid cue is constructed based on the semantic cue embedding and the spatial cue embedding. The visual features of the visible light modality are encoded into latent variables using a U-Net network, and the latent variables are iteratively denoised under the conditional guidance of the semantic-spatial hybrid cue. The denoised latent variables are then decoded to obtain the visible light features.

[0009] Furthermore, the cue construction network includes a semantic cue construction network and a spatial cue construction network, wherein: The semantic prompt construction network includes a semantic attention calculation mechanism and a first multilayer perceptron. The semantic attention calculation mechanism performs semantic attention calculation using the global semantic features of the thermal infrared modality as the query and the global semantic features of the visible light modality as the key and value, and obtains the semantic attention calculation result. The semantic attention calculation result is added to the global semantic features of the thermal infrared modality and output to the first multilayer perceptron to generate the semantic prompt embedding. The spatial cue construction network includes a spatial attention calculation mechanism and a second multilayer perceptron. The spatial attention calculation mechanism performs spatial attention calculation using local spatial features of the thermal infrared mode as queries and local spatial features of the visible light mode as keys and values ​​to obtain the spatial attention calculation result. The spatial attention calculation result is then subtracted from the local spatial features of the thermal infrared mode and output to the second multilayer perceptron to generate the spatial cue embedding. The semantic and spatial cues are concatenated and output to a third multilayer perceptron for projection fusion to generate the semantic-spatial hybrid cues.

[0010] Furthermore, the dual-stream backbone network includes a first feature extractor and a second feature extractor with identical structures but independent parameters; The first feature extraction process extracts features from the visible light image to obtain the visual features of the visible light modality; The second feature extractor extracts features from the thermal infrared image to obtain the visual features of the thermal infrared mode.

[0011] Furthermore, the image encoder is a pre-trained CLIP image encoder with frozen parameters.

[0012] Furthermore, the context enhancement module includes a text encoder, a semantic awareness network, and a context aggregation network, wherein: By using a text encoder to encode category names for texts from different scenarios, category semantic features are obtained. The initial fused features are obtained by fusing the visual features of the thermal infrared modality with the visible light features using a semantic perception network. The cosine similarity between the initial fused features and the semantic features of each category is calculated to obtain the response score of each category. The response scores of each category are averaged to generate a comprehensive semantic response map. A context aggregation network is used to perform threshold segmentation on the comprehensive semantic response map to generate a pseudo-target region mask. Based on the introduced expandable learning coefficient, an adaptive expansion operation is performed on the pseudo-target region mask to generate a context-aware mask. The visible light features and the visual features of the thermal infrared modality are weighted and pooled using context-aware masks to obtain context features of the two modalities, which are then fused to obtain a context semantic representation. The initial fusion features within the pseudo-target region are concatenated with the corresponding contextual semantic features. The resulting concatenated features are then fused through convolution, and the channel fusion result is added to the initial fusion features to obtain the enhanced fusion feature.

[0013] Furthermore, the text encoder is a pre-trained CLIP text encoder with frozen parameters.

[0014] Furthermore, before inputting a pair of visible light images and thermal infrared images into the target detection model, the method further includes: The target detection model is trained using an end-to-end joint training strategy. The loss functions used in the model training process include the detection loss of the visible light alignment branch, the semantic response loss, and the detection loss of the final fused features. The formula for the loss function is as follows:

[0015] In the formula, To align the detection loss of visible light features, For semantic response loss, , , These are bounding box regression, classification, and distribution focus losses, respectively, to enhance fusion features. , , They are respectively , , The corresponding weighting coefficients; The detection loss for aligning visible light features is:

[0016] In the formula, , , These are the bounding box regression loss, classification loss, and distribution focus loss for visible light features, respectively. , , They are respectively , , The corresponding weights; The semantic response loss is:

[0017] In the formula, Represents binary cross-entropy loss. A binary mask generated based on the real bounding box. This represents the category response score.

[0018] Furthermore, the detection head is a YOLOv8 detection head.

[0019] Furthermore, this invention also proposes a target detection system based on unmanned aerial vehicles (UAVs), the system comprising: The acquisition unit is used to acquire a pair of visible light images and thermal infrared images; The processing unit is used to process the visible light image-thermal infrared image through a pre-trained target detection model deployed therein to obtain the target detection result; The target detection model includes an alignment generation module, a context enhancement module, and a detection head; The alignment generation module is used to extract the visual features, global semantic features, and local spatial features of the bimodal mode respectively. Based on the global semantic features and local spatial features of the bimodal mode, it generates semantic-spatial hybrid cues. The latent variables obtained by encoding the visual features of the visible light mode are iteratively denoised under the guidance of the semantic-spatial hybrid cues and then decoded to obtain visible light features that are semantically and spatially aligned with the thermal infrared mode. The context enhancement module is used to fuse the visual features of the thermal infrared mode with the visible light features, and under the guidance of the category semantic features, perform semantic perception and context enhancement on the obtained initial fused features to obtain enhanced fused features; The detection head is used to process the enhanced fusion features to obtain the target detection results.

[0020] The advantages of this invention are: (1) This invention generates semantic-spatial hybrid cues based on global semantic features and local spatial features of bimodality. The latent variables obtained by encoding the visual features of the visible light modality are iteratively denoised under the guidance of the semantic-spatial hybrid cues and then decoded to obtain visible light features that are semantically and spatially aligned with the thermal infrared modality. This achieves cross-modal feature alignment in both semantics and space. By modeling the alignment process between the visible light modality and the thermal infrared modality as a feature generation problem constrained by both semantics and space, the semantic association and spatial constraints of the reference modality, i.e. the thermal infrared modality, are injected into the feature generation process. This guides the target modality, i.e. the visible light modality features, to gradually evolve into a unified representation that is consistent with the reference modality in both semantic expression and spatial structure in the latent space. The alignment generation result is not a simple transformation of the original features, but rather forms cross-modal features containing shared high-level semantic information in the latent space of the alignment generation model, i.e., visible light features that are semantically and spatially aligned with the thermal infrared modality. This can effectively remove modality-specific interference and strengthen the cross-modal semantic and structural information that is invariant to the target detection task. Then, the visible light features and thermal infrared visual features, which are aligned semantically and spatially, are fused to obtain the initial fused features. Guided by the prior information of category semantic features, the potential target region in the initial fused features is perceived, and the scene context is adaptively injected to perform context enhancement, resulting in enhanced fused features with semantic information enhancement. This suppresses background interference while maintaining target discrimination features, significantly alleviating the problems of insufficient small target features and semantic ambiguity in UAV scenarios. Therefore, the "first generate alignment, then enhance semantics" processing flow formed by this invention outputs high-quality fused features for small target detection in UAV scenarios.

[0021] (2) By introducing a pre-trained diffusion-generated U-Net network as a generation prior, the semantic association and spatial constraints of the reference modality are injected into the feature generation process, thereby guiding the target modality features to gradually evolve into a unified representation that is consistent with the reference modality in both semantic expression and spatial structure in the latent space.

[0022] (3) This invention guides the initial fusion feature representation from simply relying on local appearance information to a semantic understanding process that combines the relationship between the target and its surrounding scene by utilizing category-level semantic priors; by calculating the category response score between the initial fusion features and the category semantic features, it is possible to perceive potential target regions in the initial fusion image and adaptively determine the range of regions that need to be enhanced; subsequently, based on a learnable context aggregation mechanism, scene context information that is highly related to the target semantics is dynamically selected from the surrounding area of ​​the target, and this context semantics is injected into the target region features, thereby suppressing background interference while maintaining the target discrimination features, significantly alleviating the problem of insufficient small target features and semantic ambiguity in UAV scenarios, and improving the robustness and discrimination ability of feature representation.

[0023] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0024] The accompanying drawings, which form part of this specification, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation thereof. Figure 1 This is a flowchart illustrating a target detection method based on an unmanned aerial vehicle (UAV) according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the network structure of a target detection model in one embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a target detection system based on an unmanned aerial vehicle (UAV) according to an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] like Figures 1 to 2 As shown, the first embodiment of the present invention proposes a target detection method based on unmanned aerial vehicles (UAVs), the method comprising the following steps: S10. Input a pair of visible light images and thermal infrared images into the target detection model, the target detection model including an alignment generation module, a context enhancement module and a detection head; S20. Using the alignment generation module, extract the visual features, global semantic features, and local spatial features of the bimodal mode respectively. Generate semantic-spatial hybrid cue based on the global semantic features and local spatial features of the bimodal mode. Iteratively denoise the latent variables obtained by encoding the visual features of the visible light mode under the guidance of the semantic-spatial hybrid cue, and then decode them to obtain the visible light features that are semantically and spatially aligned with the thermal infrared mode. It should be noted that, unlike the cross-modal feature discriminative alignment mechanism proposed in the patent application document with publication number CN120953998A in the related technology, this discriminative alignment mechanism relies on the CLIP visual encoder supplemented by an adapter structure to achieve regional correspondence between visual patches and text features through a self-attention mechanism, and uses a semantic modulation module to perform affine transformation adjustment on the fused features. This modulation operation only acts on the statistical distribution level of features and lacks the ability to adjust the geometric structure of image space. Therefore, it cannot effectively solve the inherent modal space misalignment problem in UAV multimodal detection. This embodiment is a generative alignment scheme designed to address the stringent positioning accuracy requirements in UAV target detection tasks. By deconstructing cross-modal spatial structural differences and semantic relevance into semantic-spatial hybrid cues, it guides the diffusion model to directly generate visible light features that are highly aligned with the thermal infrared modality in both spatial and semantic dimensions. This generative alignment method can solve the weak alignment problem caused by field-of-view differences and imaging noise from the underlying feature distribution, rather than just making weighted adjustments in the feature fusion stage. The generated alignment features can significantly improve the stability of UAVs in recognizing small and overlapping targets in complex and dynamic scenarios.

[0027] S30. The visual features of the thermal infrared mode are fused with the visible light features using the context enhancement module, and the initial fused features are semantically perceived and context-enhanced under the guidance of category semantic features to obtain enhanced fused features. It should be noted that this embodiment addresses the pain point of targets being subtle and difficult to identify from the perspective of drones. After obtaining the visible light features and thermal infrared modal visual features that are aligned in both semantics and space, it further uses a context enhancement module guided by semantic features to actively locate and aggregate the contextual environment information around the target by utilizing category semantic prior.

[0028] S40. The enhanced fusion features are processed using the detection head to obtain the target detection results.

[0029] It should be noted that the alignment generation module and the context enhancement module work collaboratively in an end-to-end manner. The semantic-spatial guided alignment generation module first aligns the visual features of the visible light modality with the thermal infrared modality, and its output is initially fused with the visual features of the original thermal infrared modality to obtain initial fused features. Subsequently, the semantic-guided context enhancement module immediately performs semantic perception and context enhancement on the initial fused features, outputting the final enhanced fused features for target detection. Therefore, the "first generate alignment, then semantic enhancement" processing flow formed in this embodiment, which outputs high-quality fused features for small target detection in UAV scenarios, differs significantly from the cascaded strategy of "first semantic enhancement, then matching and recognition" used in related technologies, such as the patent application document with publication number CN117058667A. The "first semantic enhancement, then matching and recognition" temporal logic has inherent defects when dealing with multimodal large parallax scenarios. Specifically, because the semantic enhancement operation occurs before spatial alignment, uncalibrated feature deviations are amplified by the network, causing the model to erroneously enhance the semantic saliency of background noise or misaligned regions, which is disastrous for pixel-sensitive small target detection. In contrast, this embodiment adopts a "generate alignment first, then semantic enhancement" strategy, treating semantic information as a generation constraint rather than a simple enhancement feature, and completing the physical placement and reconstruction of features in the latent space first. This sequence ensures that the subsequent context enhancement module runs on a feature map with precisely aligned spatial coordinates, fundamentally avoiding the problems caused by semantic misalignment enhancement, thereby achieving high-precision detection under complex registration errors.

[0030] It should be understood that during the training phase of the object detection model, both visible light features and enhanced fusion features are used as inputs to the detection head to obtain the detection results of the two features, which are then used for loss calculations of the visible light features and enhanced fusion features, respectively. However, during the model inference phase, the object detection result is obtained solely from the enhanced fusion features.

[0031] As a further preferred technical solution, such as Figure 2 As shown, the alignment generation module includes a dual-stream backbone network, an image encoder, a cue building network, and a U-Net network, wherein: S21. Using a dual-stream backbone network, feature extraction is performed on visible light images and thermal infrared images respectively to obtain dual-modal visual features, which include visual features of visible light mode and visual features of thermal infrared mode. Specifically, this embodiment employs a dual-stream backbone network with identical structure but independent parameters, used for extracting visible light images. and thermal infrared images Multi-scale visual features, i.e., visual features of the visible light modality. Visual features of thermal infrared modes .

[0032] S22. The visible light image and the thermal infrared image are encoded using an image encoder to obtain the global semantic features and local spatial features of the dual-mode. The global semantic features of the dual-mode include the global semantic features of the visible light mode and the global semantic features of the thermal infrared mode. The local spatial features of the dual-mode include the local spatial features of the visible light mode and the local spatial features of the thermal infrared mode. Specifically, this embodiment introduces a pre-trained and parameter-frozen CLIP image encoder to extract global semantic features of the two-modal images, i.e., global semantic features of the visible light modality. Global semantic features of thermal infrared modes The local spatial features of the two modal images are the visible light modal features. Local spatial characteristics of thermal infrared modes .

[0033] It should be noted that in related technologies, such as patent applications CN120953998A and CN120765479A, the application of CLIP is mostly limited to using its discriminative properties for semantic feature extraction or simple feature modulation. However, this embodiment redefines the function of the CLIP component: by deconstructing the local spatial features of the CLIP visual branch, it transforms them into spatial cues that drive the diffusion model, enabling CLIP to play a guiding role in generative alignment. This modification allows this embodiment to actively correct geometric displacements across modalities, rather than merely superimposing information at the semantic level, representing a significant technological breakthrough in solving the weak alignment problem in dynamic UAV scenarios.

[0034] S23. Using a cue-based network, semantic attention is calculated on the global semantic features of the thermal infrared mode and the global semantic features of the visible light mode to obtain a semantic cue embedding; and semantic attention is calculated on the local spatial features of the visible light mode and the local spatial features of the thermal infrared mode to obtain a spatial cue embedding; and the semantic-spatial hybrid cue is constructed based on the semantic cue embedding and the spatial cue embedding. S24. The visual features of the visible light modality are encoded into latent variables using a U-Net network, and the latent variables are iteratively denoised under the conditional guidance of the semantic-spatial hybrid cue. The denoised latent variables are then decoded to obtain the visible light features.

[0035] It should be noted that while visible light images often contain richer texture and color details, they are also more susceptible to interference from parallax, blur, and illumination. Therefore, this embodiment utilizes the powerful generative capabilities of the diffusion model to regenerate high-quality and aligned visible light features under the guidance of the spatial structure of the infrared image. In the semantic-spatial dual-guided alignment generation module constructed in this embodiment, the U-Net network directly reconstructs the spatial distribution of features at the pixel level through the diffusion process, thereby achieving precise alignment in physical space. By leveraging the powerful latent space reconstruction capabilities of the diffusion model, the cross-modal spatial structural differences and semantic relevance are deconstructed into semantic-spatial hybrid cues, guiding the diffusion model to directly generate visible light features that are highly aligned with the thermal infrared modality in both spatial and semantic dimensions.

[0036] As a further preferred technical solution, in step S21, the dual-stream backbone network includes a first feature extractor and a second feature extractor with the same structure but independent parameters. The first feature extraction process extracts features from the visible light image to obtain the visual features of the visible light modality; The second feature extractor extracts features from the thermal infrared image to obtain the visual features of the thermal infrared mode.

[0037] As a further preferred technical solution, the prompting construction network includes a semantic prompting construction network and a spatial prompting construction network, wherein: S231. The semantic prompt construction network includes a semantic attention calculation mechanism and a first multilayer perceptron. The semantic attention calculation mechanism performs semantic attention calculation using the global semantic features of the thermal infrared modality as the query and the global semantic features of the visible light modality as the key and value, and obtains the semantic attention calculation result. The semantic attention calculation result is added to the global semantic features of the thermal infrared modality and output to the first multilayer perceptron to generate the semantic prompt embedding. Specifically, this embodiment uses the global semantic features of thermal infrared modes. For querying, global semantic features of the visible light modality. Semantic attention is performed on the key and value pairs to extract cross-modal semantic relevance; then the attention output is compared with the global semantic features of the thermal infrared modality. After addition, a semantic cue embedding is generated through a learnable first-level perceptron mapping. : )) in, Represents a multilayer perceptron mapping. This represents semantic attention computation on two tensors.

[0038] It should be noted that the specific formula for calculating attention is as follows: .

[0039] S232. The spatial cue construction network includes a spatial attention calculation mechanism and a second multilayer perceptron. The spatial attention calculation mechanism performs spatial attention calculation using the local spatial features of the thermal infrared mode as the query and the local spatial features of the visible light mode as the key and value, and obtains the spatial attention calculation result. The spatial attention calculation result is then subtracted from the local spatial features of the thermal infrared mode and output to the second multilayer perceptron to generate the spatial cue embedding. Specifically, this embodiment uses the local spatial features of the thermal infrared mode. For querying, local spatial features of the visible light modality. Spatial attention computation is performed on the key and value pairs to extract the local spatial features of the thermal infrared modes. The spatial attention calculation result is subtracted to extract cross-modal spatial difference features; then, after mean pooling of these spatial difference features, a spatial cue embedding is generated through a second multilayer perceptron. :

[0040] in, This represents the mean pooling operation. This represents spatial attention computation on two tensors.

[0041] S233. The semantic prompt embedding and the spatial prompt embedding are spliced ​​together and output to the third multilayer perceptron for projection fusion to generate the semantic-spatial hybrid prompt.

[0042] Specifically, this embodiment will provide semantic prompts. Spatial prompts The data is stitched together and then projected and fused using a third multilayer perceptron to generate the final semantic-spatial hybrid cue. : .

[0043] As a further preferred technical solution, the U-Net network includes a VAE encoder. Step S24 involves: using the U-Net network to encode the visual features of the visible light modality into latent variables, and iteratively denoising the latent variables under the conditional guidance of the semantic-spatial hybrid cue; then decoding the denoised latent variables to obtain the visible light features. Specifically: First, a VAE encoder is used to extract the visual features of the visible light modality. Encode into the latent space to obtain latent variables ; latent variables With semantic-spatial hybrid cues A U-Net network that shares a single, pre-trained diffusion model with a parameter-frozen input. During the diffusion process, the diffusion model learns to provide hints. Iterative denoising of latent variables under conditional guidance ,in, The representatives passed A parameterized denoising network, where t is the current diffusion time step; Decoding the denoised latent variables yields visible light features that are semantically and spatially aligned with the thermal infrared modes. :

[0044] Among them, Dec This represents a decoding operation.

[0045] It should be noted that this embodiment proposes a generative alignment paradigm based on a diffusion model, which guides the diffusion model through a semantic-spatial hybrid cue. The main purpose of using the thermal infrared mode as the reference mode is to generate a semantic-spatial hybrid cue. This cue guides the diffusion model to act on visible light features, generating visible light features that are semantically and spatially aligned with the reference mode (thermal infrared mode). Specifically, this embodiment models the alignment process between the visible light mode and the thermal infrared mode as a feature generation problem constrained by both semantic and spatial constraints, rather than a traditional geometric correction or feature offset compensation process. By introducing a pre-trained diffusion generative model as a generation prior and constructing a semantic-spatial hybrid cue that includes cross-modal shared semantic information and spatial structural differences, the semantic association and spatial constraints of the reference mode are injected into the feature generation process. This guides the target modality features in the latent space to gradually evolve into a unified representation consistent with the reference mode in both semantic expression and spatial structure. The generated result is not a simple transformation of the original features, but rather forms cross-modal features in the latent space of the generative model that contain shared high-level semantic information. This effectively removes modality-specific interference and strengthens cross-modal semantic and structural information that is invariant to the target detection task.

[0046] As a further preferred technical solution, the context enhancement module includes a text encoder, a semantic awareness network, and a context aggregation network, wherein: S31. Use a text encoder to encode category names for texts in different scenarios to obtain category semantic features; Specifically, this embodiment uses a pre-trained CLIP text encoder with frozen parameters to encode category names, thereby obtaining semantic features for multiple categories. The process of semantic understanding of a target is shifted from relying solely on local appearance information to combining the relationship between the target and its surrounding scene by utilizing category-level semantic priors to guide feature representation.

[0047] S32. Using a semantic perception network, the visual features of the thermal infrared mode are fused with the visible light features to obtain the initial fused features. The cosine similarity between the initial fused features and the semantic features of each category is calculated to obtain the response score of each category. The average response score of each category is used to generate a comprehensive semantic response map. Specifically, this embodiment uses the visible light features generated by the semantic-spatial guided alignment generation module. Visual features of thermal infrared modes extracted by the second feature extractor Perform weighted fusion to obtain initial fusion features. .

[0048] Then calculate the initial fusion features. With each category semantic features The cosine similarity is used to obtain the response score for each category. The average value across categories is then used to generate a comprehensive semantic response map. In the comprehensive semantic response map In the context, regions with high response values ​​indicate strong semantic ambiguity and are more likely to be target regions that require supplementary contextual information.

[0049] It should be noted that in this embodiment, the cosine similarity is obtained by calculating the normalized dot product between the fused features and the category semantic features. Figure 2 The element-wise product before obtaining the category response score is the operation. The cosine similarity between each category and the fused feature is the corresponding category's response score, calculated without transformation.

[0050] This embodiment utilizes a comprehensive semantic response map generated by CLIP semantic priors, combined with ground truth strongly supervised loss, to achieve accurate quantification of semantic ambiguity in target regions. The advantage of this mechanism lies in its ability to proactively identify regions that are difficult for the model to recognize, thereby guiding the injection of adaptive contextual information. This fundamentally solves the problem of missed detection of small targets on UAVs due to insufficient features, demonstrating significant technological advancement.

[0051] S33. Use a context aggregation network to perform threshold segmentation on the comprehensive semantic response map to generate a pseudo target region mask. Based on the introduced expandable learning coefficient, perform adaptive expansion operation on the pseudo target region mask to generate a context-aware mask. It should be noted that in this embodiment, the comprehensive semantic response map is first upsampled to restore the features of the comprehensive response map to the original image size, which is used to calculate the loss with the ground truth. Then, the upsampled result is filtered by a fixed threshold of 0.5 to achieve threshold segmentation and obtain the pseudo target region mask.

[0052] S34. Using context-aware masks, weighted pooling is performed on the visible light features and the visual features of the thermal infrared modality to obtain the context features of the two modalities, and then fused to obtain the context semantic representation. S35. The initial fusion feature in the pseudo-target region is concatenated with the corresponding contextual semantic feature. The concatenated feature is then fused through convolution. The channel fusion result is added to the initial fusion feature to obtain the enhanced fusion feature.

[0053] Compared to full-image convolutional augmentation, the mask-based context augmentation module proposed in this embodiment only performs feature mining in key regions with high semantic response. This precise focusing approach not only improves detection accuracy but also optimizes feature utilization while ensuring algorithm performance, enabling precise allocation of computing resources.

[0054] Specifically, this embodiment uses semantic response graphs. Threshold segmentation is performed to generate a pseudo-target region mask. Introduce a learnable inflation coefficient for... Perform adaptive dilation to generate a context-aware mask. This allows for dynamic adjustment of the receptive field range based on contextual information.

[0055] This embodiment generates a context-aware mask. This approach not only focuses on the pixel features of the target itself but also actively extracts semantic information from the surrounding environment. By re-injecting environmental features into the target features, the model can assist in classification through "logical reasoning." For example, when the front of a truck is obscured, the model can combine the road environment features covered by the mask to infer that the local object is a "truck" rather than a "roadblock," significantly reducing the false negative rate. Moreover, the context mask generated in this embodiment can effectively suppress semantic confusion between different categories by distinguishing between the target and its specific boundary environment, enabling the model to have higher discrimination accuracy when dealing with overlapping and crowded targets, suppressing semantic ambiguity, and improving robustness.

[0056] Next, using Visible light characteristics Visual features of thermal infrared modes Weighted pooling is performed to obtain the context features of the two modalities. and and context features and The final context semantic representation is obtained by fusion. The initial fusion features within the pseudo-target region (referring to the region filtered through the pseudo-target region mask, i.e., the potential region that may contain the target, initially identified based on semantic consistency) Corresponding context features The data is then concatenated, followed by channel fusion using a 1×1 convolution, and the result is combined with the initial fused features. The summation yields the final enhanced fusion feature after semantic enhancement. .

[0057] It should be noted that, in this embodiment, by calculating the similarity response between image features and category semantics, the model can perceive potential target regions and adaptively determine the range of regions that need to be enhanced. Subsequently, based on a learnable context aggregation mechanism, scene context information that is highly related to the target semantics is dynamically selected from the surrounding area of ​​the target, and this context semantics is injected into the target region features. This suppresses background interference while maintaining the target discrimination features, significantly alleviates the problems of insufficient small target features and semantic ambiguity in UAV scenarios, and improves the robustness and discrimination ability of feature representation.

[0058] While related technologies mention scene guidance prompts, they essentially utilize global semantic information to bridge cross-modal feature gaps, representing a general, full-map feature representation enhancement approach. In contrast, the context enhancement module in this embodiment exhibits significant targeting and dynamism: First, it actively locates semantically ambiguous regions through a comprehensive semantic response map, achieving precise delivery of enhancement resources; second, it introduces an adaptive dilation mask mechanism, dynamically capturing the semantic logical association between the target and its specific neighborhood environment through a learnable dilation coefficient. This approach enables this embodiment to achieve accurate identification by leveraging environmental priors when dealing with extremely small targets and severely occluded targets unique to the UAV's perspective. Its robustness in complex dynamic scenes is significantly superior to the global enhancement scheme in Comparative Document 1.

[0059] As a further preferred technical solution, before step S10: inputting a pair of visible light images and thermal infrared images into the target detection model, the method further includes the following steps: The target detection model is trained using an end-to-end joint training strategy. The loss functions used in the model training process include the detection loss of the visible light alignment branch, the semantic response loss, and the detection loss of the final fused features. The formula for the loss function is as follows:

[0060] In the formula, To align the detection loss of visible light features, For semantic response loss, , , These are bounding box regression, classification, and distribution focus losses, respectively, to enhance fusion features. , , They are respectively , , The corresponding weighting coefficients; The detection loss for aligning visible light features is:

[0061] In the formula, , , These are the bounding box regression loss, classification loss, and distribution focus loss for visible light features, respectively. , , They are respectively , , The corresponding weights; The semantic response loss is:

[0062] In the formula, Represents binary cross-entropy loss. A binary mask generated based on the real bounding box. This represents the category response score.

[0063] It should be noted that this embodiment adopts an end-to-end joint training strategy. The parameters of the CLIP image / text encoder and the diffusion model U-Net are frozen throughout the training process. Only the learnable parameters in the dual-stream backbone network, the cueing construction network, the semantic awareness network and context aggregation network in the context enhancement module, and the detection head are trained. During the model inference phase, the frozen pre-trained model (CLIP network and U-Net network) is used for forward computation and no gradient updates are required.

[0064] During training, the training objective for the alignment generation stage is to generate visible light features. The data is fed into the detection head for prediction, and the true bounding boxes annotated with thermal infrared modal analysis are used as supervision signals. The objective function is optimized as follows:

[0065] in, These are the bounding box regression loss, classification loss, and distribution focus loss for visible light features, respectively. , , These are the corresponding weights.

[0066] For semantic response supervision in the context enhancement stage: introduce a binary mask based on the real bounding box. As a supervisory function, to constrain the similarity responses of different categories, the corresponding loss function is... The definition is as follows:

[0067] in, Represents binary cross-entropy loss. This represents the category response score.

[0068] Therefore, the overall training objective of the model consists of three parts: the detection loss of the visible light alignment branch, the semantic response loss, and the detection loss of the final fused features, as detailed below:

[0069] in, To align the detection loss of visible light features, For semantic response loss, , , These are the final enhanced fusion feature bounding box regression, classification, and distribution focus loss, respectively. , , They are respectively , , The corresponding weighting coefficients.

[0070] It should be noted that the semantic response loss designed in this embodiment aims to forcibly constrain the consistency between visual features and text semantics, ensuring that the semantic response map generated by the model can accurately lock the target region and provide reliable guiding signals for the subsequent context enhancement module. The detection loss for aligned visible light features provides intermediate supervision for the aligned visible light features generated by the semantic-spatial guided alignment generation module. This ensures that the visible light features maintain strong discriminativeness after generative alignment reconstruction, preventing semantic drift in the generated features. The loss for enhanced fusion features involves joint optimization of classification, localization, and distribution regression of the final enhanced fusion features to ensure target detection accuracy.

[0071] As a further preferred technical solution, the detection head adopts the YOLOv8 detection head.

[0072] It should be noted that in this embodiment, by inputting the enhanced fusion features into the YOLOv8 detection head, the final category label, oriented bounding box, and confidence score are directly output to complete the target detection.

[0073] Furthermore, to verify the effectiveness of this method and promote related research, a large-scale, high-quality DRTV30K dataset was constructed. This dataset not only provides a large number of visible-infrared image pairs with natural spatial misalignment and fine-grained rotated bounding box annotations, but also covers a variety of lighting conditions, scene types and flight altitudes, ensuring that the model can learn a wide range of invariants and providing a solid benchmark for fair comparisons.

[0074] Regarding model training, this embodiment employs the following training environment: The learning rate was set to 0.01, momentum to 0.937, weight decay to 0.0005, batch size to 8, and input image size to 800×800. We used SGD as the optimizer and trained the model for 100 epochs. All experiments were implemented using the Python and PyTorch deep learning frameworks, and model training and testing were performed on a single NVIDIA RTX 4090 GPU.

[0075] To verify the effectiveness of this invention, comprehensive experiments were conducted on the self-built DRTV30K dataset and the publicly available DroneVehicle dataset. DRTV30K contains 30,640 pairs of aligned visible-thermal infrared images, totaling 224,619 rotated bounding box annotations, covering diverse lighting conditions (daytime, nighttime, extremely dark night, foggy days) and scene types, including urban roads, highways, overpasses, suburban areas, and parking lots. DroneVehicle is a large-scale visible-infrared UAV vehicle detection dataset containing 28,439 pairs of pre-registered visible-infrared images. We used the mAP0.5 metric, commonly used in object detection, to evaluate the model's effectiveness; this metric represents the mean average accuracy at an intersection-over-union (IoU) threshold of 0.5. The results (Tables 1, 2, and 3) show that the method proposed in this application can be applied to multimodal UAV target detection datasets, demonstrating a significant improvement over other methods.

[0076] Table 1. Comparison of test results on the DRTV30K test set

[0077] Table 2 Comparison of test results on the DroneVehicle validation set

[0078] Table 3. Comparison of test results on the DroneVehicle test set

[0079] Furthermore, a second embodiment of the present invention also proposes a target detection system based on an unmanned aerial vehicle (UAV), the system comprising: Acquisition unit 10 is used to acquire a pair of visible light images and thermal infrared images; The processing unit 20 is used to process the visible light image-thermal infrared image through a pre-trained target detection model deployed therein to obtain the target detection result; The target detection model includes an alignment generation module, a context enhancement module, and a detection head; The alignment generation module is used to extract the visual features, global semantic features, and local spatial features of the bimodal mode respectively. Based on the global semantic features and local spatial features of the bimodal mode, it generates semantic-spatial hybrid cues. The latent variables obtained by encoding the visual features of the visible light mode are iteratively denoised under the guidance of the semantic-spatial hybrid cues and then decoded to obtain visible light features that are semantically and spatially aligned with the thermal infrared mode. The context enhancement module is used to fuse the visual features of the thermal infrared mode with the visible light features, and under the guidance of the category semantic features, perform semantic perception and context enhancement on the obtained initial fused features to obtain enhanced fused features; The detection head is used to process the enhanced fusion features to obtain the target detection results.

[0080] It should be noted that other embodiments or specific implementation methods of the UAV-based target detection system described in this invention can refer to the above-described method embodiments, and will not be repeated here.

[0081] It should be noted that the computer-readable medium disclosed in this embodiment may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, and portable compact disk read-only memory (CD-ROM). ROM, optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0082] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform a zero-sample image anomaly detection method according to the above embodiments.

[0083] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server.

[0084] In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0085] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0086] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0087] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" or "several" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0088] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A target detection method based on unmanned aerial vehicles (UAVs), characterized in that, include: A pair of visible light images and thermal infrared images are input into a target detection model, which includes an alignment generation module, a context enhancement module, and a detection head. The alignment generation module extracts the visual features, global semantic features, and local spatial features of the bimodal mode, respectively. Based on the global semantic features and local spatial features of the bimodal mode, a semantic-spatial hybrid cue is generated. The latent variables obtained by encoding the visual features of the visible light mode are iteratively denoised under the guidance of the semantic-spatial hybrid cue and then decoded to obtain the visible light features that are semantically and spatially aligned with the thermal infrared mode. The visual features of the thermal infrared mode are fused with the visible light features using a context enhancement module. Then, under the guidance of category semantic features, the initial fused features are semantically perceived and context-enhanced to obtain enhanced fused features. The enhanced fusion features are processed using the detection head to obtain the target detection results.

2. The target detection method based on unmanned aerial vehicles as described in claim 1, characterized in that, The alignment generation module includes a dual-stream backbone network, an image encoder, a cueing construction network, and a U-Net network, wherein: A dual-stream backbone network is used to extract features from visible light images and thermal infrared images respectively to obtain dual-modal visual features, which include visual features of the visible light modality and visual features of the thermal infrared modality. The visible light image and the thermal infrared image are encoded separately using an image encoder to obtain the global semantic features and local spatial features of the dual-mode. The global semantic features of the dual-mode include the global semantic features of the visible light mode and the global semantic features of the thermal infrared mode. The local spatial features of the dual-mode include the local spatial features of the visible light mode and the local spatial features of the thermal infrared mode. The semantic cue embedding is obtained by performing semantic attention calculation on the global semantic features of the thermal infrared mode and the global semantic features of the visible light mode using a cue construction network; and by performing semantic attention calculation on the local spatial features of the visible light mode and the local spatial features of the thermal infrared mode to obtain the spatial cue embedding; and the semantic-spatial hybrid cue is constructed based on the semantic cue embedding and the spatial cue embedding. The visual features of the visible light modality are encoded into latent variables using a U-Net network, and the latent variables are iteratively denoised under the conditional guidance of the semantic-spatial hybrid cue. The denoised latent variables are then decoded to obtain the visible light features.

3. The target detection method based on unmanned aerial vehicles as described in claim 2, characterized in that, The prompting construction network includes a semantic prompting construction network and a spatial prompting construction network, wherein: The semantic prompt construction network includes a semantic attention calculation mechanism and a first multilayer perceptron. The semantic attention calculation mechanism performs semantic attention calculation using the global semantic features of the thermal infrared modality as the query and the global semantic features of the visible light modality as the key and value, and obtains the semantic attention calculation result. The semantic attention calculation result is added to the global semantic features of the thermal infrared modality and output to the first multilayer perceptron to generate the semantic prompt embedding. The spatial cue construction network includes a spatial attention calculation mechanism and a second multilayer perceptron. The spatial attention calculation mechanism performs spatial attention calculation using local spatial features of the thermal infrared mode as queries and local spatial features of the visible light mode as keys and values ​​to obtain the spatial attention calculation result. The spatial attention calculation result is then subtracted from the local spatial features of the thermal infrared mode and output to the second multilayer perceptron to generate the spatial cue embedding. The semantic and spatial cues are concatenated and output to a third multilayer perceptron for projection fusion to generate the semantic-spatial hybrid cues.

4. The target detection method based on unmanned aerial vehicles as described in claim 2, characterized in that, The dual-stream backbone network includes a first feature extractor and a second feature extractor with identical structures but independent parameters. The first feature extraction process extracts features from the visible light image to obtain the visual features of the visible light modality; The second feature extractor extracts features from the thermal infrared image to obtain the visual features of the thermal infrared mode.

5. The target detection method based on unmanned aerial vehicles as described in claim 2, characterized in that, The image encoder is a pre-trained CLIP image encoder with frozen parameters.

6. The target detection method based on unmanned aerial vehicles as described in claim 1, characterized in that, The context enhancement module includes a text encoder, a semantic awareness network, and a context aggregation network, wherein: By using a text encoder to encode category names for texts from different scenarios, category semantic features are obtained. The initial fused features are obtained by fusing the visual features of the thermal infrared modality with the visible light features using a semantic perception network. The cosine similarity between the initial fused features and the semantic features of each category is calculated to obtain the response score of each category. The response scores of each category are averaged to generate a comprehensive semantic response map. A context aggregation network is used to perform threshold segmentation on the comprehensive semantic response map to generate a pseudo-target region mask. Based on the introduced expandable learning coefficient, an adaptive expansion operation is performed on the pseudo-target region mask to generate a context-aware mask. The visible light features and the visual features of the thermal infrared modality are weighted and pooled using context-aware masks to obtain context features of the two modalities, which are then fused to obtain a context semantic representation. The initial fusion feature, the pseudo-target region mask, and the corresponding contextual semantic features within the pseudo-target region are concatenated. The resulting concatenated features are then fused through convolution, and the channel fusion result is added to the initial fusion feature to obtain the enhanced fusion feature.

7. The target detection method based on unmanned aerial vehicles as described in claim 6, characterized in that, The text encoder is a pre-trained CLIP text encoder with frozen parameters.

8. The target detection method based on unmanned aerial vehicles as described in claim 1, characterized in that, Before inputting a pair of visible light images and thermal infrared images into the target detection model, the method further includes: The target detection model is trained using an end-to-end joint training strategy. The loss functions used in the model training process include the detection loss of the visible light alignment branch, the semantic response loss, and the detection loss of the final fused features. The formula for the loss function is expressed as follows: In the formula, To align the detection loss of visible light features, For semantic response loss, , , These are bounding box regression, classification, and distribution focus losses, respectively, to enhance fusion features. , , They are respectively , , The corresponding weighting coefficients; The detection loss for aligning visible light features is: In the formula, , , These are the bounding box regression loss, classification loss, and distribution focus loss for visible light features, respectively. , , They are respectively , , The corresponding weights; The semantic response loss is: In the formula, Represents binary cross-entropy loss. A binary mask generated based on the real bounding box. This represents the category response score.

9. The target detection method based on unmanned aerial vehicles as described in claim 1, characterized in that, The detection head used is the YOLOv8 detection head.

10. A target detection system based on unmanned aerial vehicles (UAVs), characterized in that, include: The acquisition unit is used to acquire a pair of visible light images and thermal infrared images; The processing unit is used to process the visible light image-thermal infrared image through a pre-trained target detection model deployed therein to obtain the target detection result; The target detection model includes an alignment generation module, a context enhancement module, and a detection head; The alignment generation module is used to extract the visual features, global semantic features, and local spatial features of the bimodal mode respectively. Based on the global semantic features and local spatial features of the bimodal mode, it generates semantic-spatial hybrid cues. The latent variables obtained by encoding the visual features of the visible light mode are iteratively denoised under the guidance of the semantic-spatial hybrid cues and then decoded to obtain visible light features that are semantically and spatially aligned with the thermal infrared mode. The context enhancement module is used to fuse the visual features of the thermal infrared mode with the visible light features, and under the guidance of the category semantic features, perform semantic perception and context enhancement on the obtained initial fused features to obtain enhanced fused features; The detection head is used to process the enhanced fusion features to obtain the target detection results.

Images