An infrared-prior-driven infrared-visible light fusion unmanned aerial vehicle target detection method

Abstract

The application discloses an infrared prior-driven infrared-visible light fusion unmanned aerial vehicle target detection method, and relates to the technical fields of computer vision, image processing, multi-modal information fusion and unmanned aerial vehicle target detection. The method acquires infrared images and visible light images under the same scene, performs low-light robust modeling on the visible light images to obtain reliable visible light auxiliary information, generates an infrared saliency prior map by using the infrared images, and constructs a multi-scale infrared prior pyramid. Small target feature enhancement, cross-modal local alignment and position compensation are performed under the guidance of the infrared prior, and then quality perception cross-modal fusion is performed in combination with modal quality, alignment confidence and visible light reliability to obtain fused multi-scale features. Finally, a P2-P5 multi-scale detection structure and a decoupled detection head are used to output the category, confidence and bounding box position of the unmanned aerial vehicle target. The application can relieve the detection instability problems caused by visible light degradation under low-light conditions, non-strict registration of infrared and visible light and differences in double modal quality, and improve the accuracy and robustness of unmanned aerial vehicle small target detection under complex backgrounds.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, image processing, multimodal information fusion, and UAV target detection technology, specifically to an infrared-prior-driven infrared-visible light fusion UAV target detection method. Background Technology

[0002] With the development of unmanned aerial vehicle (UAV) platforms, airborne optoelectronic payloads, and intelligent sensing technologies, UAV target detection has been widely applied in tasks such as low-altitude security monitoring, key area patrols, target intrusion early warning, and complex environment perception. In actual low-altitude monitoring scenarios, UAV targets typically exhibit characteristics such as large variations in flight distance, small target size, low pixel ratio, blurred boundaries, insufficient texture information, and strong background interference. Especially under conditions of dusk, night, low light, and complex backgrounds, UAV targets are easily confused with the sky, buildings, trees, lights, or other small heat sources, thus increasing the difficulty of target detection.

[0003] Currently, UAV target detection typically uses visible light images, infrared images, or infrared-visible dual-modal images as input. Visible light images provide rich information on target contours, edges, textures, and colors, which is beneficial for representing target structural details. However, under low light, nighttime, shadow, or strong background interference conditions, visible light images are prone to problems such as insufficient brightness, reduced contrast, increased noise, and degradation of target details, leading to decreased target region discriminability. Infrared images can reflect the difference in thermal radiation between the target and the background, and have a strong ability to represent target saliency in low light and nighttime environments. However, infrared images usually suffer from low spatial resolution, insufficient texture details, unclear edge structures, and limited class discrimination information. Therefore, a single modality is difficult to simultaneously achieve both target saliency representation and structural detail representation.

[0004] To improve target detection performance in complex environments, existing technologies increasingly employ dual-modal fusion detection using infrared and visible light images. Infrared-visible light fusion methods typically include image-level fusion, feature-level fusion, and decision-level fusion. Image-level fusion methods directly synthesize the two modalities at the pixel level; feature-level fusion methods extract infrared and visible light features separately and fuse them within the network through methods such as splicing, addition, or attention weighting; decision-level fusion methods detect the two modalities separately and then merge the detection results. These methods, to a certain extent, can utilize the complementary information between different modalities to improve the stability of target detection.

[0005] However, in practical dual-light UAV detection systems, infrared and visible light cameras often suffer from differences in imaging resolution, field of view, installation location, lens parameters, and time synchronization errors, making it difficult to maintain strict pixel-level registration between infrared and visible light images. For small, long-range UAV targets, even a small spatial offset between the two modes can lead to incorrect correspondence between the target area in one mode and the background area in the other. Directly stitching, adding, or globally weighted fusing unregistered infrared and visible light features can easily introduce misaligned background information and mismatched features, weakening target response and increasing the risk of false positives and false negatives.

[0006] Furthermore, the information quality of different modalities varies across different scenes and spatial locations. Visible light modalities may exhibit missing target textures, amplified noise, and over-enhanced performance in low-light regions; while infrared modalities can highlight thermal targets, they may produce spurious responses in the presence of similar heat sources, reflection interference, or complex thermal backgrounds. Traditional fusion methods typically employ fixed or coarse-grained weight allocation, making it difficult to dynamically adjust the fusion strategy based on modal quality, target location, local spatial differences, and alignment reliability. This leads to the erroneous amplification of low-quality modal information, affecting the final detection results.

[0007] Therefore, to address the challenges of non-registration between infrared and visible light images, large differences in modal quality, uneven information distribution, and difficulty in detecting small targets in complex low-light environments, it is necessary to propose an infrared-visible light fusion UAV target detection method that can utilize infrared target saliency priors for spatial guidance, perform local alignment and position compensation for cross-modal features, and adaptively allocate fusion weights based on modal quality. This method aims to improve the accuracy and robustness of UAV small target detection in complex scenarios. Summary of the Invention

[0008] The purpose of this invention is to overcome the shortcomings of the existing technology and provide an infrared-visible light fusion UAV target detection method driven by infrared prior, so as to solve the problems of non-strict registration of infrared and visible light images in low-light complex environments, degradation of visible light image quality, unbalanced distribution of dual-modal information, and insufficient detection accuracy of small UAV targets.

[0009] To achieve the above objectives, the present invention adopts the following technical solution:

[0010] An infrared-prior-driven infrared-visible light fusion UAV target detection method includes the following steps:

[0011] S1. Acquire infrared and visible light images of the same UAV observation scene, using the infrared image as the source of target salience information and the visible light image as the source of target structure and texture information.

[0012] S2. Perform low-light robust visible light modeling on the visible light image to obtain robust visible light information and a visible light reliability map; step S2 includes low-light enhancement, visible light enhancement fusion, and visible light reliability modeling; the low-light enhancement includes: estimating the illuminance of the input visible light image using a lightweight convolutional subnetwork to obtain a single-channel illuminance map; performing brightness compensation on the input visible light image based on the single-channel illuminance map and learnable gain parameters to obtain a preliminary enhanced image; jointly modeling the input visible light image and the preliminary enhanced image, generating a structure compensation term through a detail recovery branch, and obtaining the enhanced visible light image based on the structure compensation term. The visible light enhancement fusion includes: inputting the original visible light image, the enhanced visible light image, and illuminance information into a fusion coefficient prediction network to obtain spatially correlated fusion weights; and adaptively fusing the original visible light image and the enhanced visible light image according to the spatially correlated fusion weights to obtain robust visible light information. The visible light reliability modeling includes: inputting the robust visible light information into a reliability estimation network to obtain a visible light reliability map, wherein the visible light reliability map is used to constrain the cross-modal local alignment process and to modulate the contribution of visible light features in quality-perceived cross-modal fusion.

[0013] S2.1: Let the input visible light image be... First, the input visible light image is illuminated using an illuminance estimation network to obtain a single-channel illuminance map:

[0014] ;

[0015] in, This represents the illuminance estimation network. This represents the relative brightness distribution at various spatial locations in the input visible light image. These represent the height and width of the image, respectively.

[0016] The original visible light image is brightness compensated based on the single-channel illuminance map and learnable gain parameters to obtain a preliminary enhanced image. :

[0017] ;

[0018] in, For learnable gain parameters, This is the illuminance map after stabilization. This represents element-wise multiplication;

[0019] S2.2: To reduce noise amplification and structural loss caused by simple brightness enhancement, the original visible light image and the preliminary enhanced image are stitched together, and a structural compensation term is generated through a detail recovery network.

[0020] ; ;

[0021] in, Indicates channel splicing. This indicates details of network recovery. Indicates the details of the compensation item. This represents the enhanced visible light image;

[0022] S2.3: Predict spatially relevant fusion weights based on the original visible light image, the enhanced visible light image, and the illuminance map:

[0023] ;

[0024] in, This represents a fusion coefficient prediction network. This indicates the degree of trust in the enhancement result at different spatial locations; based on the fusion weights, the original visible light image and the enhanced visible light image are adaptively fused to obtain robust visible light information. :

[0025] ;

[0026] S2.4: Explicitly estimate the quality of visible light information using a reliability estimation network to obtain a visible light reliability map.

[0027] ;

[0028] in, Represents a reliability estimation network. The visible light reliability map represents the reliability of visible light information in different spatial regions in UAV target detection tasks. The visible light reliability map is used to constrain the subsequent cross-modal local alignment process and to modulate the contribution of visible light features in quality-sensing cross-modal fusion.

[0029] S3. Input the infrared image and the robust visible light information into a dual-branch feature encoding network to obtain multi-scale infrared features and multi-scale visible light features, respectively. Simultaneously, input the infrared image into a pre-trained infrared prior model to generate an infrared saliency prior map and construct an infrared prior pyramid. The pre-trained infrared prior model is an infrared small target saliency prior network. The infrared small target saliency prior network generates an infrared saliency prior map of the same scale as the input infrared image based on the input infrared image. Regions with larger response values ​​in the infrared saliency prior map correspond to spatial locations that are more likely to contain UAV targets. Perform bilinear interpolation on the infrared saliency prior map according to the spatial dimensions of the feature maps at each scale to obtain a multi-scale infrared prior map with the same dimensions as the feature maps at each scale, thus forming the infrared prior pyramid.

[0030] Let the input infrared image be The saliency prior map is obtained through the infrared prior model:

[0031] ;

[0032] in, This represents a pre-trained infrared prior model. This represents an infrared saliency prior map at the same scale as the input infrared image; regions with larger response values ​​in the infrared saliency prior map correspond to locations more likely to contain UAV targets.

[0033] Since the detection network performs inference in a multi-scale feature space, the infrared saliency prior map is mapped to feature layers of different scales to construct an infrared prior pyramid:

[0034] ;

[0035] in, Indicates the relationship with the first Infrared prior maps with consistent spatial dimensions across multiple scale feature maps; through the infrared prior pyramid, infrared prior information can provide positional constraints in the processes of multi-scale feature enhancement, cross-modal alignment, quality-aware fusion, and detection response optimization.

[0036] S4. Based on the infrared prior pyramid, perform prior-guided small target feature enhancement on multi-scale infrared features to obtain prior-enhanced multi-scale infrared features; for infrared features of any scale, convert the corresponding scale infrared prior map into a prior gating map matching the number of infrared feature channels at the current scale through convolution mapping and a sigmoid activation function; spatially reweight the current scale infrared features using the prior gating map to enhance the feature response of the target-related region and suppress background interference; input the reweighted infrared features and the prior gating map into the residual enhancement branch to generate target-related residual terms; fuse the target-related residual terms with the main branch infrared features, and perform channel reshaping and semantic reconstruction through the semantic refinement module to obtain prior-enhanced infrared features.

[0037] S4.1: Let the first... Layer infrared characteristics are The corresponding infrared prior image at the scale is ; These represent the number of channels, height, and width of the infrared feature, respectively.

[0038] pass Convolution maps the infrared prior map to a prior gating map that matches the number of infrared feature channels at the current scale:

[0039] ;

[0040] in, This represents a scale-dependent linear mapping function. This represents the Sigmoid activation function. This represents the prior gating graph at the current scale;

[0041] The infrared features are spatially reweighted based on the prior gating map to obtain the reweighted infrared features. :

[0042] ;

[0043] The spatial reweighting enhances the feature response of the target-related region and suppresses background interference.

[0044] S4.2: Input the reweighted infrared features and prior gated map into the convolutional residual enhancement branch to generate target-related residual terms. :

[0045] ;

[0046] in, Indicates the first Convolutional residual enhancement branches at various scales;

[0047] S4.3: Fuse the main branch features and the residual branch, and output the enhanced infrared features based on the prior through the feature refinement function. :

[0048] ;

[0049] in, This represents a feature refinement function composed of convolutional layers, used for channel reshaping and semantic reconstruction of enhanced infrared features.

[0050] S5. Based on the infrared prior pyramid and the visible light reliability map, perform cross-modal local alignment and position compensation with prior constraints on the a priori-enhanced multi-scale infrared features and multi-scale visible light features to obtain compensated multi-scale visible light aligned features; determine the target-related regions that need cross-modal local alignment according to the infrared prior pyramid and the visible light reliability map, and perform offset estimation only in regions with high infrared prior response and reliable visible light information; stitch together the a priori-enhanced infrared features, the corresponding scale visible light features, and the prior gating map, and input them into the offset estimation network to obtain a two-dimensional local offset field; map the two-dimensional local offset field to a normalized coordinate space, and resample the visible light features through bilinear sampling to obtain preliminarily aligned visible light features.

[0051] The enhanced infrared features, the pre-aligned visible light features, and the prior gating map are jointly modeled to obtain an alignment confidence map, which is used to characterize the matching confidence level between infrared features and visible light features at different spatial locations. The residual relationship between the enhanced infrared features and the pre-aligned visible light features is analyzed, and a position compensation map is generated in combination with the prior gating map. The position compensation map is modulated using the alignment confidence map to obtain the compensated visible light alignment features.

[0052] S5.1: For any scale, the enhanced infrared features, the visible light features of the corresponding scale, and the prior gating map are stitched together and input into the migration estimation network to obtain a two-dimensional local migration field. :

[0053] ;

[0054] in, Indicates the first A offset estimation network at multiple scales This is a learnable scale parameter used to control the range of the offset field amplitude. Indicates the first Visible light characteristics at various scales;

[0055] The two-dimensional local offset field is mapped to a normalized coordinate space, and the visible light features are resampled using a bilinear sampling operation to obtain the initially aligned visible light features. :

[0056] ;

[0057] in, Indicates a bilinear sampling operation;

[0058] S5.2: Jointly model the enhanced infrared features, the preliminarily aligned visible light features, and the prior gating graph to obtain the alignment confidence map:

[0059] ;

[0060] in, This indicates the reliability of the alignment results between infrared and visible light features at the current location;

[0061] S5.3: By analyzing the residual relationship between the enhanced infrared features and the pre-aligned visible light features, and combining it with the prior gating graph, a position compensation map is generated. :

[0062] ;

[0063] in, Indicates the location compensation branch;

[0064] S5.4: Modulate the position compensation map using the alignment confidence map to obtain the compensated visible light alignment features:

[0065] ;

[0066] Through the above methods, the present invention can perform local offset estimation and feature compensation within the target-related region indicated by infrared priors, thereby reducing the misalignment fusion problem caused by non-strict registration between infrared and visible light images.

[0067] S6. Based on the modal global quality, local spatial quality, alignment confidence, and the visible light reliability map, perform quality-aware cross-modal fusion of the prior enhanced multi-scale infrared features and the compensated multi-scale visible light alignment features to obtain the fused multi-scale features. Step S6 includes: constructing a global quality estimation branch using global pooling and linear mapping to obtain image-level global quality weights for the infrared and visible light modalities; constructing a local quality estimation branch and obtaining spatially related local quality weights based on the prior enhanced infrared features, the compensated visible light alignment features, the corresponding scale infrared prior map, and the alignment confidence map; jointly modeling the image-level global quality weights and the local quality weights, and constraining the final fusion contribution of the visible light branch based on the alignment confidence map and the visible light reliability map; normalizing the infrared branch weights and the visible light branch weights, and performing weighted fusion of the infrared features and the visible light alignment features based on the normalized weights. Step S6 further includes a priori residual compensation branch; the priori residual compensation branch generates a residual compensation term based on the enhanced infrared features, the compensated visible light alignment features, and the infrared prior image at the corresponding scale, and superimposes the residual compensation term into the basic fusion feature to obtain the final fusion feature; the residual compensation term is enhanced in the infrared prior high response region to restore the information loss in the target region caused by modal differences or local alignment errors, and to suppress background interference.

[0068] S6.1: Let the first... The layer input features are the enhanced infrared features from prior knowledge. Compensated visible light alignment features Infrared prior image Alignment Confidence Plot and visible light reliability map ;

[0069] Global pooling and linear mapping are used to construct a global quality estimation branch, resulting in image-level global quality weights for two modalities:

[0070] ;

[0071] in, and These represent the global confidence levels of the infrared and visible light modes in the current image as a whole;

[0072] S6.2: Construct a local quality estimation branch, and introduce infrared prior maps and aligned confidence maps into the fusion weight calculation:

[0073] ;

[0074] in, and Represents the local mass weights related to spatial location;

[0075] S6.3: Jointly model the global quality weights and local quality weights, and impose alignment confidence and visible light reliability constraints on the visible light weights:

[0076] ; ;

[0077] S6.4: To ensure the comparability of the weights for the two modes, the infrared weights and visible light weights are normalized:

[0078] ; ;

[0079] Therefore, the basic fusion feature is represented as:

[0080] ;

[0081] S6.5: Introduce a priori residual compensation branch in the fusion module, generating residual compensation terms based on the difference between infrared features and visible light alignment features, combined with the infrared prior image.

[0082] ;

[0083] The final fusion result is expressed as follows:

[0084] ;

[0085] in, Indicates the first Fusion features at various scales. Through the quality-aware cross-modal fusion process, fusion weights can be dynamically allocated based on the reliability of different modalities in the current image and current spatial location, and target-related residual compensation can be enhanced in the infrared prior high-response region.

[0086] S7. Input the fused multi-scale features into the multi-scale small target detection head, and output the category, confidence score, and bounding box position of the UAV target through a prior-guided detection response optimization mechanism. The multi-scale small target detection head in step S7 adopts a four-scale detection structure of P2, P3, P4, and P5; wherein, the P2 layer is used for extremely small target detection, the P3 layer is used for small target detection, the P4 layer is used for medium-scale target detection, and the P5 layer is used for larger targets and global semantic modeling; each scale detection head adopts a decoupled structure, including a shared feature adaptation layer, a bounding box regression branch, a category prediction branch, and a target prediction branch. The shared feature adaptation layer is used to unify the feature expression form at different scales, the bounding box regression branch is used to output the bounding box position of the UAV target, the category prediction branch is used to output the UAV target category, and the target prediction branch is used to output the probability of the UAV target's existence. The prior-guided detection response optimization mechanism includes high-resolution layer target refinement and prior-guided target bias. Specifically, a lightweight convolutional refinement network is introduced into the target prediction branches of layers P2 and P3 to perform secondary adjustments on the initial target prediction results, thereby enhancing the response in small target regions. The corresponding scale infrared prior image is converted into a target bias term, and this bias term is introduced into the target prediction process through learnable weight parameters, allowing infrared prior information to directly participate in the detection decision. During the training phase, paired infrared images, visible light images, and UAV target bounding box annotations are used to train the detection network. During the inference phase, the infrared and visible light images to be detected are input, and without relying on manually annotated bounding boxes or strict pixel-level registration results, the UAV target detection is jointly completed by the infrared prior model, cross-modal local alignment and position compensation mechanism, quality-aware cross-modal fusion mechanism, and multi-scale small target detection head.

[0087] S7.1: Let the fused feature pyramid obtained after cross-modal feature modeling be:

[0088] ;

[0089] in, Indicates the first The multi-scale small target detection head employs a four-scale detection structure of P2, P3, P4, and P5, where the P2 layer is used for extremely small target detection, the P3 layer for small target detection, the P4 layer for medium-scale target detection, and the P5 layer for larger targets and global semantic modeling.

[0090] S7.2: Fusion features for any scale First, channel transformation is performed through a shared feature adaptation layer to obtain the input features of the detection head:

[0091] ;

[0092] in, This represents the feature adaptation function composed of convolutional layers. Subsequently, Input the bounding box regression branch, the category prediction branch, and the object prediction branch respectively, and obtain the corresponding outputs:

[0093] ; ; ;

[0094] in, This indicates the bounding box regression results. Indicates the category prediction result, Indicates the result of targeted prediction;

[0095] S7.3: Introduce target-oriented refinement branches in layers P2 and P3 to perform secondary adjustments on the initial target-oriented predictions:

[0096] ;

[0097] in, This represents the Sigmoid activation function. This represents a lightweight convolutional thinning network;

[0098] S7.4: Introduce an infrared prior bias term in the target branch, let the first... The infrared prior image of the layer is Then, the targeted prediction is expressed as:

[0099] ;

[0100] in, For learnable weight parameters, This represents the transformation that maps prior probabilities to logarithmic probability space. Through the aforementioned target bias term, infrared prior information is directly involved in the detection decision process, enhancing the target response of small target areas in the final prediction stage.

[0101] S7.5: During the training phase, the detection network is trained using paired infrared images, visible light images, and UAV target bounding box annotations. During the inference phase, the infrared and visible light images to be detected are input, and UAV target detection is completed by a combination of infrared prior models, cross-modal local alignment and position compensation mechanisms, quality-aware cross-modal fusion mechanisms, and multi-scale small target detection heads, without relying on manually annotated bounding boxes or strict pixel-level registration results.

[0102] Compared with the prior art, the present invention has the following beneficial effects:

[0103] (1) The present invention uses infrared images as the source of target saliency information and generates infrared saliency prior maps through infrared prior models, so that infrared prior information can participate in multi-scale feature enhancement, cross-modal local alignment, quality perception fusion and detection response optimization processes, thereby improving the response intensity of small target areas of UAVs in low-light complex scenes.

[0104] (2) This invention constructs a robust modeling mechanism for visible light under low illumination. Through illuminance estimation, brightness compensation, detail restoration, adaptive enhancement fusion and reliability modeling, it improves the effectiveness of target structure and texture information in visible light images under low illumination conditions, and suppresses the interference of low-quality visible light regions on the cross-modal fusion process through visible light reliability maps.

[0105] (3) The present invention proposes a cross-modal local alignment and position compensation mechanism with prior constraints, which limits the alignment range to the target correlation area with high infrared prior response and reliable visible light information. Offset estimation and feature correction are performed only in local candidate areas, which can reduce the computational overhead caused by dense alignment of the whole image and alleviate the misalignment fusion problem caused by non-strict registration of infrared and visible light images.

[0106] (4) This invention proposes a quality-perceived cross-modal fusion mechanism, which performs reliability modeling of infrared mode and visible light mode from two levels: global scene quality and local spatial quality. It also dynamically adjusts the fusion contribution of the two modes by combining the alignment confidence map and the visible light reliability map, which can suppress the negative impact of low-quality mode, misaligned mode and background interference area on the detection results.

[0107] (5) The present invention adopts a P2–P5 four-scale detection structure and a decoupled small target detection head. It enhances the spatial representation capability of extremely small targets and small targets through a high-resolution detection layer, and enhances the detection response of weak UAV targets through target-oriented refinement branches and infrared prior bias terms, thereby improving the UAV small target detection capability.

[0108] (6) Through the synergistic effect of low-light visible light robust modeling, infrared prior guidance, cross-modal local alignment, position compensation, quality perception fusion and multi-scale detection optimization, this invention can improve the target detection accuracy and robustness in dual-light UAV detection scenarios with low light, complex background, modal non-registration and significant resolution differences. Attached Figure Description

[0109] Figure 1 This is a schematic diagram of the overall network architecture of an infrared-visible light fusion UAV target detection method driven by infrared priors according to the present invention.

[0110] Figure 2 This is a schematic diagram of the overall detection process of an infrared-visible light fusion UAV target detection method driven by infrared priors according to the present invention.

[0111] Figure 3 This is a schematic diagram of the structure of the low-light visible light enhancement module of the present invention.

[0112] Figure 4 This is a schematic diagram of the visible light enhancement and fusion module of the present invention.

[0113] Figure 5 This is a schematic diagram of the visible light reliability modeling module of the present invention.

[0114] Figure 6 This is a schematic diagram of the structure of the external infrared prior network access and multi-scale prior construction module of the present invention.

[0115] Figure 7 This is a schematic diagram of the prior-guided small target feature enhancement module of the present invention.

[0116] Figure 8 This is a schematic diagram of the cross-modal alignment and position compensation module with prior constraints of the present invention.

[0117] Figure 9 This is a schematic diagram of the structure of the quality perception cross-modal fusion module of the present invention.

[0118] Figure 10 This is a schematic diagram of the P2–P5 multi-scale detection structure that integrates prior knowledge according to the present invention.

[0119] Figure 11 This is a schematic diagram of the structure of the decoupled small target detection head of the present invention.

[0120] Figure 12 This is a schematic diagram of the prior guidance detection response optimization module of the present invention.

[0121] Figure 13 This is a schematic diagram of the detection results of the method of the present invention under different background complexities and lighting conditions on a self-built dataset.

[0122] Figure 14 This is a schematic diagram of the detection results of the method of the present invention on a public dataset under different background complexities and lighting conditions.

[0123] Figure 15 This is a schematic diagram of the detection results of the infrared prior network of this invention on a self-built dataset.

[0124] Figure 16 This is a schematic diagram illustrating the detection results of the method of the present invention in distinguishing between UAV targets and similar heat source interference targets.

[0125] Figure 17 This is a visual schematic diagram of the intermodal position mapping results of the present invention. Detailed Implementation

[0126] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention. Equivalent substitutions, simple modifications, or improvements made to the present invention by those skilled in the art without departing from the concept of the present invention should all be included within the scope of protection of the present invention.

[0127] Example 1:

[0128] This embodiment provides an infrared-prior-driven infrared-visible light fusion UAV target detection method. For example... Figure 1 and Figure 2 As shown, the method takes infrared and visible light images as input, primarily guided by target saliency information provided by the infrared modality, and supplemented by target structure and texture information provided by the visible light modality. Through low-light visible light robust modeling, infrared prior construction, prior-guided feature enhancement, cross-modal local alignment and position compensation, quality-aware cross-modal fusion, and a multi-scale small target detection head, it outputs the UAV target's category, confidence level, and bounding box location. Specifically, it includes the following steps:

[0129] S1. Acquire infrared and visible light images.

[0130] Infrared and visible light images are acquired from the same UAV observation scene. The infrared image provides information on the salience of the target's thermal radiation, while the visible light image provides information on the target's contour, texture, and structural details. In this embodiment, the input infrared image is denoted as... Input visible light image denoted as Strict pixel-level registration is not required for the two modal images. Subsequently, spatial misalignment at the feature level is corrected through an infrared prior-guided cross-modal local alignment and position compensation mechanism.

[0131] S2. Construct a robust expression for visible light under low light conditions.

[0132] like Figure 3 As shown, low-light enhancement is performed on the input visible light image. Let the input visible light image be:

[0133]

[0134] First, a lightweight convolutional subnetwork is used to estimate the illuminance of the input visible light image, resulting in a single-channel illuminance map:

[0135]

[0136] in, This represents the illuminance estimation network. This represents the relative brightness distribution at various spatial locations in the input visible light image. Lower... The value indicates that the corresponding area is not bright enough and requires stronger brightness compensation.

[0137] After obtaining the illuminance map, a luminance compensation mechanism based on learnable gain parameters is introduced to initially enhance the original visible light image:

[0138]

[0139] in, For learnable gain parameters, This is the illuminance map after stabilization. This indicates element-wise multiplication. Through the above processing, low-light areas can be significantly enhanced while suppressing excessive magnification of high-light areas.

[0140] To mitigate the noise amplification and structural blurring issues caused by simple brightness enhancement, a detail restoration branch is further introduced. The original visible light image and the initially enhanced image are stitched together, and a structure compensation term is generated through the detail restoration network.

[0141]

[0142] in, Indicates channel splicing. This indicates details of network recovery. Indicates the details of the compensation item. This represents the enhanced visible light image. This detail recovery branch allows for compensation of target edges and texture structure while simultaneously increasing brightness.

[0143] like Figure 4 As shown, after obtaining the enhanced visible light image, an adaptive fusion mechanism between the enhanced image and the original image is further constructed. Specifically, the original visible light image, the enhanced visible light image, and the illuminance map are jointly modeled to predict spatially relevant fusion weights:

[0144]

[0145] in, This represents a fusion coefficient prediction network. This indicates the degree of trust in the enhancement result at different spatial locations. Based on the fusion weights, the original visible light image and the enhanced visible light image are adaptively fused to obtain robust visible light information.

[0146]

[0147] When the augmentation results for a certain region are more reliable When the image is larger, the model tends to use the enhanced visible light information; when the original image information in a certain region is more stable, the model is more likely to use the enhanced visible light information. The smaller size allows the model to better preserve the original visible light information. This avoids the problems of local over-enhancement or noise amplification that can occur when directly using enhanced images.

[0148] like Figure 5 As shown, a visible light reliability modeling branch is further constructed to explicitly estimate the quality of visible light information in different spatial regions:

[0149]

[0150] in, Represents a reliability estimation network. This represents the visible light reliability map. The visible light reliability map is not directly used for image reconstruction, but rather serves as quality constraint information in subsequent cross-modal alignment and cross-modal fusion processes to suppress interference from low-quality visible light regions on the fused features.

[0151] S3. Construct the infrared prior map and the infrared prior pyramid.

[0152] like Figure 6 As shown, the input infrared image is fed into a pre-trained infrared prior model to obtain an infrared saliency prior map. Let the input infrared image be:

[0153]

[0154] The saliency prior map is obtained through the infrared prior model:

[0155]

[0156] in, This represents a pre-trained infrared prior model. This represents an infrared saliency prior map at the same scale as the input infrared image. Regions with larger response values ​​in the infrared saliency prior map correspond to areas more likely to contain UAV targets.

[0157] In this embodiment, since the target detection network performs inference in a multi-scale feature space, it is necessary to map the infrared saliency prior map to feature layers of different scales to construct an infrared prior pyramid:

[0158]

[0159] in,

[0160]

[0161] Indicates the relationship with the first Infrared prior maps with consistent spatial dimensions for each scale feature map. These infrared prior pyramids participate in subsequent infrared feature enhancement, cross-modal local alignment, quality-aware fusion, and detection response optimization processes.

[0162] S4. Enhance small target features with prior guidance.

[0163] like Figure 7 As shown, small target feature enhancement is performed based on prior guidance using infrared prior pyramids to enhance multi-scale infrared features. Let the first... The layer infrared characteristics are:

[0164]

[0165] The corresponding infrared prior image is:

[0166]

[0167] First, through Convolution maps the infrared prior map to a prior gating map that matches the number of infrared feature channels at the current scale:

[0168]

[0169] in, This represents a scale-dependent linear mapping function. This represents the Sigmoid activation function. This represents the prior gating graph at the current scale.

[0170] Subsequently, the infrared features were spatially reweighted using a priori gating maps:

[0171]

[0172] Through the above spatial reweighting process, the infrared feature response of the target-related region is enhanced, while the interference from the background region is suppressed.

[0173] To further enhance the semantic representation of weak targets within the prior region, the reweighted infrared features and the prior gated map are input into the convolutional residual enhancement branch to generate target-related residual terms:

[0174]

[0175] in, Indicates the first Convolutional residual enhancement branches at various scales. Subsequently, the main branch features are fused with the residual branches, and the enhanced prior infrared features are output through a feature refinement function:

[0176]

[0177] in, Indicates the first Feature refinement functions at various scales are used to perform channel reshaping and semantic reconstruction on the enhanced infrared features.

[0178] S5. Perform cross-modal local alignment and position compensation with prior constraints.

[0179] like Figure 8 As shown, after infrared feature enhancement, cross-modal local alignment and position compensation are performed on the enhanced infrared and visible light features. Unlike global dense alignment of the entire image or feature map, this embodiment uses infrared prior maps and visible light reliability maps to limit the alignment range to potential UAV targets and their neighborhoods, thereby reducing mismatches and invalid calculations in the background area.

[0180] For any given scale, the enhanced infrared features, the corresponding visible light features, and the prior gating map are stitched together and input into the migration estimation network to obtain a two-dimensional local migration field:

[0181]

[0182] in, Indicates the first A offset estimation network at multiple scales Indicates the first Visible light characteristics at various scales This is a learnable scale parameter used to control the range of the migration field amplitude, avoiding excessive migration in high-resolution layers.

[0183] The two-dimensional local offset field is mapped to a normalized coordinate space, and the visible light features are resampled through bilinear sampling to obtain the initially aligned visible light features:

[0184]

[0185] in, This indicates a bilinear sampling operation. After this operation, target-related information in the visible light features is mapped to spatial coordinates closer to the infrared target's location.

[0186] Since the quality of visible light modes is unstable at night, and non-strict registration introduces uncertainty, an alignment confidence branch is further designed to explicitly evaluate the current alignment result:

[0187]

[0188] in, This indicates the reliability of the alignment results between infrared and visible light features at the current location.

[0189] Furthermore, by analyzing the residual relationship between the enhanced infrared features and the preliminarily aligned visible light features, and combining this with the prior gating map, a position compensation map is generated.

[0190]

[0191] in, Indicates the first Position compensation branches at various scales. Finally, the position compensation map is modulated using the alignment confidence map to obtain the compensated visible light alignment features:

[0192]

[0193] Therefore, in the aligned reliable region, the position compensation branch can further recover the information loss caused by local misalignment or modal differences; in the aligned unreliable region, the compensation effect is suppressed, thereby avoiding the injection of unreliable visible light information into the subsequent fusion process.

[0194] S6. Perform quality perception cross-modal fusion.

[0195] like Figure 9 As shown, after obtaining the enhanced infrared features and the compensated visible light aligned features, the two modal features are adaptively fused through a quality-aware cross-modal fusion mechanism.

[0196] Let the first The layer input features are the enhanced infrared features from prior knowledge. Compensated visible light alignment features Infrared prior image Alignment Confidence Plot and visible light reliability map First, a global quality estimation branch is constructed using global pooling and linear mapping to obtain image-level global quality weights for two modalities:

[0197]

[0198] in, and These represent the global confidence levels of the infrared and visible light modes in the current image as a whole.

[0199] Furthermore, a local quality estimation branch is constructed, incorporating the infrared prior map and the aligned confidence map to obtain spatially relevant local quality weights:

[0200]

[0201] in, and These represent the local mass weights of the infrared mode and the visible light mode at different spatial locations, respectively.

[0202] The global quality weights and local quality weights are jointly modeled, and alignment confidence and visible light reliability constraints are imposed on the visible light weights:

[0203]

[0204] To ensure the comparability of the two modal weights, the infrared weights and visible light weights are normalized:

[0205]

[0206] Therefore, the basic fusion feature is represented as:

[0207]

[0208] Considering that information loss may still exist in the high-response infrared prior region due to modal differences and local alignment errors, a priori residual compensation branch is further introduced. This branch generates residual compensation terms based on the differences between infrared features and visible light alignment features, combined with the infrared prior map.

[0209]

[0210] The final fusion result is expressed as follows:

[0211]

[0212] in, Indicates the first Fusion features at various scales. Through the aforementioned quality-aware cross-modal fusion process, the network can dynamically allocate fusion weights based on the reliability of the infrared and visible light modalities in the current image and current spatial location, and enhance effective feature representation in target-related regions.

[0213] S7. Construct a multi-scale small target detection head and output the detection results.

[0214] like Figure 10 As shown, the fused multi-scale features are input into the P2–P5 four-scale detection structure. Let the fused feature pyramid obtained after cross-modal feature modeling be:

[0215]

[0216] in, Indicates the first The features are fused across multiple scales. Layer P2 provides high-resolution spatial detail information and is mainly used for extremely small target detection; layer P3 takes into account both semantic and spatial information and is used for small target detection; layer P4 is used for medium-scale target detection; and layer P5 is used for larger targets and global semantic modeling.

[0217] like Figure 11 As shown, a decoupled detection head is used for prediction at each scale. For any scale, the fused features... First, channel transformation is performed through a shared feature adaptation layer to obtain the input features of the detection head:

[0218]

[0219] in, This represents the feature adaptation function composed of convolutional layers. Subsequently, Input the bounding box regression branch, the category prediction branch, and the object prediction branch respectively, and obtain the corresponding outputs:

[0220]

[0221] in, This indicates the bounding box regression results. Indicates the category prediction result, This indicates the result of a targeted prediction.

[0222] like Figure 12 As shown, to enhance the target response in small target areas, target refinement branches are introduced in layers P2 and P3 to perform secondary adjustments on the initial target prediction:

[0223]

[0224] in, This represents the Sigmoid activation function. This represents a lightweight convolutional refinement network.

[0225] Furthermore, an infrared prior bias term is introduced into the target-oriented branch. Let the first... The infrared prior image of the layer is Then, the targeted prediction is expressed as:

[0226]

[0227] in, For learnable weight parameters, This represents the transformation that maps prior probabilities to log-odds space. In this way, infrared prior information is further passed from the feature layer to the detection decision layer, enabling potential target regions to receive additional response support in the final target prediction.

[0228] Finally, based on the bounding box regression results, category prediction results, and target prediction results output by the detection heads at each scale, the category, confidence level, and bounding box location of the UAV target are obtained.

[0229] Example 2:

[0230] To verify the effectiveness of the method of the present invention, this embodiment constructs an infrared-visible light dual-modal UAV target dataset and conducts experimental verification on a self-built dataset and a public dataset.

[0231] In this embodiment, an infrared-visible light dual-modal UAV gimbal monitoring system is constructed, and UAV target data in a real field environment is collected based on this system. The self-built dataset contains 5000 pairs of infrared-visible light images, with the training, validation, and test sets divided in a 7:1:2 ratio, containing 3500, 500, and 1000 pairs of images respectively. The dataset contains only one category of UAVs, and the infrared image resolution is [resolution missing]. The resolution of the visible light image is This dataset is characterized by a high proportion of small targets, significant modal space shift, coverage of complex environments, and significant differences in modal resolution.

[0232] To further verify the generalization ability of the method of the present invention, this embodiment also selects approximately 5000 pairs of images from the publicly available dual-modal UAV detection dataset Anti-UAV-RGBT for experiments, where the training set, validation set, and test set are also divided in a 7:1:2 ratio. This publicly available dataset contains both infrared and visible light modalities, covering daytime, nighttime, low-light, and complex background scenes, and can be used to verify the adaptability of the method of the present invention under different modal distribution conditions.

[0233] In the experiment, the input images were uniformly processed as follows: The system aims to maintain resolution while preserving the original aspect ratio, and avoids geometric distortions introduced by scaling through padding operations. Data augmentation strategies such as random flipping, scaling, and brightness perturbation are employed during training. Considering the small size of UAV targets, the use of strong data augmentation methods is constrained to prevent the target from shrinking further during augmentation.

[0234] This embodiment uses precision, recall, average precision (AP), and mean average precision (mAP) as evaluation metrics. Precision and recall are defined as follows:

[0235]

[0236] in, This indicates the number of targets that were correctly detected. Indicates the number of false positives. This indicates the number of targets that were missed.

[0237] Average precision (AP) is defined as the area under the Precision-Recall curve:

[0238]

[0239] The mean accuracy (mAP) is defined as follows:

[0240]

[0241] in, Indicates the number of target categories. Indicates the first The average accuracy of the target class. In this embodiment, the focus is on... and Two indicators.

[0242] On a self-built dataset, the Precision, Recall, and... of the PPMF-YOLO method of this invention... and The accuracy rates were 94.6%, 95.1%, 96.3%, and 46.2%, respectively. Compared with the single-mode infrared YOLOv11-IR, the method of this invention achieves better precision, recall, and... and The accuracy was improved by 1.0, 1.2, 1.2 and 2.1 percentage points respectively. Compared with the simple dual-modal fusion methods YOLOv11-Concat and YOLOv11-Sum, the method of this invention achieves higher detection accuracy, indicating that simple stitching or element-by-element addition is insufficient to fully handle the spatial misalignment and quality differences between infrared and visible light images.

[0243] On the publicly available Anti-UAV-RGBT dataset, the Precision, Recall, and and The accuracy rates were 96.8%, 94.5%, 97.3%, and 57.2%, respectively. Compared to the single-mode infrared YOLOv11-IR, the method of this invention achieves [the desired results]. The improvement is 3.4 percentage points; compared with the dual-modal comparison method UAVD-Mamba, the method of this invention is... The improvement was 5.5 percentage points. These results demonstrate that the method of the present invention can not only improve the ability to correctly detect targets, but also improve the quality of the detection box localization.

[0244] like Figure 13 and Figure 14As shown, the method of this invention can reliably detect UAV targets under different background complexities and lighting conditions. In low-light scenarios, target information in visible light images is relatively blurry, while the method of this invention can accurately locate the target region by means of infrared saliency prior and quality-sensing cross-modal fusion mechanism, demonstrating strong small target detection capability and adaptability to complex environments.

[0245] like Figure 15 As shown, the infrared prior model can produce a relatively stable response to potential UAV target areas on the self-built dataset, indicating that infrared prior information can provide effective position guidance for the subsequent cross-modal fusion module. Even if the infrared prior model has a small number of false detections, the subsequent quality-sensing cross-modal fusion module and detection head can still combine visible light structural information, infrared thermal features, and multi-scale semantic features for further screening and correction, thereby reducing the impact of erroneous priors on the final detection results.

[0246] like Figure 16 As shown, in scenarios with interference from similar heat sources, the method of this invention can effectively distinguish between UAV targets and targets with similar heat sources, reducing false detections caused by heat source similarity. This result demonstrates that the method of this invention does not rely solely on infrared thermal response for detection, but rather achieves target discrimination through the fusion of infrared priors, visible light structural information, and quality perception.

[0247] like Figure 17 As shown, the target region obtained after intermodal position mapping can fall well near the real target and maintain a high degree of consistency with the real bounding box in spatial position. This result indicates that the prior constraint cross-modal local alignment and position compensation mechanism proposed in this invention can infer the corresponding target region in another modality based on the target position in the infrared prior image without relying on manual bounding boxes during the inference stage, thereby alleviating the spatial misalignment problem caused by the non-strict registration of infrared and visible light images.

[0248] To further verify the effectiveness of each key module, this embodiment conducts ablation experiments on a self-built dataset. The base model is... and The percentages were 95.1% and 44.1% respectively; after successively introducing low-light visible light modeling, infrared prior access, prior-guided feature enhancement, cross-modal alignment and position compensation, quality-aware fusion, P2–P5 detection layers, and prior detection optimization, the complete model achieved [a certain percentage]. and The accuracy rates were improved to 96.3% and 46.2% respectively. Among them, the cross-modal alignment and position compensation module and the quality perception fusion module played an important role in improving positioning accuracy.

[0249] The experimental results above demonstrate that the method of the present invention can effectively improve the target detection performance of UAVs under low light, complex background, modal unregistration and small target conditions through the synergistic effect of low-light visible light robust modeling, infrared prior guidance, cross-modal local alignment, position compensation, quality perception fusion and multi-scale small target detection head.

[0250] The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention. Other changes, substitutions, or modifications that do not depart from the concept and scope of the present invention should be included within the protection scope of the present invention.

Claims

1. An infrared-prior-driven infrared-visible light fusion UAV target detection method, comprising the following steps: S1. Acquire infrared and visible light images of the same UAV observation scene, using the infrared image as the source of target salience information and the visible light image as the source of target structure and texture information. S2. Perform low-light visible light robust modeling on the visible light image to obtain robust visible light information and visible light reliability map; S3. Input the infrared image and the robust visible light information into a dual-branch feature encoding network to obtain multi-scale infrared features and multi-scale visible light features; at the same time, input the infrared image into a pre-trained infrared prior model to generate an infrared saliency prior map and construct an infrared prior pyramid. S4. Based on the infrared prior pyramid, perform prior-guided small target feature enhancement on the multi-scale infrared features to obtain the prior-enhanced multi-scale infrared features. S5. Based on the infrared prior pyramid and the visible light reliability map, perform cross-modal local alignment and position compensation of the prior-enhanced multi-scale infrared features and multi-scale visible light features with prior constraints to obtain the compensated multi-scale visible light aligned features. S6. Based on the modal global quality, local spatial quality, alignment confidence and the visible light reliability map, perform quality-aware cross-modal fusion on the prior enhanced multi-scale infrared features and the compensated multi-scale visible light alignment features to obtain the fused multi-scale features. S7. Input the fused multi-scale features into the multi-scale small target detection head, and output the category, confidence level and bounding box position of the UAV target through the prior-guided detection response optimization mechanism.

2. The infrared-visible light fusion UAV target detection method driven by infrared prior as described in claim 1, characterized in that the specific method of step S2 is as follows: S2.1: Let the input visible light image be... First, the input visible light image is illuminated using an illuminance estimation network to obtain a single-channel illuminance map: ； in, This represents the illuminance estimation network. This represents the relative brightness distribution at various spatial locations in the input visible light image. These represent the height and width of the image, respectively. The original visible light image is brightness compensated based on the single-channel illuminance map and learnable gain parameters to obtain a preliminary enhanced image. : ； in, For learnable gain parameters, This is the illuminance map after stabilization. This represents element-wise multiplication; S2.2: To reduce noise amplification and structural loss caused by simple brightness enhancement, the original visible light image and the preliminary enhanced image are stitched together, and a structural compensation term is generated through a detail recovery network. ； ； in, Indicates channel splicing. This indicates details of network recovery. Indicates the details of the compensation item. This represents the enhanced visible light image; S2.3: Predict spatially relevant fusion weights based on the original visible light image, the enhanced visible light image, and the illuminance map: ； in, This represents a fusion coefficient prediction network. This indicates the degree of trust in the enhancement result at different spatial locations; based on the fusion weights, the original visible light image and the enhanced visible light image are adaptively fused to obtain robust visible light information. : ； S2.4: Explicitly estimate the quality of visible light information using a reliability estimation network to obtain a visible light reliability map. ； in, Represents a reliability estimation network. The visible light reliability map represents the reliability of visible light information in different spatial regions in UAV target detection tasks. The visible light reliability map is used to constrain the subsequent cross-modal local alignment process and to modulate the contribution of visible light features in quality-sensing cross-modal fusion.

3. The infrared-visible light fusion UAV target detection method driven by infrared prior as described in claim 1, characterized in that the specific method of step S3 is as follows: Let the input infrared image be The saliency prior map is obtained through the infrared prior model: ； in, This represents a pre-trained infrared prior model. This represents an infrared saliency prior map at the same scale as the input infrared image; regions with larger response values ​​in the infrared saliency prior map correspond to locations more likely to contain UAV targets. Since the detection network performs inference in a multi-scale feature space, the infrared saliency prior map is mapped to feature layers of different scales to construct an infrared prior pyramid: ； in, Indicates the relationship with the first Infrared prior maps with consistent spatial dimensions across multiple scale feature maps; through the infrared prior pyramid, infrared prior information can provide positional constraints in the processes of multi-scale feature enhancement, cross-modal alignment, quality-aware fusion, and detection response optimization.

4. The infrared-visible light fusion UAV target detection method driven by infrared prior as described in claim 1, characterized in that, The specific method for step S4 is as follows: S4.1: Let the first... Layer infrared characteristics are The corresponding infrared prior image at the scale is ; These represent the number of channels, height, and width of the infrared feature, respectively. pass Convolution maps the infrared prior map to a prior gating map that matches the number of infrared feature channels at the current scale: ； in, This represents a scale-dependent linear mapping function. This represents the Sigmoid activation function. This represents the prior gating graph at the current scale; The infrared features are spatially reweighted based on the prior gating map to obtain the reweighted infrared features. : ； The spatial reweighting enhances the feature response of the target-related region and suppresses background interference. S4.2: Input the reweighted infrared features and prior gated map into the convolutional residual enhancement branch to generate target-related residual terms. : ； in, Indicates the first Convolutional residual enhancement branches at various scales; S4.3: Fuse the main branch features and the residual branch, and output the enhanced infrared features based on the prior through the feature refinement function. : ； in, This represents a feature refinement function composed of convolutional layers, used for channel reshaping and semantic reconstruction of enhanced infrared features.

5. The infrared-visible light fusion UAV target detection method driven by infrared prior as described in claim 1, characterized in that, The specific method for step S5 is as follows: S5.1: For any scale, the enhanced infrared features, the visible light features of the corresponding scale, and the prior gating map are stitched together and input into the migration estimation network to obtain a two-dimensional local migration field. : ； in, Indicates the first A offset estimation network at multiple scales This is a learnable scale parameter used to control the range of the offset field amplitude. Indicates the first Visible light characteristics at various scales; The two-dimensional local offset field is mapped to a normalized coordinate space, and the visible light features are resampled using a bilinear sampling operation to obtain the initially aligned visible light features. : ； in, Indicates a bilinear sampling operation; S5.2: Jointly model the enhanced infrared features, the preliminarily aligned visible light features, and the prior gating graph to obtain the alignment confidence map: ； in, This indicates the reliability of the alignment results between infrared and visible light features at the current location; S5.3: By analyzing the residual relationship between the enhanced infrared features and the pre-aligned visible light features, and combining it with the prior gating graph, a position compensation map is generated. : ； in, Indicates the location compensation branch; S5.4: Modulate the position compensation map using the alignment confidence map to obtain the compensated visible light alignment features: 。 6. The infrared-visible light fusion UAV target detection method driven by infrared prior as described in claim 1, characterized in that the specific method of step S6 is as follows: S6.1: Let the first... The layer input features are the enhanced infrared features from prior knowledge. Compensated visible light alignment features Infrared prior image Alignment Confidence Plot and visible light reliability map ; Global pooling and linear mapping are used to construct a global quality estimation branch, resulting in image-level global quality weights for two modalities: ； in, and These represent the global confidence levels of the infrared and visible light modes in the current image as a whole; S6.2: Construct a local quality estimation branch, and introduce infrared prior maps and aligned confidence maps into the fusion weight calculation: ； in, and Represents the local mass weights related to spatial location; S6.3: Jointly model the global quality weights and local quality weights, and impose alignment confidence and visible light reliability constraints on the visible light weights: ； ； S6.4: To ensure the comparability of the weights for the two modes, the infrared weights and visible light weights are normalized: ； ； Therefore, the basic fusion feature is represented as: ； S6.5: Introduce a priori residual compensation branch in the fusion module, generating residual compensation terms based on the difference between infrared features and visible light alignment features, combined with the infrared prior image. ； The final fusion result is expressed as follows: ； in, Indicates the first Fusion features at various scales. Through the quality-aware cross-modal fusion process, fusion weights can be dynamically allocated based on the reliability of different modalities in the current image and current spatial location, and target-related residual compensation can be enhanced in the infrared prior high-response region.

7. The infrared-visible light fusion UAV target detection method driven by infrared prior as described in claim 1, characterized in that the specific method of step S7 is as follows: S7.1: Let the fused feature pyramid obtained after cross-modal feature modeling be: ； in, Indicates the first The multi-scale small target detection head employs a four-scale detection structure of P2, P3, P4, and P5, where the P2 layer is used for extremely small target detection, the P3 layer for small target detection, the P4 layer for medium-scale target detection, and the P5 layer for larger targets and global semantic modeling. S7.2: Fusion features for any scale First, channel transformation is performed through a shared feature adaptation layer to obtain the input features of the detection head: ； in, This represents the feature adaptation function composed of convolutional layers. Subsequently, Input the bounding box regression branch, the category prediction branch, and the object prediction branch respectively, and obtain the corresponding outputs: ； ； ； in, This indicates the bounding box regression results. Indicates the category prediction result, Indicates the result of targeted prediction; S7.3: Introduce target-oriented refinement branches in layers P2 and P3 to perform secondary adjustments on the initial target-oriented predictions: ； in, This represents the Sigmoid activation function. This represents a lightweight convolutional thinning network; S7.4: Introduce an infrared prior bias term in the target branch, let the first... The infrared prior image of the layer is Then, the targeted prediction is expressed as: ； in, For learnable weight parameters, This represents the transformation that maps prior probabilities to logarithmic probability space. Through the aforementioned target bias term, infrared prior information is directly involved in the detection decision process, enhancing the target response of small target areas in the final prediction stage. S7.5: During the training phase, the detection network is trained using paired infrared images, visible light images, and UAV target bounding box annotations. During the inference phase, the infrared and visible light images to be detected are input, and UAV target detection is completed by a combination of infrared prior models, cross-modal local alignment and position compensation mechanisms, quality-aware cross-modal fusion mechanisms, and multi-scale small target detection heads, without relying on manually annotated bounding boxes or strict pixel-level registration results.

Images