Progressive prior guided fusion method and system for visible and infrared target detection
By adopting a progressive prior-guided fusion method, the modal quality prior is explicitly modeled and the single-modal features are gradually corrected, which solves the problem of insufficient detection accuracy of existing RGBT target detection methods in complex lighting and occlusion scenarios, and achieves higher detection accuracy and robustness.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- UNIV OF SCI & TECH BEIJING
- Filing Date
- 2026-03-31
- Publication Date
- 2026-06-05
AI Technical Summary
Existing RGBT object detection methods have limitations in multimodal fusion due to their single-step fusion paradigm. They lack targeted correction of single-modal features and lack global prior guidance for dynamic convolution kernels, making it difficult to improve detection accuracy, especially in complex lighting and occlusion scenarios.
A progressive prior-guided fusion method is adopted. Through a modality prior generation module and a prior-guided dynamic convolution module, modality quality priors are explicitly modeled, single-modality features are gradually corrected, and modality-specific dynamic convolution kernels are generated by using multi-scale enhancement and channel modulation strategies to achieve feature feedback purification and fusion.
It significantly improves detection performance in complex scenes, enhances detection accuracy, strengthens the ability to detect small targets, solves thermal background noise in infrared images and glare interference in visible light images, and achieves higher robustness and accuracy.
Smart Images
Figure CN122156881A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of computer vision and image processing technology, and specifically refers to a progressive prior-guided fusion method and system for visible light and infrared target detection. Background Technology
[0002] With the development of intelligent applications, target detection technology plays a crucial role in fields such as autonomous driving and security monitoring. In practical applications, single visible light (RGB) sensors are easily affected by environmental factors such as changes in lighting conditions, low-light conditions at night, strong glare, and smoke obstruction, leading to a significant deterioration in image quality and consequently causing missed or false alarms. In contrast, thermal infrared (IR) imaging utilizes the thermal radiation of objects, enabling all-weather operation and unaffected by lighting conditions, but its images lack texture details and color information. Therefore, fusing visible light and infrared (RGBT) images to leverage their complementarity has become a mainstream research direction for improving detector robustness.
[0003] Existing RGBT object detection methods primarily focus on designing complex fusion modules to enhance the aggregation of cross-modal features. For example, early cyclic fusion and channel-wise gating methods attempted to reduce modal differences; in recent years, attention-based methods have been widely used, implicitly learning modal weights to distinguish the importance of different modalities; in addition, some methods have introduced illumination sensing or confidence estimation modules to assist in fusion decisions.
[0004] However, existing technologies still have the following major drawbacks when handling multimodal fusion: First, the limitation of the "one-step fusion" paradigm: Most existing methods adopt a one-way process of "feature extraction - fusion - detection". That is, after the single-modal features are extracted, they are directly aggregated. The quality information (Modality Quality) generated during the fusion process regarding "which modality is more reliable" is only used to weight the current fusion result and is immediately discarded, never fed back to the single-modal branch.
[0005] Second, there is a lack of targeted correction for single-modal features: Due to the lack of a feedback mechanism, the original single-modal features (such as infrared features including thermal halo effects or visible light features including glare) are directly used in fusion without being cleaned and calibrated. This unoptimized single-modal representation limits the discriminative power of the final fused features, making it difficult to further improve detection accuracy in scenarios with severe illumination imbalance or local occlusion.
[0006] Third, dynamic convolution kernels lack global prior guidance: Although some existing techniques attempt to use dynamic convolution to enhance features, the generation of their convolution kernels usually only depends on local differential features, lacking global cross-modal context priors, making it difficult to capture fine intermodal relationships.
[0007] Therefore, there is an urgent need for a target detection method that can explicitly model modal quality priors and use these priors to progressively correct single-modal features. Summary of the Invention
[0008] To address the technical problems existing in the prior art, the present invention provides a progressive prior-guided fusion method and system for visible light and infrared target detection, the technical solution of which is as follows: On the one hand, a progressive prior-guided fusion method for visible light and infrared target detection is provided, which includes: S1. Acquire paired visible light and infrared images in the same scene; S2. Input the visible light image and infrared image into the dual-stream backbone network of the detector respectively, and extract the single-mode visible light features and infrared features of multiple stages respectively; S3. Input the single-modal visible light features and infrared features extracted from each key stage of the backbone network into the progressive prior-guided fusion module PGPF; The PGPF module includes a modal prior generation module MPG and a prior-guided dynamic convolution module PGDC. The MPG performs initial fusion of the single-modal visible light features and infrared features to generate a modal quality prior that includes modal quality information. The PGDC enhances the modal quality prior and generates modality-specific dynamic convolutional kernels. It corrects the single-modal features and the enhanced modal quality prior, enhances and fuses the corrected single-modal features with the modal quality prior to obtain the final fused features, and replaces the original single-modal features with the corrected single-modal features as the input of the backbone network in the next stage. S4. Input the final fused features from each key stage into the encoder of the detector for feature reorganization. Use the decoder and detector head to regress and classify the reorganized features, and output the detection results including target category and location coordinates.
[0009] Optionally, the processing procedure of the modal prior generation module MPG in S3 specifically includes: S31. Using the normal perturbation module, the extracted visible light features are respectively... and infrared features The distribution is perturbed to simulate different scenarios in order to enhance the model's generalization ability; S32. Expand the receptive field and enhance small target features using the multi-scale enhancement module MSEB to obtain enhanced features. and ; S33, Based on Enhanced Features and MPG uses two parallel branches: a shared feature branch to extract common semantics and a feature difference branch to extract complementary information. The shared feature branch extracts mode-invariant features using convolutional layers with shared weights, and enhances the target feature responses at both the channel and spatial scales using an attention mechanism through a convolutional block attention module (CBAM) to obtain common features. The single-modal attention weights generated by CBAM This will be used for subsequent discretization of modal embedding module features; The feature difference branch calculates the feature difference. Differential features are obtained through the CNA convolutional module and the CBAM convolutional block attention module. ; S34. The common features and differences The modal quality priors, including modal quality information, are added together and fused through the CNA convolution module. , Explicit representation indicates the reliability of the fusion feature in terms of space and channel location.
[0010] Optionally, the process of enhancing the multi-scale block MSEB specifically includes: For input single-modal features MSEB redirects it to In the parallel dilated convolution branches, the first... The expansion rate used for each branch is set as follows: The output characteristics of each branch are calculated as follows:
[0011] in, Indicates the expansion rate of Convolution kernel; A channel modulation strategy is introduced to avoid the dilution of small target features by large receptive field branches, and the number of channels in each branch is dynamically allocated. As the expansion rate increases, the number of channels decreases. The calculation formula is:
[0012] in, The total number of channels for the input features; Finally, the outputs of all branches are concatenated along the channel dimension to obtain the enhanced feature. .
[0013] Optionally, the processing procedure of the prior-guided dynamic convolution module PGDC in S3 specifically includes: S35, Modal quality prior Combined with modal embeddings generated by the modal embedding module MEB, an enhanced multimodal quality prior is constructed. ; S36, Use Guiding the correction of single-modal features, including: Generate key features The It includes not only the prior knowledge of the current stage, but also the updated prior knowledge from the previous stage:
[0014] Using projection matrix Single-modal features Transform into Q, and extract key features Using projection matrix Transform K, calculate the attention map between the two, and pass it through a linear layer. Projection to generate modality-specific dynamic convolution kernels :
[0015] in, Let K be the feature dimension; Utilizing the generated dynamic convolution kernel Modify the features:
[0016] in, This is the corrected single-modal feature. For the corrected modal quality prior; S37. Modify the single-modal features Each with the corresponding modality The features are concatenated, fused using the CNA convolutional module, then fused between modalities using the CNA convolutional module, and finally rescaled using the LayerScale module for residual connections to obtain the final fused features. :
[0017] in, This represents the LayerScale operation, used to stabilize training and preserve the semantics of the original features. This represents a convolution operation used to fuse features.
[0018] Optionally, the processing procedure of the modal embedding module (MEB) specifically includes: A soft discretization mechanism was designed to transform the attention weights generated by the CBAM convolutional block attention module in the MPG shared feature branch into explicit semantic states, including: Obtaining Spatial Attention Weights , preset Anchor point states will divide the interval Divide evenly and calculate The Euclidean distances to the states of each anchor point are converted into probability distributions using the Softmax function. This maps continuous weight values to discrete modal quality states. The features are mapped to modality embedding features via a multilayer perceptron (MLP). and modal quality prior Adding them together yields an enhanced multimodal prior. .
[0019] Optionally, the training process of the detector specifically includes: A special two-stage training strategy is adopted to ensure the reliability of modal priors; In the first phase, PGDC is removed, and the modal quality prior generated by MPG is directly applied. As the final feature input to the detection head, MPG is forced to complete the target detection task solely based on prior features without any assistance. This forces MPG to learn the most discriminative multimodal features, and only MPG parameters are optimized at this stage. In the second stage, joint fine-tuning is performed by loading the MPG parameters trained in the first stage, connecting to PGDC, and conducting end-to-end training of the entire network.
[0020] Optionally, during the second phase of training, the learning rate of MPG is set to 1 / 10 of that of other modules to prevent disruption of the learned prior distribution.
[0021] On the other hand, a progressive priori-guided fusion visible light and infrared target detection system is provided, the system comprising: The image acquisition module is configured to acquire paired visible light and infrared images; The feature extraction module is configured to extract multi-scale features using a dual-stream backbone network. The progressive prior-guided fusion module is configured to: perform the steps as described in any one of claims 1 to 7, construct a quality prior through the modality prior generation module, and modify and fuse features through the prior-guided dynamic convolution module; The target detection module is configured to output detection results based on the fused features.
[0022] On the other hand, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, which is loaded and executed by the processor to implement the aforementioned progressive prior-guided fusion visible light and infrared target detection method.
[0023] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, the at least one instruction being loaded and executed by a processor to implement the above-described progressive prior-guided fusion visible light and infrared target detection method.
[0024] The beneficial effects of the technical solution provided by this invention include at least the following: 1. Significantly improved detection performance in complex scenes: By explicitly modeling modal quality priors, this invention achieves detection accuracy (mAP) superior to existing state-of-the-art methods on the LLVIP (low-light pedestrian), M3FD (multi-scene), and FLIR (regular) datasets.
[0025] 2. Achieves "feedback-based" feature purification: This invention breaks through the limitations of traditional single-step fusion by using the high-quality prior after fusion to guide the correction of single-modal features. This mechanism can effectively suppress thermal background noise in infrared images and glare interference in visible light images, making the features input to the next stage purer.
[0026] 3. Enhanced detection capability for small targets: By introducing multi-scale enhancement and channel modulation strategies in the modal prior generation module, this invention effectively balances deep semantics and shallow details, solves the problem that large receptive fields easily lose information about small targets, and significantly reduces the false negative rate of distant small targets.
[0027] 4. Modal quality representation: By using soft discretization techniques, continuous attention weights are transformed into discrete semantic states, making the generation of dynamic convolutional kernels more accurate and enabling adaptive handling of modal conflicts and misalignment issues. Attached Figure Description
[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0029] Figure 1This is a flowchart of a progressive priori-guided fusion visible light and infrared target detection method provided in an embodiment of the present invention; Figure 2 This is a general block diagram of a progressive priori-guided fusion visible light and infrared target detection method provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the processing procedure of the Modal Prior Generation (MPG) module provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the processing of the priori guided dynamic convolution module PGDC provided in an embodiment of the present invention; Figure 5 This is a block diagram of a progressive priori-guided fusion visible light and infrared target detection system provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0030] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.
[0031] This invention provides a progressive priori-guided fusion method for visible light and infrared target detection, which can be implemented by an electronic device, such as a terminal or a server. Figure 1 The flowchart of this method is shown below. Figure 2 The diagram shown is an overall block diagram of the method. The processing flow may include the following steps: S1. Acquire paired visible light and infrared images in the same scene; Images of the same scene are acquired using a dual-modal camera device (such as a device that includes a visible light lens and an infrared thermal imaging lens). The acquired visible light images are then analyzed. and infrared images Preprocessing is performed, including image registration and size normalization, to ensure spatial alignment. In this embodiment, the resolution of the input image can be set to [resolution value missing]. (Refer to FLIR dataset settings).
[0032] S2. Input the visible light image and infrared image into the dual-stream backbone network of the detector respectively, and extract the single-mode visible light features and infrared features of multiple stages respectively; The detector base of this invention includes a dual-stream backbone network, an encoder, a decoder, and a detection head. In this embodiment, both the visible light branch and the infrared branch adopt a D-FINE backbone network structure, including four feature extraction stages (Stage 1 to Stage 4). The visible light image... Input visible light branch, then input infrared image Input infrared branch. In the... Each stage ( Single-mode visible light features were extracted respectively. and infrared features In this process, the size of the feature map decreases progressively with each stage, while the number of channels increases progressively. For example, the output feature map of Stage 1 is 1 / 4 the size of the original image, and the output feature map of Stage 4 is 1 / 32 the size of the original image.
[0033] S3. The single-modal visible light and infrared features extracted from each key stage of the backbone network (specifically after Stage 2, Stage 3, and Stage 4) are input into the Progressive Prior Guided Fusion Module (PGPF). stage( ), PGPF receives the raw single-modal features of the current stage. and (as input) This invention adds PGPF to a dual-stream backbone network.
[0034] like Figure 2 As shown, the PGPF module includes a modal prior generation module MPG and a prior-guided dynamic convolution module PGDC. The MPG performs initial fusion of the single-modal visible light features and infrared features to generate a modal quality prior that includes modal quality information. The PGDC enhances the modal quality prior and generates modality-specific dynamic convolutional kernels. It corrects the single-modal features and the enhanced modal quality prior, enhances and fuses the corrected single-modal features with the modal quality prior to obtain the final fused features, and replaces the original single-modal features with the corrected single-modal features as the input of the backbone network in the next stage. Optionally, such as Figure 3 As shown, the processing procedure of the Modal Prior Generation (MPG) module in S3 specifically includes: S31. Using the normal perturbation module, the extracted visible light features are respectively... and infrared features The distribution is perturbed to simulate different scenarios in order to enhance the model's generalization ability; S32. Expand the receptive field and enhance small target features using the multi-scale enhancement module MSEB to obtain enhanced features. and ; Optionally, such as Figure 3 As shown, the processing procedure of the multi-scale enhanced block MSEB specifically includes: For input single-modal features MSEB redirects it to In the parallel dilated convolution branches, the first... The dilation rate used for each branch is set to... The output characteristics of each branch are calculated as follows:
[0035] in, Indicates the expansion rate of Convolution kernel; A channel modulation strategy is introduced to avoid the dilution of small target features by large receptive field branches, and the number of channels in each branch is dynamically allocated. As the expansion rate increases, the number of channels decreases. The calculation formula is:
[0036] in, The total number of channels for the input features; This design allows the low expansion rate branch (focusing on small targets) to receive more channel weights, while the high expansion rate branch (focusing on the broader context) receives fewer channels, achieving a balance between scale and semantics.
[0037] Finally, the outputs of all branches are concatenated along the channel dimension to obtain the enhanced feature. .
[0038] S33, Based on Enhanced Features and MPG uses two parallel branches: a shared feature branch to extract common semantics and a feature difference branch to extract complementary information. The shared feature branch extracts mode-invariant features using convolutional layers with shared weights, and enhances the target feature responses at both the channel and spatial scales using an attention mechanism through a convolutional block attention module (CBAM) to obtain common features. The single-modal attention weights generated by CBAM This will be used for subsequent discretization of modal embedding module features; The feature difference branch calculates the feature difference. Differential features are obtained through the CNA convolutional module and the CBAM convolutional block attention module. (This step can capture complementary information unique to a single mode (such as heat sources in infrared or textures in visible light)); S34. The common features and differences The modal quality priors, including modal quality information, are added together and fused through the CNA convolution module. , Explicit representation indicates the reliability of the fusion feature in terms of space and channel location.
[0039] Optionally, such as Figure 4 As shown, the processing procedure of the prior-guided dynamic convolution module PGDC in S3 specifically includes: S35, Modal quality prior Combined with modal embeddings generated by the modal embedding module MEB, an enhanced multimodal quality prior is constructed. ; Optionally, such as Figure 4 As shown, the processing procedure of the Modal Embedding Module (MEB) specifically includes: A soft discretization mechanism was designed to transform the attention weights generated by the CBAM convolutional block attention module in the MPG shared feature branch into explicit semantic states, including: Obtaining Spatial Attention Weights , preset Anchor point states will divide the interval Divide evenly and calculate The Euclidean distances to the states of each anchor point are converted into probability distributions using the Softmax function. This maps continuous weight values to discrete modal quality states (e.g., fully reliable, partially occluded, completely failed). The features are mapped to modality embedding features via a multilayer perceptron (MLP). and modal quality prior Adding them together yields an enhanced multimodal prior. .
[0040] This invention maps continuous weight values to discrete modal quality states, introducing an inductive bias to more robustly represent semantic states such as "fully reliable" and "partially failed." The inductive bias reflects objective laws: while interference in real-world scenes (such as rain or occlusion) is complex, modal degradation (referring to multimodal image degradation in complex scenes, such as severe loss of visible light image information at night, which is visible light image degradation) can usually be categorized into a limited number of classes. Setting these as discrete states essentially injects prior knowledge into the model that "modal quality has clustering properties," guiding it to output a consistent probability distribution in similar scenarios. Furthermore, since continuous random noise always exists at the image's underlying layer, directly passing continuous weights can easily cause performance fluctuations. The discretization processing in this invention effectively filters out these meaningless continuous numerical fluctuations, allowing subsequent MLPs to only process discrete state probability distributions, thereby significantly reducing noise interference and significantly improving the system's robustness.
[0041] S36, Use Guiding the correction of single-modal features, including: Generate key features The It includes not only the prior knowledge of the current stage, but also the updated prior knowledge from the previous stage (in order to maintain progressive consistency):
[0042] Using projection matrix Single-modal features Transform into a Query (Q) and extract key features Using projection matrix Transform K (Key), calculate the attention graph between them, and pass it through a linear layer. Projection to generate modality-specific dynamic convolution kernels :
[0043] in, Let K be the feature dimension; Utilizing the generated dynamic convolution kernel Modify the features:
[0044] in, This is the corrected single-modal feature. For the corrected modal quality prior; S37. Modify the single-modal features Each with the corresponding modality The features are concatenated, fused using the CNA convolutional module, then fused between modalities using the CNA convolutional module, and finally rescaled using the LayerScale module for residual connections to obtain the final fused features. :
[0045] in, This represents the LayerScale operation, used to stabilize training and preserve the semantics of the original features. This represents a convolution operation used to fuse features.
[0046] S4. Input the final fused features from each key stage into the encoder of the detector for feature reorganization. Use the decoder and detector head to regress and classify the reorganized features, and output the detection results including target category and location coordinates.
[0047] Optionally, the training process of the detector specifically includes: A special two-stage training strategy is adopted to ensure the reliability of modal priors; Phase 1 (Prior Warm-up): Remove PGDC and directly use the modal quality prior generated by MPG. As the final feature input to the detection head, MPG is forced to complete the target detection task solely based on prior features without any assistance. This forces MPG to learn the most discriminative multimodal features, and only MPG parameters are optimized at this stage. In the second stage, joint fine-tuning is performed by loading the MPG parameters trained in the first stage, connecting to PGDC, and conducting end-to-end training of the entire network.
[0048] Optionally, during the second phase of training, the learning rate of MPG is set to 1 / 10 of that of other modules to prevent disruption of the learned prior distribution.
[0049] The total loss function for overall training of the detector in this embodiment of the invention Defined as:
[0050] in, For generalized IoU regression loss, Varifocal classification loss, and To decouple the distillation focus loss from the fine-grained localization loss (derived from the D-FINE detector), this method is used to improve the precision of bounding box regression.
[0051] like Figure 5As shown, this embodiment of the invention also provides a progressive priori-guided fusion visible light and infrared target detection system, the system comprising: The image acquisition module 510 is configured to acquire paired visible light and infrared images; Feature extraction module 520 is configured to extract multi-scale features using a dual-stream backbone network; The progressive prior-guided fusion module 530 is configured to: execute the above-described progressive prior-guided fusion method for visible light and infrared target detection, construct a quality prior through a modality prior generation module, and modify and fuse features through a prior-guided dynamic convolution module; The target detection module 540 is configured to output detection results based on the fused features.
[0052] The progressive prior-guided fusion visible light and infrared target detection system provided in this embodiment of the invention has a functional structure that corresponds to the progressive prior-guided fusion visible light and infrared target detection method provided in this embodiment of the invention, and will not be described again here.
[0053] Figure 6 This is a schematic diagram of the structure of an electronic device 600 provided in an embodiment of the present invention. The electronic device 600 may vary considerably due to different configurations or performance. It may include one or more central processing units (CPUs) 601 and one or more memories 602. The memory 602 stores at least one instruction, which is loaded and executed by the processor 601 to implement the steps of the aforementioned progressive priori guided fusion visible light and infrared target detection method.
[0054] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions that can be executed by a processor in a terminal to complete the aforementioned progressive a priori-guided fusion visible light and infrared target detection method. For example, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device.
[0055] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.
[0056] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A progressive prior-guided fusion method for visible light and infrared target detection, characterized in that, The method includes: S1. Acquire paired visible light and infrared images in the same scene; S2. Input the visible light image and infrared image into the dual-stream backbone network of the detector respectively, and extract the single-mode visible light features and infrared features of multiple stages respectively; S3. Input the single-modal visible light features and infrared features extracted from each key stage of the backbone network into the progressive prior-guided fusion module PGPF; The PGPF module includes a modal prior generation module MPG and a prior-guided dynamic convolution module PGDC. The MPG performs initial fusion of the single-modal visible light features and infrared features to generate a modal quality prior that includes modal quality information. The PGDC enhances the modal quality prior and generates modality-specific dynamic convolutional kernels. It corrects the single-modal features and the enhanced modal quality prior, enhances and fuses the corrected single-modal features with the modal quality prior to obtain the final fused features, and replaces the original single-modal features with the corrected single-modal features as the input of the backbone network in the next stage. S4. Input the final fused features from each key stage into the encoder of the detector for feature reorganization. Use the decoder and detector head to regress and classify the reorganized features, and output the detection results including target category and location coordinates.
2. The method according to claim 1, characterized in that, The processing procedure of the Modal Prior Generation (MPG) module in S3 specifically includes: S31. Using the normal perturbation module, the extracted visible light features are respectively... and infrared features The distribution is perturbed to simulate different scenarios in order to enhance the model's generalization ability; S32. Expand the receptive field and enhance small target features using the multi-scale enhancement module MSEB to obtain enhanced features. and ; S33, Based on Enhanced Features and MPG uses two parallel branches: a shared feature branch to extract common semantics and a feature difference branch to extract complementary information. The shared feature branch extracts mode-invariant features using convolutional layers with shared weights, and enhances the target feature responses at both the channel and spatial scales using an attention mechanism through a convolutional block attention module (CBAM) to obtain common features. The single-modal attention weights generated by CBAM This will be used for subsequent discretization of modal embedding module features; The feature difference branch calculates the feature difference. Differential features are obtained through the CNA convolutional module and the CBAM convolutional block attention module. ; S34. The common features and differences The modal quality priors, including modal quality information, are added together and fused through the CNA convolution module. , Explicit representation indicates the reliability of the fusion feature in terms of space and channel location.
3. The method according to claim 2, characterized in that, The process of enhancing the multi-scale block MSEB specifically includes: For input single-modal features MSEB redirects it to In the parallel dilated convolution branches, the first... The expansion rate used for each branch is set as follows: The output characteristics of each branch are calculated as follows: in, Indicates the expansion rate of Convolution kernel; A channel modulation strategy is introduced to avoid the dilution of small target features by large receptive field branches, and the number of channels in each branch is dynamically allocated. As the expansion rate increases, the number of channels decreases. The calculation formula is: in, The total number of channels for the input features; Finally, the outputs of all branches are concatenated along the channel dimension to obtain the enhanced feature. .
4. The method according to claim 2, characterized in that, The prior-guided dynamic convolution module PGDC processing in S3 specifically includes: S35, Modal quality prior Combined with modal embeddings generated by the modal embedding module MEB, an enhanced multimodal quality prior is constructed. ; S36, Use The correction of single-modal features includes: Generate key features The It includes not only the prior knowledge of the current stage, but also the updated prior knowledge from the previous stage: Using projection matrix Single-modal features Transform into Q, and extract key features Using projection matrix Transform K, calculate the attention map between the two, and pass it through a linear layer. Projection to generate modality-specific dynamic convolution kernels : in, Let K be the feature dimension; Utilizing the generated dynamic convolution kernel Modify the features: in, This is the corrected single-modal feature. For the corrected modal quality prior; S37. Modify the single-modal features Each with the corresponding modality The features are concatenated, fused using the CNA convolutional module, then fused between modalities using the CNA convolutional module, and finally rescaled using the LayerScale module for residual connections to obtain the final fused features. : in, This represents the LayerScale operation, used to stabilize training and preserve the semantics of the original features. This represents a convolution operation used to fuse features.
5. The method according to claim 4, characterized in that, The processing procedure of the modal embedding module (MEB) specifically includes: A soft discretization mechanism was designed to transform the attention weights generated by the CBAM convolutional block attention module in the MPG shared feature branch into explicit semantic states, including: Obtaining Spatial Attention Weights , preset Anchor point states will divide the interval Divide evenly and calculate The Euclidean distances to the states of each anchor point are converted into probability distributions using the Softmax function. This maps continuous weight values to discrete modal quality states. The features are mapped to modality embedding features via a multilayer perceptron (MLP). and modal quality prior Adding them together yields an enhanced multimodal prior. .
6. The method according to claim 1, characterized in that, The training process of the detector specifically includes: A special two-stage training strategy is adopted to ensure the reliability of modal priors; In the first phase, PGDC is removed, and the modal quality prior generated by MPG is directly applied. As the final feature input to the detection head, MPG is forced to complete the target detection task solely based on prior features without any assistance. This forces MPG to learn the most discriminative multimodal features, and only MPG parameters are optimized at this stage. In the second stage, joint fine-tuning is performed by loading the MPG parameters trained in the first stage, connecting to PGDC, and conducting end-to-end training of the entire network.
7. The method according to claim 6, characterized in that, In the second phase of training, the learning rate of MPG is set to 1 / 10 of that of other modules to prevent disruption of the learned prior distribution.
8. A progressive prior-guided fusion visible light and infrared target detection system, characterized in that, The system includes: The image acquisition module is configured to acquire paired visible light and infrared images; The feature extraction module is configured to extract multi-scale features using a dual-stream backbone network. The progressive prior-guided fusion module is configured to: perform the steps as described in any one of claims 1 to 7, construct a quality prior through the modality prior generation module, and modify and fuse features through the prior-guided dynamic convolution module; The target detection module is configured to output detection results based on the fused features.
9. An electronic device comprising a processor and a memory, wherein the memory stores at least one instruction, characterized in that, The at least one instruction is loaded and executed by the processor to implement the progressive prior-guided fusion visible and infrared target detection method as described in any one of claims 1-7.
10. A computer-readable storage medium storing at least one instruction, characterized in that, The at least one instruction is loaded and executed by the processor to implement the progressive prior-guided fusion visible and infrared target detection method as described in any one of claims 1-7.