A cross-modal prior-guided diffusion system for visible and infrared small target detection

Abstract

The application relates to the technical field of small target detection, in particular to a diffusion type visible light and infrared small target detection system guided by cross-modal priori, which comprises a data preprocessing module, a double-branch multi-scale feature extraction module, a frequency domain priori guided multi-modal feature enhancement module, a diffusion type detection module and a result output module, and a spatial distribution priori guided candidate frame updating submodule is arranged in the diffusion type detection module. The system dynamically adjusts cross-modal interaction and fusion weights by explicitly evaluating the effective information contribution of visible light and infrared in different scenes, so that higher detection precision is achieved. The priori guided candidate frame supplement strategy is used to constrain and resample the generation and supplement of candidate frames in each iteration, so that the newly added candidate frames are more concentrated in potential target areas, the effectiveness of the denoising process is improved, unnecessary calculation overhead is reduced, and the small target detection capability is greatly improved.

Description

Technical Field

[0001] The embodiments of this application relate to the field of small target detection technology, and in particular to a cross-modal prior-guided diffusion-type visible light and infrared small target detection system. Background Technology

[0002] Visible light and infrared images are complementary in their imaging mechanisms. Visible light images provide rich texture and detail information, while infrared images can maintain the thermal radiation contour and salience of targets even in complex environments such as low light, fog, and occlusion. Therefore, the fusion of visible light and infrared images has a natural advantage in improving the robustness and reliability of perception in complex environments, especially suitable for small target detection scenarios. Existing visible light and infrared small target detection methods can be generally summarized into two technical paths: one is the structural design of the fusion layer, and the other is the modeling and optimization of cross-modal interaction and aggregation mechanisms. The former often adopts a multi-stage fusion framework, combining pixel-level fusion with feature-level fusion to improve the visibility and local contrast of small targets at the image level, while the latter focuses on improving feature representation capabilities through multi-scale interaction, saliency enhancement, or noise suppression.

[0003] In recent years, diffusion models have attracted attention in the field of object detection. This type of method describes the detection process as starting from randomly initialized noisy candidate boxes, and gradually approximating the position and geometry of the true target box through multiple iterative denoising and updating steps, thereby achieving object detection.

[0004] In 2023, Chen et al. proposed the first diffusion-based object detection framework, DiffusionDet. This framework describes object detection as a gradual denoising diffusion process from noisy boxes to ground truth object boxes. During the training phase, Gaussian noise is added to the ground truth labeled boxes according to variance scheduling, and inverse diffusion denoising is learned. During the inference phase, starting from a set of randomly initialized noisy boxes, the position and geometry of the object box are gradually approximated through multiple iterations, thus completing the detection prediction. However, as a basic implementation of an iterative object detection framework, DiffusionDet is mainly aimed at single-modal natural image scenes and lacks cross-modal difference modeling and small object feature enhancement mechanisms for visible light and infrared small object detection. Summary of the Invention

[0005] In view of this, embodiments of this application propose a cross-modal prior-guided diffusion-type visible light and infrared small target detection system. By explicitly evaluating the effective information contribution of visible light and infrared in different scenarios, the system dynamically adjusts the cross-modal interaction and fusion weights to achieve higher detection accuracy. Utilizing a prior-guided candidate box supplementation strategy, the generation and supplementation of candidate boxes are constrained and resampled in each iteration, making the newly added candidate boxes more concentrated in the potential target region, thereby improving the effectiveness of the denoising process, reducing unnecessary computational overhead, and further enhancing the small target detection capability.

[0006] To achieve the above objectives, embodiments of this application propose a cross-modal prior-guided diffusion-based visible and infrared small target detection system. The system includes: a data preprocessing module, a dual-branch multi-scale feature extraction module, a frequency domain prior-guided multi-modal feature enhancement module, a diffusion-based detection module, and a result output module. The diffusion-based detection module includes a spatially distributed prior-guided candidate box update submodule. The data preprocessing module acquires aligned visible and infrared images and performs necessary formatting and preprocessing to obtain preprocessed visible and infrared images. The dual-branch multi-scale feature extraction module is used to process the preprocessed visible and infrared images respectively. Multi-scale feature extraction is performed on the external image to obtain multi-scale features of the visible light mode and the infrared mode. The multi-modal feature enhancement module is used to perform frequency domain complementary enhancement and cross-modal hierarchical fusion on the multi-scale features of the visible light mode and the infrared mode to obtain fused enhanced features. The diffusion detection module is used to perform multi-step denoising iterative decoding based on the fused enhanced features to generate target candidate boxes and obtain detection results. The candidate box update sub-module is used to supplement or update the candidate box set according to the prior of the bimodal spatial distribution during the iteration process. The result output module is used to post-process the detection results and output the target box position, category and confidence information.

[0007] To achieve the above objectives, embodiments of this application also propose a cross-modal prior-guided diffusion-type visible and infrared small target detection method, implemented based on the cross-modal prior-guided diffusion-type visible and infrared small target detection system described above. The method includes: acquiring aligned visible light and infrared images, and performing necessary formatting and preprocessing to obtain preprocessed visible light and infrared images; extracting multi-scale features from the preprocessed visible light and infrared images respectively to obtain multi-scale features of the visible light modality and the infrared modality; performing frequency domain complementary enhancement and cross-modal hierarchical fusion on the multi-scale features of the visible light and infrared modalities to obtain fused enhanced features; performing multi-step denoising iterative decoding based on the fused enhanced features to generate target candidate boxes and obtain detection results; during the iteration process, supplementing or updating the candidate box set according to the bimodal spatial distribution prior; and post-processing the detection results and outputting the target box position, category, and confidence information.

[0008] To achieve the above objectives, embodiments of this application also propose an electronic device, including: a processor and a memory storing a program, the program including instructions executable by the processor, the processor being configured to, when executing the instructions, enable the electronic device to implement a cross-modal prior-guided diffuse visible light and infrared small target detection method as described above.

[0009] To achieve the above objectives, embodiments of this application also propose a computer-readable storage medium storing a computer program that, when executed by a processor, enables a cross-modal prior-guided diffusion-based visible and infrared small target detection method as described above.

[0010] Optionally, the multimodal feature enhancement module consists of a cross-modal frequency domain complementary enhancement submodule and a cross-modal hierarchical multi-scale fusion submodule; The cross-modal frequency domain complementary enhancement submodules respectively enhance the multi-scale features of visible light modes. Multiscale features of infrared modes Perform a two-dimensional discrete cosine transform to obtain the corresponding frequency domain coefficients. and Among them, the lower right corner mark Indicates the first At each scale, the low-frequency part of the frequency domain coefficients corresponds to the global structure and background components, while the high-frequency part of the frequency domain coefficients corresponds to detailed information including edges and textures. Constructing a binary mask in the frequency domain frequency domain coefficients and With binary mask respectively Element-wise multiplication is performed to reduce low-frequency background information while preserving high-frequency detail information, resulting in the frequency domain coefficients after mask selection. and ; Frequency domain coefficients after mask screening and By performing two-dimensional inverse discrete cosine transforms, the high-frequency response characteristics in the spatial domain are obtained. and .

[0011] Optionally, a binary mask This can be expressed by the formula: ; in, To control the hyperparameters in the low-frequency suppression region, and The first The height and width corresponding to each scale; High frequency response characteristics and This can be expressed by the formula: ; ; in, Represents the two-dimensional discrete cosine transform. This represents the two-dimensional discrete cosine inverse transform.

[0012] Optionally, the cross-modal hierarchical multi-scale fusion submodules respectively process the high-frequency response features and Global average pooling is performed to obtain the channel description vectors of the two modes. The two vectors are concatenated along the channel dimension and then transformed to generate two sets of channel-gated weights. Softmax normalization is performed on the two sets of channel-gated weights to obtain the channel-level weight coefficients corresponding to the visible light mode and the infrared mode. ; Based on channel-level weighting coefficients Infrared high-frequency response characteristics After channel weighting, it is compared with the high-frequency response characteristics of visible light. Element-wise addition yields the enhanced high-frequency features in visible light. High-frequency response characteristics of visible light After channel weighting, it is compared with the infrared high-frequency response characteristics. Element-by-element addition yields the infrared-enhanced high-frequency features. ; Calculate enhanced high-frequency features and The convolutional attention is used to obtain the corresponding spatial attention weights and channel attention weights. These spatial and channel attention weights are then multiplied element-wise with the high-frequency response features, and the final enhanced features are output by summing the residuals. and ; The first Each scale With the Each scale To conduct cross-scale multimodal interaction, Perform upsampling to obtain Spatial resolution aligned , respectively Apply independent 1×1 convolutional transformations to generate the key feature matrix required for cross-modal attention. AND-valued characteristic matrix ,right Applying a 1×1 convolutional transformation to generate the query feature matrix required for cross-modal attention ; Will , , Divide the space into several non-overlapping, one-to-one corresponding blocks, and denote the corresponding first block as... Each block is respectively , , In the Within each block, calculate and The similarity is calculated and then normalized using Softmax to obtain the intra-block cross-modal attention weights, which are then further processed. Perform a weighted summation to obtain the first... Cross-modal interaction results of individual blocks ; By concatenating the cross-modal interaction results of all blocks according to their original spatial locations, the cross-modal attention output at the full scale is restored. As a visible light mode in the 1st The cross-modal multi-scale fusion features are constructed at various scales, while the infrared modality is constructed in a symmetrical manner. As an infrared mode in the first Cross-modal multi-scale fusion features at various scales; Will and and , , No. The fusion enhancement features at each scale are aggregated using a residual addition method to obtain the th scale. Fusion enhancement features at various scales .

[0013] Optionally, the first Fusion enhancement features at various scales This can be expressed by the formula: ; , ; ; ; in, This indicates that the parts are pieced together according to their original spatial positions. Indicates dimension.

[0014] Optionally, during the progressive denoising iteration of the diffusion detection module, a candidate box update strategy guided by spatial distribution priors is introduced. In each iteration, the candidate box set is supplemented and updated. The diffusion detection module updates the candidate box set at each time step. Fuse enhanced features with candidate noise box set The data is fed into a prediction network, which outputs a set of candidate boxes with high confidence. The candidate box set for the next time step is updated through diffusion sampling. ; exist In this case, the number of candidate boxes is less than the number of candidate boxes input in the previous round, so a candidate box replenishment operation needs to be performed to maintain the total number of candidate boxes at the preset value. Let the number of candidate boxes to be added in this round be... , , The number of candidate boxes currently retained; Using the spatial distribution prior map of single-mode high-frequency response and As the basis for candidate box generation and supplementation strategies, and After flattening, Softmax normalization is performed to obtain the corresponding spatial probability distribution map. and , used to characterize the spatial distribution of potential target centers; right and Perform sampling operations separately, drawing from their respective spatial probability distributions. By grouping the center coordinates, we obtain the set of center coordinates guided by visible light and infrared light. and ;in, This is a proportionality coefficient used to control the proportion of supplementary prior guidance frames in the two-modal framework; First generate A set of Gaussian random candidate boxes is used as the basic supplementary set, and then selected from the basic supplementary set... Each candidate box is replaced with its center coordinate point to obtain two sets of unimodal spatial prior guided candidate boxes. and The remaining Each candidate box is kept as a Gaussian random candidate box. Finally, the three sets are combined to obtain the supplementary set of candidate boxes required for this round of updates. , ; Will and Merge, forming a quantity of The candidate box set is used as the input candidate box set for the next time step and fed into the prediction network. The iterative process of prediction, diffusion update and replenishment is repeated until the time step is 0, and the final detection result is output.

[0015] Optionally, the system is deployed on edge computing devices, edge servers, or cloud inference services. At the deployment level, the system supports exporting the trained model into a general intermediate representation format and converting it into a deployment format supported by the target inference engine. At the same time, it encapsulates the inference engine interface to form a unified inference calling method, so as to achieve interface consistency and convenient migration between different hardware platforms or inference engines.

[0016] This application proposes a cross-modal prior-guided diffusion-based visible and infrared small target detection system, comprising a data preprocessing module, a dual-branch multi-scale feature extraction module, a frequency domain prior-guided multi-modal feature enhancement module, a diffusion-based detection module, and a result output module. The diffusion-based detection module includes a spatially distributed prior-guided candidate box update sub-module. The preprocessing module acquires aligned visible and infrared images and performs necessary formatting and preprocessing to obtain preprocessed visible and infrared images. The dual-branch multi-scale feature extraction module performs multi-scale feature enhancement on the preprocessed visible and infrared images respectively. The system extracts multi-scale features for both the visible and infrared modes. A multi-modal feature enhancement module performs frequency-domain complementary enhancement and cross-modal hierarchical fusion on these features to obtain fused enhanced features. A diffusion detection module performs multi-step denoising iterative decoding based on the fused enhanced features to generate target candidate boxes and obtain detection results. A candidate box update submodule supplements or updates the candidate box set during iteration based on a priori bimodal spatial distribution. A result output module post-processes the detection results and outputs the target box position, category, and confidence information. The system achieves higher detection accuracy by explicitly evaluating the effective information contribution of visible and infrared light in different scenarios and dynamically adjusting cross-modal interaction and fusion weights. Utilizing a priori-guided candidate box supplementation strategy, the generation and supplementation of candidate boxes are constrained and resampled in each iteration, making the newly added candidate boxes more concentrated in potential target regions, improving the effectiveness of the denoising process, reducing unnecessary computational overhead, and significantly enhancing the small target detection capability. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies of this application will be briefly introduced below. The following drawings are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. The drawings described herein are only used to explain this application and are not intended to limit this application.

[0018] Figure 1 This is a schematic diagram of the structure of a cross-modal prior-guided diffusion-type visible light and infrared small target detection system provided in one embodiment of this application; Figure 2 This is a detailed schematic diagram of a cross-modal prior-guided diffusion-type visible light and infrared small target detection system provided in one embodiment of this application; Figure 3 This is a structural diagram of a pyramid model of a frequency domain prior-guided multimodal feature enhancement module provided in one embodiment of this application; Figure 4 This is a schematic diagram of a candidate box update strategy guided by spatial distribution priors provided in one embodiment of this application; Figure 5 This is a simulation experiment result diagram provided in one embodiment of this application; Figure 6 This is a flowchart of a cross-modal prior-guided diffusion-based visible and infrared small target detection method provided in another embodiment of this application; Figure 7 This is a schematic diagram of the structure of an electronic device provided in another embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. Those skilled in the art will understand that many technical details have been provided in the embodiments of this application to facilitate better understanding. However, the technical solutions claimed in this application can be implemented even without these technical details and various variations and modifications based on the following embodiments. The division of the following embodiments is for ease of description and should not constitute any limitation on the specific implementation of this application. The following embodiments can be combined with and referenced by each other without contradiction.

[0020] Existing visible and infrared small target detection fusion techniques suffer from the following drawbacks. Most existing methods employ a multi-scale top-down fusion structure, enhancing small target representation through upsampling and cross-layer interaction. However, when visible and infrared imaging differ significantly and their information distributions are inconsistent, they often still rely on generic fusion strategies, lacking explicit modeling of the differences and complementary relationships between the two modes. This can easily lead to insufficient complementary cue mining and amplification of redundant background responses, thus limiting the accuracy and robustness of small target detection in complex scenes. Therefore, this application aims to design a cross-modal prior-guided fusion detection method for visible and infrared small targets. By extracting prior information such as frequency domain response and spatial saliency from the two modal features, it explicitly characterizes modal differences and complementary relationships. Furthermore, it uses this prior information to modulate the multi-scale feature enhancement and cross-layer interaction processes, thereby suppressing redundant background responses and improving the stability and accuracy of small target representation.

[0021] Existing diffusion-based detection frameworks suffer from the following drawbacks in candidate box generation and update strategies: They typically initialize candidate boxes randomly with a Gaussian distribution and continue to supplement them with Gaussian random boxes during iterative decoding to maintain the number of candidate boxes. However, the center position and scale distribution of these Gaussian random boxes lack prior constraints, often resulting in a mismatch between the spatial layout and size statistics of the real target. This leads to a large number of candidate boxes falling into background regions or forming degenerate invalid boxes, thereby diluting the effective supervision signal, affecting convergence stability, and increasing computational overhead, thus limiting the improvement of detection accuracy and robustness. To address this, this application proposes a spatially prior-guided candidate box supplementation and update strategy. This strategy utilizes the spatial prior of multimodal feature extraction to guide the candidate box generation and supplementation process, making newly added candidate boxes more likely to be located in potential small target regions during multiple iterations, thereby improving the effectiveness of denoising and updating and enhancing detection performance.

[0022] To address the above issues, one embodiment of this application proposes a cross-modal prior-guided diffusion-type visible and infrared small target detection system. The implementation details of the cross-modal prior-guided diffusion-type visible and infrared small target detection system proposed in this embodiment are described below. The following implementation details are provided for ease of understanding and are not necessary for implementing this solution.

[0023] The overall architecture of the cross-modal prior-guided diffusion-type visible and infrared small target detection system proposed in this embodiment can be as follows: Figure 1 As shown, specific details are as follows: Figure 2 As shown, it includes: a data preprocessing module 11, a dual-branch multi-scale feature extraction module 12, a frequency domain prior-guided multimodal feature enhancement module 13, a diffusion detection module 14, and a result output module 15. The diffusion detection module 14 is equipped with a candidate box update submodule 141 that is guided by spatial distribution prior.

[0024] The data preprocessing module 11 is used to acquire aligned visible light and infrared images, and perform necessary formatting and preprocessing to obtain preprocessed visible light and infrared images.

[0025] The dual-branch multi-scale feature extraction module 12 is used to extract multi-scale features from the preprocessed visible light image and infrared image respectively, so as to obtain multi-scale features of the visible light mode and multi-scale features of the infrared mode.

[0026] The multimodal feature enhancement module 13 is used to perform frequency domain complementary enhancement and cross-modal level fusion of the multi-scale features of the visible light mode and the multi-scale features of the infrared mode to obtain fused enhanced features.

[0027] The diffusion detection module 14 is used to perform multi-step denoising iterative decoding based on fused enhancement features to generate target candidate boxes and obtain detection results. The candidate box update submodule 141 is used to supplement or update the candidate box set according to the prior bimodal spatial distribution during the iteration process.

[0028] The result output module 15 is used to post-process the detection results and output the target box position, category, and confidence information.

[0029] The following is a detailed description of each module of the cross-modal prior-guided diffusion-type visible and infrared small target detection system proposed in this embodiment.

[0030] 1) Dual-branch multi-scale feature extraction module.

[0031] The main purpose of the dual-branch multi-scale feature extraction module is to extract high-frequency response features of two modes in the frequency domain through discrete cosine transform, which is used to highlight the edges of small targets and fine-grained texture information. The multi-modal feature enhancement module consists of a cross-modal frequency domain complementary enhancement submodule and a cross-modal hierarchical multi-scale fusion submodule.

[0032] The cross-modal frequency domain complementary enhancement submodules respectively enhance the multi-scale features of visible light modes. Multiscale features of infrared modes Perform a two-dimensional discrete cosine transform to obtain the corresponding frequency domain coefficients. and Among them, the bottom right corner mark Indicates the first At each scale, the low-frequency part of the frequency domain coefficients corresponds to the global structure and background components, while the high-frequency part of the frequency domain coefficients corresponds to detailed information including edges and textures.

[0033] Next, a binary mask is constructed in the frequency domain. A binary mask is used to suppress low-frequency components and retain high-frequency components. The value in the low-frequency region is 0, and the value in the rest of the region is 1. The size of the low-frequency suppression region is determined by the hyperparameter. The control is to set 0 in the low-frequency block region in the upper left corner of the frequency domain, and set 1 in the other frequency positions.

[0034] Binary mask This can be expressed by the formula: ; in, To control the hyperparameters in the low-frequency suppression region, and The first The height and width corresponding to each scale.

[0035] frequency domain coefficients and With binary mask respectively Element-wise multiplication is performed to reduce low-frequency background information while preserving high-frequency detail information, resulting in the frequency domain coefficients after mask selection. and .

[0036] Finally, the frequency domain coefficients after mask selection were analyzed. and By performing two-dimensional inverse discrete cosine transforms, the high-frequency response characteristics in the spatial domain are obtained. and High-frequency response characteristics can highlight the edges of small targets, local contrast changes, and detailed structures in the spatial domain, while suppressing large-scale smooth backgrounds and low-frequency interference.

[0037] High frequency response characteristics and This can be expressed by the formula: ; ; in, Represents the two-dimensional discrete cosine transform. This represents the two-dimensional discrete cosine inverse transform.

[0038] Because small targets inherently possess weak texture and structural cues and are susceptible to environmental factors such as illumination variations, thermal noise, and occlusion, high-frequency representations often exhibit significant attenuation when relying solely on a single modality. This leads to unstable responses and modal asymmetry in small target representations. To enhance the robustness and discriminability of small target representations, this embodiment designs a cross-modal dynamic gating fusion mechanism guided by frequency domain response. This mechanism adaptively shares and complementaryly injects high-frequency information between two modalities, mitigating the inconsistency in cross-modal responses of small targets. This mechanism is executed by a cross-modal hierarchical multi-scale fusion submodule.

[0039] refer to Figure 3 The cross-modal hierarchical multi-scale fusion submodules respectively analyze high-frequency response features and Global average pooling is performed to obtain the channel description vectors of the two modes. The two vectors are concatenated along the channel dimension and then transformed to generate two sets of channel-gated weights. Softmax normalization is performed on the two sets of channel-gated weights to obtain the channel-level weight coefficients corresponding to the visible light mode and the infrared mode. .

[0040] ; in, Indicates global average pooling. This indicates channel dimension splicing and channel transformation. This indicates Softmax normalization.

[0041] Based on channel-level weighting coefficients Infrared high-frequency response characteristics After channel weighting, it is compared with the high-frequency response characteristics of visible light. Element-wise addition yields the enhanced high-frequency features in visible light. High-frequency response characteristics of visible light After channel weighting, it is compared with the infrared high-frequency response characteristics. Element-by-element addition yields the infrared-enhanced high-frequency features. .

[0042] , .

[0043] Calculate enhanced high-frequency features and The convolutional attention is used to obtain the corresponding spatial attention weights and channel attention weights. These spatial and channel attention weights are then multiplied element-wise with the high-frequency response features, and the final enhanced features are output by summing the residuals. and .

[0044] , ; in, , Represents spatial attention weights. , This represents the channel attention weight.

[0045] The first Each scale With the Each scale To conduct cross-scale multimodal interaction, Perform upsampling to obtain Spatial resolution aligned , respectively Apply independent 1×1 convolutional transformations to generate the key feature matrix required for cross-modal attention. AND-valued characteristic matrix ,right Applying a 1×1 convolutional transformation to generate the query feature matrix required for cross-modal attention .

[0046] To reduce the computational cost of global attention while ensuring spatial alignment, the spatial dimensions of the smallest-scale feature maps in the backbone network are used to... , , Divide the space into several non-overlapping, one-to-one corresponding blocks, and denote the corresponding first block as... Each block is respectively , , In the Within each block, calculate and The similarity is calculated and then normalized using Softmax to obtain the intra-block cross-modal attention weights, which are then further processed. Perform a weighted summation to obtain the first... Cross-modal interaction results of individual blocks The cross-modal interaction results of all blocks are concatenated according to their original spatial positions to restore the full-scale cross-modal attention output. As a visible light mode in the 1st The cross-modal multi-scale fusion features are constructed at various scales, while the infrared modality is constructed in a symmetrical manner. As an infrared mode in the first Multi-scale fusion features across multiple scales.

[0047] Will and and , , No. The fusion enhancement features at each scale are aggregated using a residual addition method to obtain the th scale. Fusion enhancement features at various scales .

[0048] No. Fusion enhancement features at various scales This can be expressed by the formula: ; , ; ; ; in, This indicates that the parts are pieced together according to their original spatial positions. Indicates dimension.

[0049] 2) Diffusion detection module.

[0050] During the progressive denoising iteration of the diffusion detection module, a candidate box update strategy guided by spatial distribution priors is introduced (e.g., Figure 4 As shown in the figure, the candidate box set is supplemented and updated in each iteration. The diffusion detection module updates the candidate box set at each time step. Fuse enhanced features with candidate noise box set The data is fed into a prediction network, which outputs a set of candidate boxes with high confidence. The candidate box set for the next time step is updated through diffusion sampling. .

[0051] exist In this case, the number of candidate boxes is less than the number of candidate boxes input in the previous round, so a candidate box replenishment operation needs to be performed to maintain the total number of candidate boxes at the preset value. Let the number of candidate boxes to be added in this round be... , , The number of candidate boxes currently retained.

[0052] To avoid the problem of a large number of candidate boxes falling into the background region due to the traditional method of supplementing only with Gaussian random boxes, this embodiment utilizes a priori maps of the spatial distribution of single-modal high-frequency response. and As the basis for candidate box generation and supplementation strategies, and After flattening, Softmax normalization is performed to obtain the corresponding spatial probability distribution map. and , used to characterize the spatial distribution of potential target centers.

[0053] Next, for and Perform sampling operations separately, drawing from their respective spatial probability distributions. By grouping the center coordinates, we obtain the set of center coordinates guided by visible light and infrared light. and ;in, This is a scaling factor used to control the proportion of supplementary prior guidance frames in the two-modal framework. First, generate... A set of Gaussian random candidate boxes is used as the basic supplementary set, and then selected from the basic supplementary set... Each candidate box is replaced with its center coordinate point to obtain two sets of unimodal spatial prior guided candidate boxes. and The remaining Each candidate box is kept as a Gaussian random candidate box. Finally, the three sets are combined to obtain the supplementary set of candidate boxes required for this round of updates. , .

[0054] Finally, and Merge, forming a quantity of The candidate box set is used as the input candidate box set for the next time step and fed into the prediction network. The iterative process of prediction, diffusion update and replenishment is repeated until the time step is 0, and the final detection result is output.

[0055] 3) Actual deployment.

[0056] This embodiment proposes a cross-modal prior-guided diffusion-type visible light and infrared small target detection system that can be deployed on edge computing devices, edge servers, or cloud inference services. At the deployment level, the system supports exporting the trained model into a general intermediate representation format and converting it into a deployment format supported by the target inference engine. At the same time, it encapsulates the inference engine interface to form a unified inference calling method, so as to achieve interface consistency and convenient migration between different hardware platforms or inference engines.

[0057] This embodiment is designed for engineering deployment scenarios. Addressing the issues of instability in preprocessing due to diverse sources of bimodal input data, inconsistent registration and alignment, and inconsistent data formats, as well as the difficulty in balancing inference latency and stability caused by multiple rounds of denoising iterations required for diffusion-based detection, a modular software system flow and iterative control scheme are implemented. Logically, the software system includes: a data input and preprocessing unit, a dual-branch multi-scale feature extraction unit, a frequency-domain prior-guided multimodal feature enhancement unit, a diffusion-based denoising detection iteration unit, a spatially distributed prior-guided candidate box supplementation and update unit, and a result post-processing and output unit. These units are executed sequentially and cyclically according to the software flow. The software system can be deployed on edge computing devices, edge servers, or cloud-based inference services, and the processor can execute program instructions from memory to implement the above functional units. To facilitate engineering deployment, the system supports exporting the trained model to a universal intermediate representation format and converting it to a deployment format supported by the target inference engine. Simultaneously, it encapsulates the inference engine interface to form a unified inference calling method, achieving interface consistency and convenient migration between different hardware platforms or inference engines. For example, in the deployment of edge prototypes, corresponding model conversion tools can be used to complete the conversion of trained models, and efficient languages ​​can be used to implement edge inference code. By encapsulating the inference engine API, a unified inference interface can be provided, thereby ensuring real-time inference capabilities and system maintainability on low-computing-power devices. The system first receives aligned visible light and infrared images from the processing unit and performs preprocessing such as size normalization, formatting, and standardization. Then, it extracts multi-scale features from the visible light and infrared branches respectively, and performs cross-modal frequency domain complementary enhancement and hierarchical fusion through a multi-modal feature enhancement unit to generate a fused feature pyramid for detection. During the diffusion-based denoising detection iteration process, the iteration control logic schedules the denoising decoding loop according to a preset maximum number of iterations and termination conditions (e.g., reaching the maximum number of iterations, the denoising time step returning to zero, or the output result converging). When the number of candidate boxes needs to be maintained in each round, the diffusion-based denoising detection iteration unit is invoked to supplement or update the candidate box set based on the spatial probability distribution (or its fused prior) constructed from the visible light / infrared single-modal prior map before entering the next iteration. Finally, the result post-processing and output unit filters the candidate boxes, calculates category confidence, and formats the output to obtain structured detection results such as target box location, category, and confidence. Through the above software flow and iteration control design, the stability, adaptability, and maintainability of the system in actual deployment scenarios can be improved.

[0058] This embodiment proposes a cross-modal prior-guided diffusion-based visible and infrared small target detection system, including a data preprocessing module, a dual-branch multi-scale feature extraction module, a frequency domain prior-guided multi-modal feature enhancement module, a diffusion-based detection module, and a result output module. The diffusion-based detection module includes a spatially distributed prior-guided candidate box update sub-module. The preprocessing module acquires aligned visible and infrared images and performs necessary formatting and preprocessing to obtain preprocessed visible and infrared images. The dual-branch multi-scale feature extraction module performs multi-modal enhancement on the preprocessed visible and infrared images respectively. The system extracts multi-scale features for both the visible and infrared modes. A multi-modal feature enhancement module performs frequency-domain complementary enhancement and cross-modal hierarchical fusion on these features to obtain fused enhanced features. A diffusion-based detection module performs multi-step denoising iterative decoding based on the fused enhanced features, generating target candidate boxes and obtaining detection results. A candidate box update submodule supplements or updates the candidate box set during iteration based on prior information about the bimodal spatial distribution. A result output module post-processes the detection results and outputs the target box location, category, and confidence information. The system achieves higher detection accuracy by explicitly evaluating the effective information contribution of visible and infrared light in different scenarios and dynamically adjusting cross-modal interaction and fusion weights. Utilizing a priori-guided candidate box supplementation strategy, the generation and supplementation of candidate boxes are constrained and resampled in each iteration, making the newly added candidate boxes more concentrated in potential target regions, improving the effectiveness of the denoising process, reducing unnecessary computational overhead, and significantly enhancing the detection capability for small targets.

[0059] It is worth noting that all modules involved in this embodiment are logical modules. In practical applications, a logical module can be a physical module, a part of a physical module, or an organic combination of multiple physical modules. Furthermore, to highlight the innovative aspects of this application, this embodiment does not introduce modules that are not closely related to solving the technical problems proposed in this application. However, this does not mean that other modules are absent from this embodiment.

[0060] In one embodiment, to verify the effectiveness of the cross-modal prior-guided diffusion-type visible and infrared small target detection system proposed in this application (hereinafter referred to as this application or Ours), we conducted relevant simulation experiments.

[0061] We compared our proposed method with several representative single-modal detectors and visible-infrared multimodal detectors under the same evaluation settings. The results are shown in Tables 1, 2, and 3. The VEDAI dataset is a typical aerial remote sensing vehicle detection dataset, where targets are generally small in scale and densely distributed. Furthermore, it is affected by factors such as changes in the overhead viewpoint, background texture interference, and local occlusion, making the localization and discrimination of small targets quite challenging. The FLIR dataset is a typical visible-infrared and thermal-infrared dual-modal target detection dataset for vehicle-mounted scenes. It includes various imaging conditions such as day-night cycles, low illumination, strong glare, and complex road backgrounds. Pedestrians and vehicles exhibit significant small target and low signal-to-noise ratio characteristics at long distances and under weak contrast conditions, placing higher demands on cross-modal alignment and robust fusion. The comparative results show that among all multimodal methods, the cross-modal prior-guided diffusion-based visible-infrared small target detection method proposed in this application achieves the best performance, improving mAP by approximately 2% to 3% on both the VEDAI and FLIR datasets. This improvement is mainly due to two factors: First, the multi-scale feature fusion network with cross-modal frequency domain mutual enhancement uses prior-guided modeling of the differences and complementary information between the two modalities to strengthen the high-frequency details and cross-scale representation of small targets. Second, the spatial prior-guided candidate box supplementation and update strategy constrains the generation of candidate boxes during the diffusion iteration process, making the newly added candidate boxes more inclined towards potential small target regions, thereby reducing the interference of invalid background candidates on training and inference and improving the effectiveness of multi-round denoising updates.

[0062] Table 1: Performance comparison with several state-of-the-art methods on the Vedai dataset

[0063] Table 2: Performance comparison with several state-of-the-art methods on the FLIR dataset

[0064] Table 3: Comparison of parameters and frame rates of several multimodal detection methods on the Vedai dataset.

[0065] Experimental results show that the cross-modal prior-guided diffusion-based visible and infrared small target detection method proposed in this application achieves a good balance between detection accuracy and computational cost, and obtains the best detection accuracy on both VEDAI and FLIR datasets, while maintaining usable engineering efficiency in terms of parameter size and inference frame rate.

[0066] Lightweight model.

[0067] The parameter count in this application is 185.7M, significantly smaller than ICA Fusion's 517.1M (approximately a 64% reduction), reducing model size while maintaining accuracy advantages. The inference frame rate is 24.4 FPS, higher than ICA Fusion's 17.3 FPS, and also higher than... The FPS is 22.1. Although lower than CrossModalNet's 37.5 FPS, this application is superior in accuracy metrics and demonstrates a more balanced accuracy-efficiency trade-off overall.

[0068] Detection accuracy is guaranteed.

[0069] In comparisons with multimodal methods on the VEDAI dataset, this application achieved the highest performance, with an mAP50 of 83.5% and an mAP of 52.8%. Compared to representative multimodal methods, the mAP is improved by 1.7 percentage points compared to LF-MDet and by 2.5 percentage points compared to COMO; the mAP50 is improved by 2.7 percentage points compared to LF-MDet and by 1.8 percentage points compared to COMO. Compared to single-modal detectors (visible or infrared branches), this application leverages the advantages of cross-modal fusion and diffusion-based iterative updates, achieving an mAP improvement of approximately 7.7 to 8.5 percentage points compared to DiffusionDet (single-modal).

[0070] In comparison with multimodal methods on the FLIR dataset, this application achieves the best performance, with an mAP50 of 84.6% and an mAP of 45.6%. Compared to This application achieves a 3.1 percentage point improvement in mAP and a 1.7 percentage point improvement in mAP50. Compared to ICAFusion, the mAP improvement is 4.4 percentage points. Compared to single-modal methods, this application still shows a significant gain in mAP (approximately 7.7 to 8.5 percentage points), demonstrating the effective utilization of cross-modal complementary information under complex imaging conditions.

[0071] Furthermore, we designed ablation experiments to demonstrate the effectiveness of our proposed frequency-domain prior-guided multimodal feature enhancement pyramid (FP-MFEP) module and spatially distributed prior-guided candidate box update strategy (SDP-PRS). The ablation experiment results are shown in Tables 4 to 6.

[0072] Table 4: Performance comparison of different components of this application tested on the Vedai dataset.

[0073]

[0074] Table 5: Performance comparison of the FP-MFEP module with and without different components tested on the Vedai dataset.

[0075]

[0076] Table 6: Different SDP-PRS Performance comparison of values ​​on the Vedai dataset

[0077] Ablation experiments demonstrate that our proposed frequency-domain prior-guided multimodal feature enhancement pyramid (FP-MFEP) module and spatially distributed prior-guided candidate box update strategy (SDP-PRS) significantly improve model performance, validating their effectiveness.

[0078] The overall contribution of the FP-MFEP module and the SDP-PRS strategy.

[0079] Comparing different component combinations on the VEDAI dataset reveals that, with only SDP-PRS contributing, the mAP50 increases from 77.2% to 79.3% and the mAP from 46.9% to 48.7% without FP-MFEP. This demonstrates that the spatial distribution prior-guided candidate box update strategy can improve the effectiveness of candidate box updates during diffusion iterations without altering the fusion structure. With only FP-MFEP contributing, the mAP50 increases from 77.2% to 81.8% and the mAP from 46.9% to 51.0% without SDP-PRS, showing a greater improvement. This indicates that frequency domain prior-guided multimodal feature enhancement is more crucial for representing small targets. The combined effect of FP-MFEP and SDP-PRS shows the best performance, with mAP50 reaching 83.5% and mAP reaching 52.8%. Compared to the baseline, the mAP50 increased by +6.3% and the mAP increased by +5.9%, respectively, demonstrating that the two have complementary gains.

[0080] FP-MFEP internal component contributions (FCE and HMF).

[0081] Table 5 further analyzes the roles of the two key mechanisms (FCE and HMF) within the FP-MFEP. Regarding the role of FCE and the necessity of the gate, without HMF, adding FCE (without gate) results in mAP50 / mAP of 81.4% / 50.1%; adding FCE (with gate) improves this to 82.6% / 51.4%. This indicates that the gate mechanism can further enhance the effectiveness of FCE, making the high-frequency enhancement contribution to the two modes more reasonable. The contribution of the HMF cross-modal mechanism is 80.2% / 50.3% without FCE; with cross-modal, it improves to 82.1% / 51.0%. The combined effect of FCE (with gate) and HMF (with cross-modal) is optimal, reaching 83.5% / 52.8% when both are enabled simultaneously, a further improvement compared to enabling either module alone, verifying the synergistic effect of frequency domain enhancement + cross-modal mutual injection. This demonstrates that cross-modal interaction is crucial for HMF, enabling high-frequency cues from one modality to supplement those from another.

[0082] SDP-PRS Sensitivity to the value.

[0083] Table 6 lists the key hyperparameters of SDP-PRS. A comparison was made. When The optimal performance was achieved at [time value], with mAP50 at 83.5% and mAP at 52.8%. Performance degrades when the value is too small (e.g., 0, 0.1) or too large (e.g., 0.4, 0.5), indicating that the prior guidance strength of the candidate boxes needs to be moderate. Too weak a value cannot effectively constrain the distribution of candidate boxes, while too strong a value may reduce candidate diversity and affect the diffusion iterative search.

[0084] The combined ablation experiments show that FP-MFEP significantly improves the representation ability of small targets through frequency domain prior enhancement and cross-modal complementary injection, while SDP-PRS improves the effectiveness of candidate box updates and supplementation through spatial prior guidance. The combined use of these two technologies achieves state-of-the-art detection performance (83.5% mAP50 / 52.8% mAP) on the VEDAI dataset, validating the effectiveness and complementarity of this application.

[0085] Visualized results comparison.

[0086] Figure 5The paper presents a comparison of the detection visualization results of the baseline method, representative comparative methods, and the proposed method in the same visible-infrared small target scene. It can be observed that the baseline method and some comparative methods exhibit significant missed detections and false detections in the small target region: on the one hand, under conditions of weak contrast, background texture interference, or local occlusion, the small target response is unstable, leading to missing target boxes or low confidence; on the other hand, high-frequency textures or thermal noise regions in the background are easily falsely activated, generating redundant detection boxes. In contrast, the proposed method can output more stable detection boxes that better fit the target boundary in the small target region, while significantly reducing redundant responses in the background region. This is because the proposed method enhances the fine-grained cues of the small target in both modalities through a multi-scale high-frequency feature enhancement fusion network, and uses a spatial prior-guided candidate box supplementation and update strategy to concentrate newly added candidate boxes in the potential small target region during the diffusion iteration process, thereby improving the effectiveness of multi-round denoising updates and reducing missed detections and false detections.

[0087] With the increasing demand for all-weather, all-scenario perception capabilities in fields such as intelligent transportation, urban security, unmanned systems, and remote sensing monitoring, small target detection still faces common challenges under conditions such as low illumination at night, fog, rain, snow, and complex background occlusion, including weak visible light texture, few infrared details, and extremely low target pixel ratio. Cross-modal fusion of visible light and infrared can simultaneously utilize the texture details of visible light and the saliency of infrared thermal radiation, providing a natural advantage for small target detection in complex environments. The cross-modal prior-guided diffusion-type visible light and infrared small target detection system proposed in this application enhances the feature representation of small targets through cross-modal prior modeling and uses spatial prior constraints to generate and update candidate boxes during diffusion iteration. This can improve detection stability and robustness while ensuring accuracy, and has broad engineering application and promotion value.

[0088] 1) Intelligent transportation and vehicle-mounted assisted driving field.

[0089] In environments such as nighttime driving, backlighting, tunnel entrances and exits, and rain, fog, and haze, small targets such as pedestrians, bicycles, motorcycles, traffic cones, and animals at a distance are easily missed or falsely detected, directly affecting driving safety. This application can be used in vehicle-mounted multi-sensor perception modules to improve the stability of small target detection in low light and complex weather conditions, reduce the risk of false alarms caused by background interference, and provide more reliable forward perception input for advanced driver assistance systems.

[0090] 2) Urban security and perimeter monitoring.

[0091] In scenarios such as parks, communities, border fences, and critical infrastructure, small target intrusions (remote personnel, drones, abnormal heat sources, etc.) often have characteristics such as small scale, low contrast, and irregular movement. Based on the fusion detection of visible light and infrared light, continuous day and night monitoring can be achieved. This application, through a priori-guided diffusion-style candidate box iterative update mechanism, can improve alarm accuracy under complex backgrounds and weak target conditions, and is suitable for deployment in both fixed monitoring and mobile inspection systems.

[0092] 3) The field of UAV and remote sensing small target detection.

[0093] In aerial photography and remote sensing scenarios, targets such as vehicles, ships, and pedestrians are typically small-scale, densely distributed, and subject to strong background texture interference. They also suffer from variations in the overhead view and occlusion issues. This application can be applied to multimodal aerial photography platforms or airborne / spaceborne imaging systems to achieve stable detection and localization of small targets, serving tasks such as traffic flow monitoring, maritime surveillance, disaster assessment, and emergency search and rescue.

[0094] 4) Search and rescue emergency response and disaster site perception.

[0095] In rescue scenarios such as fires, earthquakes, and mountainous forest areas, smoke and dust obscure the view, low light levels, and complex terrain limit visible light imaging, while infrared technology can highlight abnormal body temperature and heat sources. This application can improve the detectability and positioning reliability of small targets under various complex conditions and can be used in rescue robots, portable thermal imaging terminals, or emergency command systems to assist in the rapid discovery of trapped personnel or abnormal heat sources.

[0096] 5) Industrial inspection and energy facility safety.

[0097] In scenarios such as power line inspection, pipeline inspection, petrochemical plants, and energy storage sites, long-distance small-scale defects, localized overheating, and abnormal hot spots often require joint judgment based on visible light details and infrared temperature distribution. The fusion detection capability of this application can be used for defect alarms and risk warnings, improving inspection efficiency and reducing the risk of missed detections, and is compatible with fixed-point monitoring and mobile inspection platforms.

[0098] In summary, this application can form an engineering-deployable visible light infrared small target detection software system. The system consists of modules such as data input and preprocessing, dual-branch feature extraction, frequency domain prior-guided multimodal feature enhancement, diffusion-based denoising detection iteration, spatial distribution prior-guided candidate box supplementation and updating, and result post-processing and output. It can be deployed on vehicle-mounted devices, edge computing devices, or cloud servers, and outputs detection results such as target boxes, categories, and confidence scores through standardized interfaces. Thus, it provides stable, robust, and scalable engineering application capabilities for "low-light, strong interference, and small-scale" scenarios, and has significant industrial application prospects.

[0099] Another embodiment of this application proposes a cross-modal prior-guided diffusion-type visible and infrared small target detection method. The implementation details of this cross-modal prior-guided diffusion-type visible and infrared small target detection method are described below. The following details are provided for ease of understanding and are not essential for implementing this solution. The specific flow of the cross-modal prior-guided diffusion-type visible and infrared small target detection method proposed in this embodiment can be as follows: Figure 6 As shown, it includes: Step 21: Obtain the aligned visible light image and infrared image, and perform necessary formatting and preprocessing to obtain the preprocessed visible light image and infrared image.

[0100] Step 22: Perform multi-scale feature extraction on the preprocessed visible light image and infrared image respectively to obtain multi-scale features of the visible light mode and the infrared mode.

[0101] Step 23: Perform frequency domain complementary enhancement and cross-modal hierarchical fusion on the multi-scale features of the visible light mode and the multi-scale features of the infrared mode to obtain fused enhanced features.

[0102] Step 24: Perform multi-step denoising iterative decoding based on the fused enhanced features to generate target candidate boxes and obtain detection results. During the iteration process, the candidate box set is supplemented or updated according to the prior of the bimodal spatial distribution.

[0103] Step 25: Post-process the detection results and output the target bounding box location, category, and confidence information.

[0104] The steps described above are merely for clarity in describing the technical solution. In actual implementation, they can be combined into one step, or certain steps can be broken down into multiple steps, as long as they involve the same logical relationship, they are all within the scope of protection of this application. Any insignificant modifications or designs added to the algorithm or process, as long as they do not change the core of the algorithm or process, are also within the scope of protection of this application.

[0105] It is not difficult to see that this embodiment is a method embodiment corresponding to the above system embodiment, and this embodiment can be implemented in conjunction with the above system embodiment. The relevant technical details and technical effects mentioned in the above system embodiment are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above system embodiment.

[0106] Another embodiment of this application provides an electronic device, such as Figure 7As shown, it includes: a processor 31 and a memory 32 storing a program, the program including instructions executable by the processor 31, the processor 31 being configured to, when executing the instructions, enable the electronic device to implement a cross-modal prior-guided diffuse visible light and infrared small target detection method as described in the above method embodiments.

[0107] The memory and processor are connected via a bus, which includes any number of interconnecting buses and bridges. The bus connects various circuits of one or more processors and memories. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0108] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0109] Another embodiment of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, enables a cross-modal prior-guided diffusion-based visible light and infrared small target detection method as described in the above method embodiments.

[0110] That is, those skilled in the art will understand that all or part of the steps in the above method embodiments can be implemented by a program instructing related hardware. The program is stored in a storage medium and includes several instructions to cause a device (such as a microcontroller, chip, etc.) or processor to execute all or part of the steps of the method described in the method embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks.

[0111] It will be understood by those skilled in the art that the above embodiments are specific implementations of this application, and various changes in form and detail can be made in practical applications without departing from the spirit and scope of this application. For those skilled in the art, several improvements and modifications can be made without departing from the principles of this application, and these improvements and modifications are also considered to be within the scope of protection of this application.

Claims

1. A cross-modal prior-guided diffusion-type visible and infrared small target detection system, characterized in that, The system includes: a data preprocessing module, a dual-branch multi-scale feature extraction module, a frequency domain prior-guided multimodal feature enhancement module, a diffusion detection module, and a result output module. The diffusion detection module is equipped with a candidate box update sub-module guided by spatial distribution prior. The data preprocessing module is used to acquire aligned visible light and infrared images, and perform necessary formatting and preprocessing to obtain preprocessed visible light and infrared images. The dual-branch multi-scale feature extraction module is used to extract multi-scale features from the preprocessed visible light image and infrared image respectively, so as to obtain the multi-scale features of the visible light mode and the multi-scale features of the infrared mode. The multimodal feature enhancement module is used to perform frequency domain complementary enhancement and cross-modal hierarchical fusion of multi-scale features of visible light modes and multi-scale features of infrared modes to obtain fused enhanced features; The diffusion detection module is used to perform multi-step denoising iterative decoding based on fused enhancement features to generate target candidate boxes and obtain detection results. The candidate box update submodule is used to supplement or update the candidate box set according to the prior bimodal spatial distribution during the iteration process. The results output module is used to post-process the detection results and output the target bounding box position, category, and confidence information.

2. The cross-modal prior-guided diffusion-type visible and infrared small target detection system according to claim 1, characterized in that, The multimodal feature enhancement module consists of a cross-modal frequency domain complementary enhancement submodule and a cross-modal hierarchical multi-scale fusion submodule; The cross-modal frequency domain complementary enhancement submodules respectively enhance the multi-scale features of visible light modes. Multiscale features of infrared modes Perform a two-dimensional discrete cosine transform to obtain the corresponding frequency domain coefficients. and Among them, the lower right corner mark Indicates the first At each scale, the low-frequency part of the frequency domain coefficients corresponds to the global structure and background components, while the high-frequency part of the frequency domain coefficients corresponds to detailed information including edges and textures. Constructing a binary mask in the frequency domain frequency domain coefficients and With binary mask respectively Element-wise multiplication is performed to reduce low-frequency background information while preserving high-frequency detail information, resulting in the frequency domain coefficients after mask selection. and ; Frequency domain coefficients after mask screening and By performing two-dimensional inverse discrete cosine transforms, the high-frequency response characteristics in the spatial domain are obtained. and .

3. The cross-modal prior-guided diffusion-type visible and infrared small target detection system according to claim 2, characterized in that, Binary mask This can be expressed by the formula: ； in, To control the hyperparameters in the low-frequency suppression region, and The first The height and width corresponding to each scale; High frequency response characteristics and This can be expressed by the formula: ； ； in, Represents the two-dimensional discrete cosine transform. This represents the two-dimensional discrete cosine inverse transform.

4. The cross-modal prior-guided diffusion-type visible and infrared small target detection system according to claim 2, characterized in that, The cross-modal hierarchical multi-scale fusion submodules respectively analyze high-frequency response features and Global average pooling is performed to obtain the channel description vectors of the two modes. The two vectors are concatenated along the channel dimension and then transformed to generate two sets of channel-gated weights. Softmax normalization is performed on the two sets of channel-gated weights to obtain the channel-level weight coefficients corresponding to the visible light mode and the infrared mode. ; Based on channel-level weighting coefficients Infrared high-frequency response characteristics After channel weighting, it is compared with the high-frequency response characteristics of visible light. Element-wise addition yields the enhanced high-frequency features in visible light. High-frequency response characteristics of visible light After channel weighting, it is compared with the infrared high-frequency response characteristics. Element-by-element addition yields the infrared-enhanced high-frequency features. ; Calculate enhanced high-frequency features and The convolutional attention is used to obtain the corresponding spatial attention weights and channel attention weights. These spatial and channel attention weights are then multiplied element-wise with the high-frequency response features, and the final enhanced features are output by summing the residuals. and ; The first Each scale With the Each scale To conduct cross-scale multimodal interaction, Perform upsampling to obtain Spatial resolution aligned , respectively Apply independent 1×1 convolutional transformations to generate the key feature matrix required for cross-modal attention. AND-valued characteristic matrix ,right Applying a 1×1 convolutional transformation to generate the query feature matrix required for cross-modal attention ; Will , , Divide the space into several non-overlapping, one-to-one corresponding blocks, and denote the corresponding first block as... Each block is respectively , , In the Within each block, calculate and The similarity is calculated and then normalized using Softmax to obtain the intra-block cross-modal attention weights, which are then further processed. Perform a weighted summation to obtain the first... Cross-modal interaction results of individual blocks ; By concatenating the cross-modal interaction results of all blocks according to their original spatial positions, the full-scale cross-modal attention output is restored. As a visible light mode in the 1st The cross-modal multi-scale fusion features are constructed at various scales, while the infrared modality is constructed in a symmetrical manner. As an infrared mode in the first Cross-modal multi-scale fusion features at various scales; Will and and , , No. The fusion enhancement features at each scale are aggregated using a residual addition method to obtain the th scale. Fusion enhancement features at various scales .

5. The cross-modal prior-guided diffusion-type visible and infrared small target detection system according to claim 4, characterized in that, No. Fusion enhancement features at various scales This can be expressed by the formula: ； ， ； ； ； in, This indicates that the parts are pieced together according to their original spatial positions. Indicates dimension.

6. The cross-modal prior-guided diffusion-type visible and infrared small target detection system according to claim 4, characterized in that, In the progressive denoising iteration process of the diffusion detection module, a candidate box update strategy guided by spatial distribution priors is introduced. The candidate box set is supplemented and updated in each iteration. The diffusion detection module updates the candidate box set at each time step. Fuse enhanced features with candidate noise box set The data is fed into a prediction network, which outputs a set of candidate boxes with high confidence. The candidate box set for the next time step is updated through diffusion sampling. ; exist In this case, the number of candidate boxes is less than the number of candidate boxes input in the previous round, so a candidate box replenishment operation needs to be performed to maintain the total number of candidate boxes at the preset value. Let the number of candidate boxes to be added in this round be... , , The number of candidate boxes currently retained; Using the spatial distribution prior map of single-mode high-frequency response and As the basis for candidate box generation and supplementation strategies, and After flattening, Softmax normalization is performed to obtain the corresponding spatial probability distribution map. and , used to characterize the spatial distribution of potential target centers; right and Perform sampling operations separately, drawing from their respective spatial probability distributions. By grouping the center coordinates, we obtain the set of center coordinates guided by visible light and infrared light. and ;in, This is a proportionality coefficient used to control the proportion of supplementary prior guidance frames in the two-modal framework; First generate A set of Gaussian random candidate boxes is used as the basic supplementary set, and then selected from the basic supplementary set... Each candidate box is replaced with its center coordinate point to obtain two sets of unimodal spatial prior guided candidate boxes. and The remaining Each candidate box is kept as a Gaussian random candidate box. Finally, the three sets are combined to obtain the supplementary set of candidate boxes required for this round of updates. , ; Will and Merge to form a quantity of The candidate box set is used as the input candidate box set for the next time step and fed into the prediction network. The iterative process of prediction, diffusion update and replenishment is repeated until the time step is 0, and the final detection result is output.

7. A cross-modal prior-guided diffusion-type visible and infrared small target detection system according to any one of claims 1 to 6, characterized in that, The system is deployed on edge computing devices, edge servers, or cloud inference services. At the deployment level, the system supports exporting the trained model into a general intermediate representation format and converting it into a deployment format supported by the target inference engine. At the same time, it encapsulates the inference engine interface to form a unified inference call method, so as to achieve interface consistency and convenient migration between different hardware platforms or inference engines.

8. A cross-modal prior-guided diffusion-type visible and infrared small target detection method, implemented based on a cross-modal prior-guided diffusion-type visible and infrared small target detection system as described in any one of claims 1 to 7, characterized in that, The method includes: The aligned visible light and infrared images are acquired, and necessary formatting and preprocessing are performed to obtain the preprocessed visible light and infrared images. Multi-scale feature extraction was performed on the preprocessed visible light image and infrared image respectively to obtain the multi-scale features of the visible light mode and the infrared mode. Frequency domain complementary enhancement and cross-modal hierarchical fusion are performed on the multi-scale features of the visible light mode and the multi-scale features of the infrared mode to obtain fused enhanced features; Multi-step denoising iterative decoding is performed based on fused and enhanced features to generate target candidate boxes and obtain detection results. During the iteration process, the candidate box set is supplemented or updated according to the prior of bimodal spatial distribution. The detection results are post-processed and the target bounding box location, category, and confidence information are output.

9. An electronic device, characterized in that, include: The processor and a memory storing a program, the program including instructions executable by the processor, the processor being configured to, when executing the instructions, enable the electronic device to implement the cross-modal prior-guided diffuse visible and infrared small target detection method as described in claim 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it can realize the cross-modal prior-guided diffusion-type visible light and infrared small target detection method as described in claim 8.

Images