Visible-infrared cross-modal object detection method guided by perception discriminative prior

Abstract

The present application relates to a kind of visible light-infrared cross-modal target detection method of perceptual discriminant prior guidance, comprising the following steps: step 1, bimodal feature preprocessing;Step 2, modal difference feature generation;Step 3, difference gate generation and bimodal feature enhancement;Step 4, cross-modal global interaction;Step 5, discriminant prior fusion feature construction;Step 6, multi-response graph construction;Step 7, soft foreground / background region division;Step 8, foreground / background prototype extraction;Step 9, discriminant foreground prior generation;Step 10, modal reliability estimation;Step 11, joint re-calibration;Step 12, final fusion output.The beneficial effects of the present application are: through modal difference perception, cross-modal global interaction, discriminant prior generation and modal reliability gate, adaptive fusion bimodal feature, enhance target area, suppress background interference, improve the detection accuracy of small target and occluded target in complex scene.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and deep learning target detection technology, specifically involving a cross-modal feature fusion enhancement method for target detection tasks in multimodal images such as visible light and infrared, and particularly involving a cross-modal global recalibration module guided by difference perception discrimination prior and its application in a multimodal RT-DETR target detection network. Background Technology

[0002] In scenarios such as low light, nighttime, rain, fog, occlusion, and complex traffic backgrounds, single visible light images are easily affected by insufficient illumination, weakened target texture, and enhanced background interference, resulting in unclear target area response. This is especially true for distant small targets, occluded targets, and low-contrast targets, which are prone to missed detection or false detection. Infrared images can provide thermal radiation information and have better target perception capabilities in nighttime and low-light environments. However, infrared images typically suffer from insufficient texture detail, unclear edge structures, and limited ability to distinguish categories.

[0003] Therefore, multimodal fusion of visible light and infrared images is an important way to improve target detection performance in complex scenes. Existing multimodal target detection methods typically employ feature stitching, element-wise addition, fixed-weight fusion, or simple attention mechanisms for modal fusion. However, these methods often have the following shortcomings: First, they are difficult to fully model the global complementary relationship between visible light and infrared features; second, they lack adaptive judgment of reliability changes between the two modalities under different scenes; third, the fusion process lacks clear foreground target discrimination guidance, easily amplifying background noise along with target features; fourth, in the detection of complex backgrounds, small targets, and occluded targets, the discriminability between the target region and the background region is insufficient, affecting detection accuracy and localization performance.

[0004] To address the aforementioned issues, this invention proposes a cross-modal global recalibration module guided by a difference perception and discrimination prior. By constructing modal difference features between visible light and infrared features, cross-modal global interaction relationships, foreground-background discrimination priors, and modal reliability gating, it achieves adaptive fusion enhancement of dual-modal features, thereby improving the target detection model's perception and localization capabilities in complex scenarios. Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide a visible light-infrared cross-modal target detection method guided by prior perception and discrimination. It mainly solves the problems of insufficient cross-modal fusion, insufficient foreground target discrimination, difficulty in suppressing background interference, and lack of adaptive adjustment of modal reliability in existing visible light-infrared multimodal target detection methods.

[0006] Specifically, in complex traffic scenes under low light, the visible light modality tends to lose target texture and detail information, while the infrared modality, although able to highlight thermal targets, lacks sufficient edge structure and category semantic information. Existing methods often employ simple stitching or weighted fusion, failing to fully utilize the complementary relationship between the two modalities. Furthermore, in complex backgrounds, the feature differences between the target and background regions are weak, leading to the model's tendency to amplify irrelevant background information, resulting in decreased detection performance for small, occluded, and low-contrast targets.

[0007] To address the aforementioned technical problems, this invention proposes a difference-aware discrimination prior-guided cross-modal global recalibration module, DPCGRM. This module takes visible light and infrared features as input. First, it unifies the bimodal feature representation through a feature preprocessing unit. Then, it constructs a modal difference-aware branch, generating modal difference features using the absolute difference between visible light and infrared features, and enhances the bimodal input features through difference gating. Subsequently, it establishes a bidirectional guiding relationship between visible light and infrared features through a cross-modal global interaction branch, obtaining the initially enhanced bimodal global features.

[0008] Building upon this foundation, the present invention further constructs a discriminative prior generation branch, fusing enhanced visible light features, enhanced infrared features, and modal difference features to generate fused features for foreground-background discrimination. This branch generates a base response map using the average response, maximum response, and modal difference response, and adaptively divides the soft foreground and soft background regions based on the response map. Subsequently, foreground and background prototypes are extracted separately, and the similarity between each spatial location in the fused features and the foreground and background prototypes is calculated. Finally, a discriminative foreground prior map is generated based on the difference between the foreground and background similarities.

[0009] Finally, this invention estimates the channel reliability of visible light and infrared features separately through modal reliability gating, and applies the discriminative foreground prior map and modal reliability weights together to the dual-modal enhancement features, thereby enhancing the target region and suppressing background interference. The recalibrated visible light features, infrared features, and modal difference features are further fused to output cross-modal fusion enhancement features for target perception, which are then fed into the subsequent detection head of the target detection network to complete target classification and localization.

[0010] The technical solution of this invention to solve the above-mentioned technical problems is as follows: a visible light-infrared cross-modal target detection method guided by prior perception and discrimination, comprising the following steps: Step 1, Dual-modal feature preprocessing: Visible light and infrared features are mapped to a unified channel space through 1×1 convolution to obtain preprocessed dual-modal features; Step 2, Modal difference feature generation: Based on the bimodal features preprocessed in Step 1, modal difference features are obtained through absolute difference calculation and convolution extraction; Step 3, Difference Gating Generation and Bimodal Feature Enhancement: Based on the modal difference features from Step 2 and the preprocessed bimodal features from Step 1, the difference-enhanced bimodal features are obtained through gating weight calculation and element-wise weighting. Step 4, Cross-modal global interaction: Based on the bimodal features enhanced by differences in Step 3, cross-modal global enhanced features are obtained through bidirectional global guided enhancement; Step 5, constructing discriminative prior fusion features: Based on the cross-modal global enhancement features from Step 4 and the modal difference features from Step 2, discriminative prior fusion features are obtained through channel concatenation and convolution fusion. Step 6, Multi-response map construction: Based on the discriminative prior fusion features from Step 5 and the modal difference features from Step 2, the basic response map is obtained through channel dimension statistics and convolutional fusion; Step 7, Soft foreground / background region segmentation: Based on the base response map from Step 6, soft foreground and soft background masks are obtained through mean adaptive and Sigmoid mapping; Step 8, Foreground / Background Prototype Extraction: Based on the discriminative prior fusion features from Step 5 and the soft foreground / background mask from Step 7, the foreground prototype and background prototype are obtained through weighted global average pooling. Step 9, Discriminative Foreground Prior Generation: Based on the discriminative prior fusion features from Step 5 and the foreground / background prototypes from Step 8, a discriminative foreground prior map is obtained through similarity calculation and feature mapping; Step 10, Modal reliability estimation: Based on the cross-modal global enhancement features from Step 4, bimodal reliability weights are obtained through global pooling, convolutional mapping, and Sigmoid activation; Step 11, Joint Recalibration: Based on the cross-modal global enhancement features from Step 4, the discriminative foreground prior map from Step 9, the modal reliability weights from Step 10, and the original bimodal features from Step 1, the recalibrated bimodal features are obtained through element-wise weighting, convolutional mapping, and residual connection. Step 12, final fusion output: Based on the recalibrated bimodal features from Step 11 and the modal difference features from Step 2, cross-modal fusion enhancement features are obtained through channel splicing and convolution fusion.

[0011] In some possible implementations, step 1 specifically includes: Step 1.1, Input the original visible light features Original infrared features ; Step 1.2, Visible Light Characteristics and infrared features Each process is preprocessed using a dedicated 1×1 convolution mapping: ; ; In the formula, Represents the visible light characteristics of the input; This indicates the visible light characteristics after preprocessing; Represents the visible light branch Convolution preprocessing mapping; Indicates the input infrared features; This indicates the infrared characteristics after preprocessing. Represents the 1×1 convolutional preprocessing mapping for the infrared branch; Step 1.3: Output the preprocessed visible light features. Preprocessed infrared features ; Step 2 specifically involves: Step 2.1, input the output of Step 1 , ; Step 2.2: Calculate the element-wise absolute difference between the two features to generate the initial modal difference response. : ; In the formula, This represents the initial difference response between visible light features and infrared features; This represents element-wise absolute value operation; Step 2.3, initial difference response The input consists of differential feature extraction maps composed of 3×3 convolutions, normalization, and activation functions. Extract modal difference features : ; In the formula, This represents the extracted modal difference features; Indicates by A differential feature extraction mapping consisting of convolution, normalization, and activation functions; Step 2.4, output modal difference features .

[0012] Step 1 eliminates feature distribution discrepancies between visible light and infrared due to differences in imaging mechanisms, unifies feature channel dimensions, and provides a standard input basis for subsequent modal difference calculation and cross-modal interaction, avoiding fusion failure caused by mismatch in modal feature dimensions. Step 2 explicitly models the difference regions of bimodal features in spatial and channel dimensions, accurately locating complementary and inconsistent regions of the two modalities, providing a core basis for subsequent difference gating and discriminative prior generation, and avoiding dilution of effective complementary information due to simple fusion.

[0013] In some possible implementations, step 3 specifically involves: Step 3.1, input the output of step 2. Step 1 output ; Step 3.2, Modal Difference Features After 1×1 convolution mapping Sigmoid activation function Processing to generate differential gating weights : ; In the formula, Indicates the difference gating weight; Represents a 1×1 convolution mapping; This represents the Sigmoid activation function, used to normalize the gate weights to the range of 0 to 1; Step 3.3, using Element-wise weighted enhancement The visible light characteristics obtained after difference enhancement : ; In the formula, This indicates the visible light characteristics after differential gating enhancement; This indicates the visible light characteristics after preprocessing; Indicates the difference gating weight; This indicates element-wise multiplication; Step 3.4, using Element-wise weighted enhancement After obtaining the difference-enhanced infrared features : ; In the formula, This indicates the infrared signature after differential gating enhancement; Indicates the infrared features after preprocessing; Indicates the difference gating weight; This indicates element-wise multiplication; Step 3.5: Output the enhanced visible light features. Infrared features after difference enhancement .

[0014] Step 3 utilizes modal difference information to locate and enhance complementary value regions, enabling subsequent cross-modal interactions to focus on effective complementary regions, avoiding background noise interference, amplifying bimodal difference complementary information in advance, and improving the accuracy of subsequent interactions and fusion.

[0015] In some possible implementations, step 4 specifically includes: Step 4.1, input the output of step 3. , ; Step 4.2: Establish the global guidance relationship from infrared to visible light, and enhance the global guidance function of infrared to visible light. Processing to generate visible light global enhancement features : ; In the formula, This indicates the global enhancement features of visible light after infrared mode guidance; This indicates the visible light characteristics after difference-gated enhancement; Indicates the infrared characteristics enhanced by differential gating; This represents the global guidance enhancement function of the infrared mode on the visible light mode; Step 4.3: Establish the global guidance relationship between visible light and infrared light, and enhance the global guidance function of visible light to infrared light. Processing to generate infrared global augmentation features : ; In the formula, This indicates the global infrared enhancement features after being guided by visible light modes; This represents the global guidance enhancement function of the visible light mode on the infrared mode; Step 4.4: Output visible light global enhancement features Infrared global enhancement features ; Step 5 specifically involves: Step 5.1, input the output of step 4. , Step 2 output ; Step 5.2, will , , splicing along the channel dimension; Step 5.3: The spliced ​​result is then fused and mapped using a 1×1 convolution. Processing and generating discriminative prior fusion features : ; In the formula, This represents the fusion features generated from prior information for foreground-background discrimination; This indicates a channel-level concatenation operation; express Convolutional fusion mapping; Indicates RGB global enhancement features; Indicates the global enhancement features of IR; Indicates modal difference characteristics; Step 5.4, output discriminative prior fusion features .

[0016] Step 4 constructs a bimodal, two-way global dependency, with infrared supplementing the target response missing under weak visible light, and visible light supplementing the insufficient texture and edge structure information of infrared, establishing long-distance spatial and cross-channel modal associations to maximize the complementary value of the two modes. Step 5 fuses cross-modal global enhancement information and modal difference information, enabling the fused features to possess both bimodal complementary characteristics and difference sensitivity, providing a comprehensive and accurate feature foundation for subsequent foreground-background discrimination and improving the reliability of the discrimination prior.

[0017] In some possible implementations, step 6 specifically includes: Step 6.1, input the output of step 5. Step 2 output ; Step 6.2, for Take the absolute value of each element, calculate the mean along the channel dimension, and then normalize using Min-Max to generate the average response plot. : ; In the formula, This represents the average response map, used to reflect the overall activation intensity of the fusion features; This indicates averaging along the channel dimension; This represents the Min-Max normalization operation; This represents element-wise absolute value operation; Step 6.3, for Take the absolute value of each element, find the maximum value along the channel dimension, and then perform Min-Max normalization to generate the maximum response map. : ; In the formula, This represents the maximum response map, used to highlight areas of significant local response. This indicates that the maximum value is taken along the channel dimension; Step 6.4, for Take the absolute value of each element, calculate the mean along the channel dimension, and then normalize using Min-Max to generate a differential response map. : ; In the formula, This represents a difference response map, used to reflect the modal difference regions between visible and infrared light; Indicates modal difference characteristics; Step 6.5, splicing , , Response-fused convolutional mapping Sigmoid activation function Process and generate the basic response graph. : ; In the formula, Represents the basic response diagram; This represents a response-fused convolutional mapping; This represents the Sigmoid activation function; Step 6.6, Output the basic response graph .

[0018] Step 6 constructs the response from three dimensions: global activation, local saliency, and modal difference. It takes into account the integrity of features, the saliency of the target, and the complementarity of modalities, accurately captures the response features of the target region, and provides a reliable basis for adaptive segmentation of foreground and background regions.

[0019] In some possible implementations, step 7 specifically includes: Step 7.1, input the output of step 6 ; Step 7.2, Calculate the basic response diagram Spatial dimension mean Combined with temperature coefficient Sigmoid activation function Generate a soft foreground mask : In the formula, Indicates a soft foreground mask; Represents the basic response diagram; Representing the basic response diagram The mean in the spatial dimension; This represents the temperature coefficient, used to control the smoothness of the soft partitioning; This represents the Sigmoid function; Step 7.3, from soft foreground mask Derivation of soft background mask : In the formula, Indicates a soft background mask; Indicates a soft foreground mask; Step 7.4, output the soft foreground mask. Soft background mask ; Step 8 specifically involves: Step 8.1, input the output of step 5. Step 7 output and ; Step 8.2, and Element-wise multiplication, weighted global average pooling, to generate the foreground prototype. : In the formula, Represents the foreground prototype; This represents the fusion features used to discriminate prior-generated features; Indicates a soft foreground mask; This indicates summing over spatial dimensions; This indicates element-wise multiplication; This represents a very small constant to prevent the denominator from being zero; Step 8.3, and Element-wise multiplication, weighted global average pooling, to generate the background prototype. : In the formula, Represents the background prototype; Indicates a soft background mask; Step 8.4, Output the foreground prototype Background Prototype .

[0020] Step 7 abandons the fixed threshold and adaptively divides soft foreground and soft background regions based on the current feature response distribution. This adapts to the differences in target response under different scenes and lighting conditions, improving the robustness of foreground-background segmentation and avoiding misclassification caused by a fixed threshold. Step 8 transforms complex spatial pixel discrimination into prototype similarity comparison. The foreground prototype represents the global semantic center of the target, and the background prototype represents the global semantic center of the background. This simplifies the discrimination logic, accurately anchors the feature benchmarks of the target and background, and improves the accuracy of subsequent similarity calculations.

[0021] In some possible implementations, step 9 specifically includes: Step 9.1, input the output of step 5. Step 8 output and ; Step 9.2, Calculation and Cosine similarity is used to generate a foreground similarity map. : In the formula, Represents a foreground similarity graph; This represents the function for calculating cosine similarity. Indicates fusion characteristics; Represents the foreground prototype; Step 9.3, Calculation and Cosine similarity is used to generate a background similarity map. : In the formula, Represents a background similarity map; Represents the background prototype; Step 9.4: Calculate the similarity difference between the foreground and background, and then perform ReLU activation, Min-Max normalization, and convolutional thinning mapping. Processing to generate a discriminative foreground prior map : In the formula, Represents a discriminative foreground prior graph; Used to filter negative response regions where the background similarity is higher than the foreground similarity; This indicates Min-Max normalization; This represents convolution thinning and sigmoid mapping operations; Step 9.5, output the discriminative foreground prior map. .

[0022] Step 9 accurately distinguishes between the target foreground and background regions, highlights the target region response, suppresses background interference, and generates target guidance weights in the spatial dimension. This provides a clear target enhancement direction for subsequent feature recalibration and solves the problem of targets not standing out in complex backgrounds.

[0023] In some possible implementations, step 10 specifically includes: Step 10.1, input the output of step 4 , ; Step 10.2, Global average pooling, followed by visible light reliability mapping Sigmoid activation function Processing to generate visible light reliability weights : In the formula, Represents the reliability weight of the visible light mode; Indicates RGB global enhancement features; Indicates global average pooling; Represents visible light reliability mapping; This represents the Sigmoid activation function; Step 10.3, Global average pooling, followed by infrared reliability mapping Sigmoid activation function Processing to generate infrared reliability weights : In the formula, Indicates the reliability weight of infrared modes; Indicates the global enhancement features of IR; Represents infrared reliability mapping; Step 10.4, output visible light reliability weights Infrared reliability weight .

[0024] Step 10 dynamically evaluates the reliability of the dual-modality under different scenarios. When the light is sufficient, the visible light weight is enhanced, and when the light is weak or at night, the infrared weight is enhanced. The modal contribution is adaptively adjusted to avoid the waste of modal information caused by fixed weights and improve the fusion robustness.

[0025] In some possible implementations, step 11 specifically includes: Step 11.1, input the output of step 4 , Step 9 output Step 10 output , Step 1: Original Features , ; Step 11.2, and , Element-wise multiplication generates intermediate modulation features in visible light. : In the formula, This represents the intermediate feature in the visible light branch that is modulated by both the discriminative foreground prior and the visible light reliability weight; Indicates RGB global enhancement features; Represents a discriminative foreground prior graph; Represents the reliability weight of the visible light mode; This indicates element-wise multiplication; Step 11.3, Visible light recalibration mapping The original visible light feature residuals are processed and superimposed to generate recalibrated visible light features. : In the formula, This indicates the visible light characteristics after recalibration; This represents visible light recalibration mapping; This indicates the modulation characteristics of the visible light branch; This represents the characteristics of the original input visible light. Step 11.4, and , Element-wise multiplication generates infrared intermediate modulation features. : In the formula, This represents the intermediate features in the infrared branch that are modulated by both the discriminative foreground prior and the infrared reliability weights. Indicates the global enhancement features of IR; Indicates the reliability weight of infrared modes; Step 11.5, Infrared recalibration mapping The original infrared feature residuals are processed and superimposed to generate recalibrated infrared features. : In the formula, Indicates the infrared characteristics after recalibration; Indicates infrared recalibration mapping; Indicates the intermediate modulation characteristics of the infrared branch; Represents the original input infrared characteristics; Step 11.6: Output the recalibrated visible light characteristics. Infrared characteristics after recalibration .

[0026] Step 11 Joint Space Target Guidance ( Modal reliability / By comparing the residuals with the original features, the target region is accurately enhanced and background noise is suppressed, while the original modal details are preserved, avoiding information loss due to over-modulation and improving the target discriminability and integrity of the features.

[0027] In some possible implementations, step 12 specifically includes: Step 12.1, input the output of step 11 , Step 2 output ; Step 12.2, will , , splicing along the channel dimension; Step 12.3: The splicing result is then fused and mapped by the final output. Processing to generate cross-modal fusion enhanced features : In the formula, This represents the cross-modal fusion enhancement features of the final output; This indicates the final output fusion mapping; This indicates the visible light characteristics after recalibration; Indicates the infrared characteristics after recalibration; Indicates modal difference characteristics; Step 12.4: Output cross-modal fusion enhancement features The sample is fed into the detection head to complete the target detection.

[0028] Step 12 integrates the recalibrated dual-modal features and modal difference features, while retaining visible light details, infrared thermal response and modal complementary difference information, and outputs fused features that combine target discrimination, modal complementarity and difference sensitivity, significantly improving the detection accuracy of small targets and occluded targets in low light and complex scenes.

[0029] The beneficial effects of this invention are: First, this invention explicitly models the difference information between visible light and infrared features through a modal difference perception branch, enabling the network to perceive complementary and inconsistent regions of different modalities in complex scenarios, thus avoiding the weakening of effective information caused by simple fusion.

[0030] Second, the present invention establishes a bidirectional guiding relationship between visible light and infrared features through a cross-modal global interaction mechanism, enabling the infrared mode to supplement the target response missing in the visible light under low light conditions, while enabling the visible light mode to supplement the insufficient texture, edge and structural information in the infrared image.

[0031] Third, the present invention enables the module to adaptively obtain foreground and background prototypes based on the fusion features through a foreground-background discrimination prior generation mechanism, and to generate a target foreground prior map by utilizing the difference between foreground similarity and background similarity, thereby enhancing the target region and suppressing complex background interference.

[0032] Fourth, this invention adaptively weights visible light and infrared features through modal reliability gating, enabling more reliable modes to obtain higher responses in different scenarios and improving the robustness of cross-modal fusion.

[0033] Fifth, this invention can be embedded in multimodal RT-DETR and other target detection networks for vehicle, pedestrian and other target detection tasks in low light, night and complex traffic scenarios, which helps to improve the detection accuracy of small targets, occluded targets and low contrast targets. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the overall structure of the differential perception and discrimination prior-guided cross-modal global recalibration module DPCGRM provided by the present invention; Figure 2 This is a schematic diagram of the modal difference sensing branch structure provided by the present invention; Figure 3 This is a schematic diagram of the cross-modal global interaction branch structure provided by the present invention; Figure 4 This is a schematic diagram of the foreground-background discriminative prior generation process provided by the present invention; Figure 5 This is a schematic diagram of the joint recalibration and fusion output of the discriminative prior and modal reliability provided by the present invention. Detailed Implementation

[0035] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0036] To make the objectives, technical solutions, and advantages of this invention clearer, the following is combined with... Figures 1 to 5This invention provides a detailed description of the difference-sensing discrimination prior-guided cross-modal global recalibration module DPCGRM proposed in this invention. This embodiment uses a visible-infrared multimodal target detection task as an example, where visible light features are denoted as... Infrared features are denoted as The DPCGRM module achieves adaptive fusion enhancement of visible light and infrared features through modal difference perception, cross-modal global interaction, discriminative foreground prior generation, modal reliability estimation, and joint recalibration fusion output.

[0037] like Figure 1 As shown, the DPCGRM module comprises six main parts: a bimodal feature preprocessing unit, a modal difference sensing branch, a cross-modal global interaction branch, a discriminative foreground prior generation branch, a modal reliability estimation branch, and a joint recalibration fusion output branch. The input is visible light features. and infrared features The output is a cross-modal fusion enhanced feature. The branches are not simply parallel but have a clear sequential dependency: the bimodal feature preprocessing results are used for modality difference perception; the modality difference perception results are used to enhance the bimodal input and simultaneously provide difference features for discriminative prior generation and final fusion output; the cross-modal global interaction results are simultaneously input into the discriminative prior generation, modality reliability estimation, and joint recalibration branches; the discriminative foreground prior and modality reliability weights work together in the final recalibration process.

[0038] Step 1: Dual-modal feature preprocessing.

[0039] like Figure 1 The "Dual-modal feature preprocessing" section on the left shows the input visible light features. and infrared features Enter the corresponding The convolutional preprocessing unit obtains the preprocessed visible light features. and infrared features The calculation process is as follows: In the formula, Represents the visible light characteristics of the input; This indicates the visible light characteristics after preprocessing; Represents the visible light branch Convolution preprocessing mapping.

[0040]

[0041] In the formula, Indicates the input infrared features; This indicates the infrared characteristics after preprocessing. This represents the 1×1 convolutional preprocessing mapping for the infrared branch.

[0042] This step corresponds to Figure 1 The main data stream flows from the input features to the preprocessing unit. Its function is to map visible light features and infrared features into a unified channel representation space, reducing the feature distribution differences between the two modalities caused by different imaging mechanisms, and providing a unified input for subsequent modal difference calculation and cross-modal interaction.

[0043] Step 2: Modal difference feature generation.

[0044] like Figure 2 As shown, the modal difference sensing branch is obtained in step 1. and As input, the absolute difference between the two is first calculated to obtain the initial modal difference response. : In the formula, This represents the initial difference response between visible light features and infrared features; This indicates element-wise absolute value operation.

[0045] Subsequently, the initial difference response Inputting a 3×3 convolutional feature extraction unit yields modality difference features. : In the formula, This represents the extracted modal difference features; Indicates by A differential feature extraction map consisting of convolution, normalization, and activation functions.

[0046] like Figure 1 and Figure 2 As shown in the joint, It has multiple functions in this invention: First, Used to generate differential gating weights; secondly... Cross-modal global enhancement features , Jointly construct the fusion features required for discriminative prior generation ;third, It participates in the final fusion output to retain supplementary information about the differences between visible and infrared light. Therefore, It is not an intermediate temporary variable, but a crucial distinguishing feature that runs through the discriminative prior generation and the final output.

[0047] This step enables the module to perceive inconsistencies in spatial location and channel response between the two modes by explicitly calculating the differences between visible light and infrared features, thus providing a basis for subsequent difference gating and discrimination prior generation.

[0048] Step 3: Differential gating generation and bimodal feature enhancement.

[0049] See also Figure 2 The modal difference features obtained in step 2 Input the difference gating generation unit to obtain the difference gating weights. The calculation process is as follows: In the formula, Indicates the difference gating weight; Represents a 1×1 convolution mapping; This represents the Sigmoid activation function, used to normalize the gate weights to the range of 0 to 1.

[0050] like Figure 2 The modulation path of the differential gating for bimodal features is shown, utilizing differential gating weights. The visible light features after preprocessing were analyzed separately. and infrared features Enhancement is performed to obtain the differentially enhanced visible light features. and infrared features : In the formula, This indicates the visible light characteristics after differential gating enhancement; This indicates the visible light characteristics after preprocessing; Indicates the difference gating weight; This indicates element-wise multiplication.

[0051] In the formula, This indicates the infrared signature after differential gating enhancement; Indicates the infrared features after preprocessing; Indicates the difference gating weight; This indicates element-wise multiplication.

[0052] It should be noted that, as Figure 1 and Figure 2 As shown, the branch that leads to the cross-modal global interaction is not the original one. and Instead, it is enhanced through differential gating. and This connection relationship embodies an important technical feature of the present invention: modal difference enhancement is performed before cross-modal global interaction, so that subsequent interactions pay more attention to the regions with complementary value between the two modalities.

[0053] The principle behind this step is: Generated from the difference information between the two modes, it can therefore indicate regions with complementary value or inconsistencies in visible and infrared features. By... Acting on and The module can enhance the difference areas before subsequent cross-modal interactions, avoiding the problem of dilution of effective complementary information in the traditional simple splicing method.

[0054] Step 4: Cross-modal global interaction.

[0055] like Figure 3 As shown, the cross-modal global interaction branch is enhanced with differences. and As input, channel statistics, spatial weights, and global context information for both modalities are extracted. For the visible light branch, a global description of the visible light is obtained through global average pooling, global max pooling, key feature generation, and value feature generation; for the infrared branch, a global description of the infrared is obtained in the same way.

[0056] like Figure 3 As indicated by the bidirectional arrows in the middle, this branch establishes a bidirectional global guidance relationship between RGB→IR and IR→RGB. Its overall calculation process can be represented as follows: In the formula, This indicates the global enhancement features of visible light after infrared mode guidance; This indicates the visible light characteristics after difference-gated enhancement; Indicates the infrared characteristics enhanced by differential gating; This represents the global guidance enhancement function of the infrared mode on the visible light mode.

[0057] In the formula, This indicates the global infrared enhancement features after being guided by visible light modes; This represents the global guidance enhancement function of the visible light mode on the infrared mode.

[0058] like Figure 1 As shown, the cross-modal global interaction branch output and The water then flowed to three different locations: First, and Enter the discriminative foreground prior generation branch to construct fused features. ;second, and Enter the modal reliability estimation branch, used to generate and ;third, and Entering the joint recalibration branch, it serves as the main feature for judging priors and reliability weight modulation.

[0059] The working principle of this step is that traditional fusion methods typically only stitch or add along the channel dimension, making it difficult to establish cross-modal dependencies between long-distance spatial locations and between different channels. This invention, through bidirectional global interaction, enables the two modalities to guide each other in both the channel and spatial dimensions. The infrared mode can supplement the visible light mode with stable target responses under low-light conditions, while the visible light mode can supplement the infrared mode with texture, edge, and structural information, thereby improving the utilization efficiency of cross-modal complementary information.

[0060] Step 5: Construction of fusion features for prior discriminant analysis.

[0061] like Figure 4 As shown, the discriminative foreground prior generation process first requires constructing fused features. .Will Figure 3 Received , and Figure 2 Received Perform channel splicing, and through Convolutional fusion yields fused features used to discriminate prior-generated features. : In the formula, This represents the fusion features generated from prior information for foreground-background discrimination; This indicates a channel-level concatenation operation; express Convolutional fusion mapping; Indicates RGB global enhancement features; Indicates global enhancement features of IR; This indicates modal difference characteristics.

[0062] Combination Figure 1 and Figure 4 It can be seen that the input for discriminating the prior generation branch only includes , and Modal reliability weights and Do not participate The construction of this property does not participate in the discriminative prior generation process. This connection relationship guarantees the discriminative prior. It is determined by both cross-modal enhancement information and modal difference information, rather than by the reliability weight directly.

[0063] This step aims to unify cross-modal global enhancement information and modal difference information into a single feature representation. (Compared to using only...) and Compared to generating priors, introducing This allows the discriminative prior to simultaneously focus on the enhanced and differential responses of both modes, thereby more accurately distinguishing between the target foreground and background interference.

[0064] Step 6: Construct the multi-response graph.

[0065] like Figure 4 As shown, in obtaining fusion features Afterwards, based on and Construct average response plots respectively Maximum response graph and difference response diagram : In the formula, This represents the average response map, used to reflect the overall activation intensity of the fusion features; This indicates averaging along the channel dimension; This represents the Min-Max normalization operation; This indicates element-wise absolute value operation.

[0066] In the formula, This represents the maximum response map, used to highlight areas of significant local response. This indicates that the maximum value is taken along the channel dimension.

[0067] In the formula, This represents a difference response map, used to reflect the modal difference regions between visible and infrared light; This indicates modal difference characteristics.

[0068] Then, the three response maps are concatenated and subjected to convolutional mapping to obtain the base response map. : In the formula, Represents the basic response diagram; This represents a response-fused convolutional mapping; This represents the Sigmoid activation function.

[0069] like Figure 4 As shown, the average response map, maximum response map, and differential response map together constitute the response fusion input. The average response map reflects the global activation trend, the maximum response map highlights significant target regions, and the differential response map introduces modal complementarity information. The three work together to construct the response, making the base response map... It possesses integrity, significance, and sensitivity to differences.

[0070] Step 7: Divide the soft foreground area and the soft background area.

[0071] See also Figure 4 According to the basic response diagram The mean of the values ​​is used to adaptively generate soft foreground masks. and soft background mask : In the formula, Indicates a soft foreground mask; Represents the basic response diagram; Representing the basic response diagram The mean in the spatial dimension; This represents the temperature coefficient, used to control the smoothness of the soft partitioning; This represents the Sigmoid function.

[0072] In the formula, Indicates a soft background mask; This represents a soft foreground mask.

[0073] like Figure 4 As shown, this step occurs after the response map construction and before the foreground and background prototype extraction. Instead of using a manually fixed threshold, this step adaptively generates soft masks for the foreground and background based on the response distribution of the current input features. Its advantage lies in the fact that the target response intensity may vary under different images, scenes, and lighting conditions, and adaptive soft segmentation is more robust than a fixed threshold.

[0074] Step 8: Extract the foreground and background prototypes.

[0075] like Figure 4 As shown, the soft foreground mask obtained in step 7 is used. and soft background mask , respectively fusion features Perform weighted global average pooling to obtain the foreground prototype. and background prototype : In the formula, Represents the foreground prototype; This represents the fusion features used to discriminate prior-generated features; Indicates a soft foreground mask; This indicates summing over spatial dimensions; This indicates element-wise multiplication; This represents a minimal constant to prevent the denominator from being zero.

[0076] In the formula, Represents the background prototype; This indicates a soft background mask.

[0077] like Figure 4 As shown, the foreground prototype and background prototype are extracted from the soft foreground region and soft background region, respectively. Foreground Prototype The background prototype represents the feature response in the current input that is closer to the global semantic center of the target region. It represents the global semantic center where the feature response in the current input is closer to the background region. By constructing foreground / background prototypes, the module can transform the complex problem of spatial pixel discrimination into a similarity comparison problem with the prototypes.

[0078] Step 9: Foreground-background similarity calculation and prior generation for discrimination.

[0079] like Figure 4 As shown, the fusion features Flatten and normalize the channels, then calculate the relationship between each spatial location and the foreground prototype. Background Prototype Similarity between them: In the formula, Represents a foreground similarity graph; This represents the function for calculating cosine similarity. Indicates fusion characteristics; This represents the foreground prototype.

[0080] In the formula, Represents a background similarity map; This represents the background prototype.

[0081] Then, a discriminative foreground prior is generated by the difference between foreground similarity and background similarity. : In the formula, Represents a discriminative foreground prior graph; Used to filter negative response regions where the background similarity is higher than the foreground similarity; This indicates Min-Max normalization; This represents convolution thinning and Sigmoid mapping operations.

[0082] Combination Figure 1 and Figure 4 It can be seen that, The branch output is generated by the discriminative prior, and in Figure 5 The joint recalibration phase shown is used as a spatial foreground guiding weight. If a certain spatial location is closer to the foreground prototype, then... Greater than This location will be A higher response is obtained at a given spatial location; if a location is closer to the background prototype, its response will be suppressed by ReLU. This results in... It can be used as a spatial guiding weight for the target foreground region for subsequent feature recalibration.

[0083] Step 10: Modal reliability estimation.

[0084] like Figure 5 As shown, the modal reliability estimation branch is based on Figure 3 Output and As input, generate visible light reliability weights respectively. and infrared reliability weight : In the formula, Represents the reliability weight of the visible light mode; Indicates RGB global enhancement features; Indicates global average pooling; Represents visible light reliability mapping; This represents the Sigmoid activation function.

[0085] In the formula, Indicates the reliability weight of infrared modes; Indicates the global enhancement features of IR; This represents infrared reliability mapping.

[0086] like Figure 1 and Figure 5 As shown, and The input sources are respectively and The output destination is the joint recalibration branch. and The discriminative prior generation branch is not entered. This step dynamically determines the reliability of the two modalities based on the current input features. In well-lit scenes with clear textures, the visible light modality may be more reliable; in nighttime, low-light, or occluded conditions, the infrared modality may be more reliable. and The module can avoid the problem of using fixed modal weights in all scenarios.

[0087] Step 11: Determine the joint recalibration of prior and modal reliability.

[0088] like Figure 5 As shown, Figure 3 Received , , Figure 4 Received and the result obtained in step 10 , Input the joint recalibration branch. Also, as... Figure 1 and Figure 5 As shown in the residual connections, the original input features and These are added as residual terms to the corresponding branches. The recalibration process for the visible light branch is as follows: In the formula, This represents the intermediate feature in the visible light branch that is modulated by both the discriminative foreground prior and the visible light reliability weight; Indicates RGB global enhancement features; Represents a discriminative foreground prior graph; Represents the reliability weight of the visible light mode; This indicates element-wise multiplication.

[0089] In the formula, This indicates the visible light characteristics after recalibration; This represents visible light recalibration mapping; This indicates the modulation characteristics of the visible light branch; This represents the original input visible light features, used for residual connections.

[0090] The recalibration process for the infrared branch is as follows: In the formula, This represents the intermediate features in the infrared branch that are modulated by both the discriminative foreground prior and the infrared reliability weights. Indicates the global enhancement features of IR; This represents the reliability weight of the infrared mode.

[0091] In the formula, Indicates the infrared characteristics after recalibration; Indicates infrared recalibration mapping; Indicates the intermediate modulation characteristics of the infrared branch; This represents the original input infrared characteristics, used for residual connections.

[0092] It should be noted that, as Figure 5 As shown, It does not participate in the single-mode recalibration calculations in formulas (24) to (27); Only in the final fusion output stage and and splicing. In formulas (25) and (27) and All features are derived from the original input features, not from preprocessed features. , or after the difference is enhanced , This residual connection is used to preserve the original modal information and avoid the loss of fundamental details due to overmodulation.

[0093] The principle behind this step is: Used to answer the question "Which spatial areas should be strengthened?" and Used to answer "Which modality should be trusted more in the current scenario", while the original residual and This is used to preserve basic information about the input mode. Through the combined action of these three elements, the module can enhance the target region while avoiding synchronous enhancement of background noise and reducing information loss caused by overmodulation.

[0094] Step 12: Final fusion output.

[0095] like Figure 5 As shown, the result obtained in step 11 , Modal difference features obtained in step 2 Perform channel splicing, and through Convolutional fusion yields the final cross-modal fusion enhanced features. : In the formula, This represents the cross-modal fusion enhancement features of the final output; This indicates the final output fusion mapping; This indicates the visible light characteristics after recalibration; Indicates the infrared characteristics after recalibration; This indicates modal difference characteristics.

[0096] Combination Figure 1 and Figure 5 It can be seen that the input of the final fused output branch includes three parts: the recalibrated visible light features. Recalibrated infrared features and modal difference characteristics This step is reintroduced. This is to preserve explicit modal difference information in the final output, ensuring that the output features include not only visible light details and infrared thermal response, but also complementary difference information between the two modes. The final result... It can be used as input features for subsequent object detection networks for object classification and bounding box localization.

[0097] The DPCGRM module of this invention does not simply stitch together visible light and infrared features, but achieves cross-modal fusion enhancement through a continuous processing flow of "difference perception—cross-modal global interaction—discriminative prior generation—modal reliability estimation—joint recalibration fusion output". Among these, Figure 2 The modal difference sensing branch shown is used to detect and enhance intermodal differences; Figure 3 The cross-modal global interaction branch shown is used to establish a bidirectional global complementary relationship between RGB and IR; Figure 4 The discriminative foreground prior generation process shown is used to obtain target foreground space guidance; Figure 5 The joint recalibration fusion output structure shown is used to combine foreground prior, modal reliability and original residual information into the fusion output.

[0098] Therefore, this invention addresses the following technical problems in terms of its working principle: First, it utilizes modal difference perception to solve the problem of insufficient utilization of visible light and infrared difference information; second, it utilizes cross-modal global interaction to solve the problem of insufficient cross-modal complementary modeling in traditional fusion methods; third, it utilizes foreground-background discrimination prior to solve the problem of target regions not being prominent in complex backgrounds; fourth, it utilizes modal reliability estimation to solve the problem of the inability to adaptively adjust modal contributions under different scenarios; and fifth, it utilizes residual joint recalibration to solve the problem of potential loss of original detail information during enhancement. Ultimately, this invention can obtain more stable fusion enhancement features with better foreground discrimination and cross-modal complementarity. This improves the target recognition and localization capabilities in visible-infrared target detection tasks.

[0099] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A perception-discrimination prior-guided visible-infrared cross-modal target detection method, characterized in that, The steps include the following: Step 1, Dual-modal feature preprocessing: Visible light and infrared features are mapped to a unified channel space through 1×1 convolution to obtain preprocessed dual-modal features; Step 2, Modal difference feature generation: Based on the bimodal features preprocessed in Step 1, modal difference features are obtained through absolute difference calculation and convolution extraction; Step 3, Difference Gating Generation and Bimodal Feature Enhancement: Based on the modal difference features from Step 2 and the preprocessed bimodal features from Step 1, the difference-enhanced bimodal features are obtained through gating weight calculation and element-wise weighting. Step 4, Cross-modal global interaction: Based on the bimodal features enhanced by differences in Step 3, cross-modal global enhanced features are obtained through bidirectional global guided enhancement; Step 5, constructing discriminative prior fusion features: Based on the cross-modal global enhancement features from Step 4 and the modal difference features from Step 2, discriminative prior fusion features are obtained through channel concatenation and convolution fusion. Step 6, Multi-response map construction: Based on the discriminative prior fusion features from Step 5 and the modal difference features from Step 2, the basic response map is obtained through channel dimension statistics and convolutional fusion; Step 7, Soft foreground / background region segmentation: Based on the base response map from Step 6, soft foreground and soft background masks are obtained through mean adaptive and Sigmoid mapping; Step 8, Foreground / Background Prototype Extraction: Based on the discriminative prior fusion features from Step 5 and the soft foreground / background mask from Step 7, the foreground prototype and background prototype are obtained through weighted global average pooling. Step 9, Discriminative Foreground Prior Generation: Based on the discriminative prior fusion features from Step 5 and the foreground / background prototypes from Step 8, a discriminative foreground prior map is obtained through similarity calculation and feature mapping; Step 10, Modal reliability estimation: Based on the cross-modal global enhancement features from Step 4, bimodal reliability weights are obtained through global pooling, convolutional mapping, and Sigmoid activation; Step 11, Joint Recalibration: Based on the cross-modal global enhancement features from Step 4, the discriminative foreground prior map from Step 9, the modal reliability weights from Step 10, and the original bimodal features from Step 1, the recalibrated bimodal features are obtained through element-wise weighting, convolutional mapping, and residual connection. Step 12, final fusion output: Based on the recalibrated bimodal features from Step 11 and the modal difference features from Step 2, cross-modal fusion enhancement features are obtained through channel splicing and convolution fusion.

2. The visible-infrared cross-modal target detection method guided by prior perception and discrimination according to claim 1, characterized in that, Step 1 specifically involves: Step 1.1, Input the original visible light features Original infrared features ; Step 1.2, Visible Light Characteristics and infrared features Each process is preprocessed using a dedicated 1×1 convolution mapping: ； ； In the formula, Represents the visible light characteristics of the input; This indicates the visible light characteristics after preprocessing; Represents the visible light branch Convolution preprocessing mapping; Indicates the input infrared features; This indicates the infrared characteristics after preprocessing. Represents the 1×1 convolutional preprocessing mapping for the infrared branch; Step 1.3: Output the preprocessed visible light features. Preprocessed infrared features ; Step 2 specifically involves: Step 2.1, input the output of Step 1 , ; Step 2.2: Calculate the element-wise absolute difference between the two features to generate the initial modal difference response. : ； In the formula, This represents the initial difference response between visible light features and infrared features; This represents element-wise absolute value operation; Step 2.3, initial difference response The input consists of differential feature extraction maps composed of 3×3 convolutions, normalization, and activation functions. Extract modal difference features : ； In the formula, This represents the extracted modal difference features; Indicates by A differential feature extraction mapping consisting of convolution, normalization, and activation functions; Step 2.4, output modal difference features .

3. The visible-infrared cross-modal target detection method guided by prior perception and discrimination according to claim 2, characterized in that, Step 3 specifically involves: Step 3.1, input the output of step 2. Step 1 output ; Step 3.2, Modal Difference Features After 1×1 convolution mapping Sigmoid activation function Processing to generate differential gating weights : ； In the formula, Indicates the difference gating weight; Represents a 1×1 convolution mapping; This represents the Sigmoid activation function, used to normalize the gate weights to the range of 0 to 1; Step 3.3, using Element-wise weighted enhancement The visible light characteristics obtained after difference enhancement : ； In the formula, This indicates the visible light characteristics after differential gating enhancement; This indicates the visible light characteristics after preprocessing; Indicates the difference gating weight; This indicates element-wise multiplication; Step 3.4, using Element-wise weighted enhancement After obtaining the difference-enhanced infrared features : ； In the formula, This indicates the infrared signature after differential gating enhancement; Indicates the infrared features after preprocessing; Indicates the difference gating weight; This indicates element-wise multiplication; Step 3.5: Output the enhanced visible light features. Infrared features after difference enhancement .

4. The visible-infrared cross-modal target detection method guided by prior perception and discrimination according to claim 3, characterized in that, Step 4 specifically involves: Step 4.1, input the output of step 3. , ; Step 4.2: Establish the global guidance relationship from infrared to visible light, and enhance the global guidance function of infrared to visible light. Processing to generate visible light global enhancement features : ； In the formula, This indicates the global enhancement features of visible light after infrared mode guidance; This indicates the visible light characteristics after difference-gated enhancement; Indicates the infrared characteristics enhanced by differential gating; This represents the global guidance enhancement function of the infrared mode on the visible light mode; Step 4.3: Establish the global guidance relationship between visible light and infrared light, and enhance the global guidance function of visible light to infrared light. Processing to generate infrared global augmentation features : ； In the formula, This indicates the global infrared enhancement features after being guided by visible light modes; This represents the global guidance enhancement function of the visible light mode on the infrared mode; Step 4.4: Output visible light global enhancement features Infrared global enhancement features ; Step 5 specifically involves: Step 5.1, input the output of step 4. , Step 2 output ; Step 5.2, will , , splicing along the channel dimension; Step 5.3: The spliced ​​result is then fused and mapped using a 1×1 convolution. Processing and generating discriminative prior fusion features : ； In the formula, This represents the fusion features generated from prior information for foreground-background discrimination; This indicates a channel-level concatenation operation; express Convolutional fusion mapping; Indicates RGB global enhancement features; Indicates the global enhancement features of IR; Indicates modal difference characteristics; Step 5.4, output discriminative prior fusion features .

5. The visible-infrared cross-modal target detection method guided by prior perception and discrimination according to claim 4, characterized in that, Step 6 specifically involves: Step 6.1, input the output of step 5. Step 2 output ; Step 6.2, for Take the absolute value of each element, calculate the mean along the channel dimension, and then normalize using Min-Max to generate the average response plot. : ； In the formula, This represents the average response map, used to reflect the overall activation intensity of the fusion features; This indicates averaging along the channel dimension; This represents the Min-Max normalization operation; This represents element-wise absolute value operation; Step 6.3, for Take the absolute value of each element, find the maximum value along the channel dimension, and then perform Min-Max normalization to generate the maximum response map. : ； In the formula, This represents the maximum response map, used to highlight areas of significant local response. This indicates that the maximum value is taken along the channel dimension; Step 6.4, for Take the absolute value of each element, calculate the mean along the channel dimension, and then normalize using Min-Max to generate a differential response map. : ； In the formula, This represents a difference response map, used to reflect the modal difference regions between visible and infrared light; Indicates modal difference characteristics; Step 6.5, splicing , , Response-fused convolutional mapping Sigmoid activation function Process and generate the basic response graph. : ； In the formula, Represents the basic response diagram; This represents a response-fused convolutional mapping; This represents the Sigmoid activation function; Step 6.6, Output the basic response graph .

6. The visible-infrared cross-modal target detection method guided by prior perception and discrimination according to claim 5, characterized in that, Step 7 specifically involves: Step 7.1, input the output of step 6 ; Step 7.2, Calculate the basic response diagram Spatial dimension mean Combined with temperature coefficient Sigmoid activation function Generate a soft foreground mask : In the formula, Indicates a soft foreground mask; Represents the basic response diagram; Representing the basic response diagram The mean in the spatial dimension; This represents the temperature coefficient, used to control the smoothness of the soft partitioning; This represents the Sigmoid function; Step 7.3, from soft foreground mask Derivation of soft background mask : In the formula, Indicates a soft background mask; Indicates a soft foreground mask; Step 7.4, output the soft foreground mask. Soft background mask ; Step 8 specifically involves: Step 8.1, input the output of step 5. Step 7 output and ; Step 8.2, and Element-wise multiplication, weighted global average pooling, to generate the foreground prototype. : In the formula, Represents the foreground prototype; This represents the fusion features used to discriminate prior-generated features; Indicates a soft foreground mask; This indicates summing over spatial dimensions; This indicates element-wise multiplication; This represents a very small constant to prevent the denominator from being zero; Step 8.3, and Element-wise multiplication, weighted global average pooling, to generate the background prototype. : In the formula, Represents the background prototype; Indicates a soft background mask; Step 8.4, Output the foreground prototype Background Prototype .

7. The visible-infrared cross-modal target detection method guided by prior perception and discrimination according to claim 6, characterized in that, Step 9 specifically involves: Step 9.1, input the output of step 5. Step 8 output and ; Step 9.2, Calculation and Cosine similarity is used to generate a foreground similarity map. : In the formula, Represents a foreground similarity graph; This represents the function for calculating cosine similarity. Indicates fusion characteristics; Represents the foreground prototype; Step 9.3, Calculation and Cosine similarity is used to generate a background similarity map. : In the formula, Represents a background similarity map; Represents the background prototype; Step 9.4: Calculate the similarity difference between the foreground and background, and then perform ReLU activation, Min-Max normalization, and convolutional thinning mapping. Processing to generate a discriminative foreground prior map : In the formula, Represents a discriminative foreground prior graph; Used to filter negative response regions where the background similarity is higher than the foreground similarity; This indicates Min-Max normalization; This represents convolution thinning and sigmoid mapping operations; Step 9.5, output the discriminative foreground prior map. .

8. The visible-infrared cross-modal target detection method guided by prior perception and discrimination according to claim 7, characterized in that, Step 10 specifically involves: Step 10.1, input the output of step 4 , ; Step 10.2, Global average pooling, followed by visible light reliability mapping Sigmoid activation function Processing to generate visible light reliability weights : In the formula, Represents the reliability weight of the visible light mode; Indicates RGB global enhancement features; Indicates global average pooling; Represents visible light reliability mapping; This represents the Sigmoid activation function; Step 10.3, Global average pooling, followed by infrared reliability mapping Sigmoid activation function Processing to generate infrared reliability weights : In the formula, Indicates the reliability weight of infrared modes; Indicates the global enhancement features of IR; Represents infrared reliability mapping; Step 10.4, output visible light reliability weights Infrared reliability weight .

9. The visible-infrared cross-modal target detection method guided by prior perception and discrimination according to claim 8, characterized in that, Step 11 specifically involves: Step 11.1, input the output of step 4 , Step 9 output Step 10 output , Step 1: Original Features , ; Step 11.2, and , Element-wise multiplication generates intermediate modulation features in visible light. : In the formula, This represents the intermediate feature in the visible light branch that is modulated by both the discriminative foreground prior and the visible light reliability weight; Indicates RGB global enhancement features; Represents a discriminative foreground prior graph; Represents the reliability weight of the visible light mode; This indicates element-wise multiplication; Step 11.3, Visible light recalibration mapping The original visible light feature residuals are processed and superimposed to generate recalibrated visible light features. : In the formula, This indicates the visible light characteristics after recalibration; This represents visible light recalibration mapping; This indicates the modulation characteristics of the visible light branch; This represents the characteristics of the original input visible light. Step 11.4, and , Element-wise multiplication generates infrared intermediate modulation features. : In the formula, This represents the intermediate features in the infrared branch that are modulated by both the discriminative foreground prior and the infrared reliability weights. Indicates the global enhancement features of IR; Indicates the reliability weight of infrared modes; Step 11.5, Infrared recalibration mapping The original infrared feature residuals are processed and superimposed to generate recalibrated infrared features. : In the formula, Indicates the infrared characteristics after recalibration; Indicates infrared recalibration mapping; Indicates the intermediate modulation characteristics of the infrared branch; Represents the original input infrared characteristics; Step 11.6: Output the recalibrated visible light characteristics. Infrared characteristics after recalibration .

10. The visible-infrared cross-modal target detection method guided by prior perception and discrimination according to claim 9, characterized in that, Step 12 specifically involves: Step 12.1, input the output of step 11 , Step 2 output ; Step 12.2, will , , splicing along the channel dimension; Step 12.3: The splicing result is then fused and mapped by the final output. Processing to generate cross-modal fusion enhanced features : In the formula, This represents the cross-modal fusion enhancement features of the final output; This indicates the final output fusion mapping; This indicates the visible light characteristics after recalibration; Indicates the infrared characteristics after recalibration; Indicates modal difference characteristics; Step 12.4: Output cross-modal fusion enhancement features The sample is fed into the detection head to complete the target detection.

Images