A visible-infrared image fusion method and system based on nonlinear feature decoupling and texture enhancement

Abstract

The application provides a visible-infrared image fusion method and system based on nonlinear feature decoupling and texture enhancement. The method takes the nonlinear decoupling of basic structural features and texture detail features as the core, reduces the statistical correlation between features and enhances the feature complementarity by introducing nonlinear independence constraints. Further, the adaptive interactive fusion of infrared thermal radiation target information and visible light texture detail information is realized by using a complementary gating fusion mechanism, which improves the structure preservation ability, target saliency and detail fidelity of the fused image. The visible-infrared fusion image generated by the method can provide higher quality multi-modal visual input for night monitoring, intelligent inspection and construction safety risk detection, and help to improve the accuracy and robustness of visual perception tasks in low-light environments.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and deep learning, specifically relating to a visible light-infrared image fusion method and system based on nonlinear feature decoupling and texture enhancement, which is suitable for low-light visual perception scenarios. Background Technology

[0002] In the field of visual environment perception, deep learning-based visual perception methods, benefiting from the strong feature representation and parallel computing capabilities of neural network models, have been widely applied to visual tasks such as object detection, semantic segmentation, and scene understanding, and are gradually becoming an important technical solution for intelligent perception of complex scenes. However, in real-world applications, there are often complex lighting conditions such as low illumination and backlighting, or situations involving partial target occlusion, complex backgrounds, and varied scene structures. These factors can easily make it difficult for a single modality image to continuously and stably provide complete and reliable visual information, thereby affecting the detection accuracy and robustness of downstream visual perception tasks.

[0003] Existing visual perception systems typically employ visible-infrared image fusion, integrating the texture details and structural information of a visible light image with the thermal radiation target information of an infrared image within the same scene to obtain a more complete fused image. However, existing visible-infrared image fusion methods still suffer from problems such as insufficient decoupling between modal shared features and modal-specific features, and insufficient interaction between infrared thermal radiation information and visible light texture information.

[0004] Therefore, there is an urgent need for a visible-infrared image fusion method and system to provide stable and reliable image input for downstream visual perception tasks in complex environments. This method and system need to effectively decouple modal-shared structural information from modal-specific information during the fusion process, and achieve adaptive complementary fusion of infrared thermal radiation target information and visible light texture detail information. This would alleviate problems such as insufficient target saliency, loss of texture details, and unstable scene structure consistency in existing fused images. Summary of the Invention

[0005] To address the aforementioned problems, this invention proposes a visible light-infrared image fusion method and system based on nonlinear feature decoupling and texture enhancement. The method comprises the following steps:

[0006] Step S1: Acquire infrared and visible light images of the same scene, and perform preprocessing such as registration and size adjustment on the infrared and visible light images to obtain source image pairs for image fusion.

[0007] Furthermore, step S1 specifically includes the following steps:

[0008] Step 1-1: Use a pre-deployed dual-modal vision image acquisition device to acquire target images in the same scene. The target images include visible light images and infrared images, and perform initial cropping based on the effective field-of-view overlap area of ​​the two types of images.

[0009] Steps 1-2: Calculate the homography matrix based on the manually selected corresponding points, map the infrared image coordinates to the visible light image coordinate system, and perform cropping, correction, and registration error checks based on the mapping results to ensure that the source image pairs meet the requirements of subsequent fusion processing.

[0010] Step S2: Perform shallow feature extraction on the source image pair obtained in step S1 to obtain shallow features of the infrared image and shallow features of the visible light image, so as to retain the basic visual information in the infrared image and the visible light image.

[0011] Furthermore, step S2 specifically includes the following steps:

[0012] Step 2-1: For the image saved in step S1, use a feature extraction network to obtain the image's feature information. The feature extraction network includes, but is not limited to, convolutional neural networks and Transformer-based feature extraction networks.

[0013] The shallow features are extracted using the feature extraction processing network in step 2-1 for a given input infrared image. and visible light images Through shallow feature mapping function with shared parameters Shallow features were extracted from the infrared and visible light images respectively, yielding:

[0014] ;

[0015] ;

[0016] in, Indicates the first Shallow feature mapping function for each stage, Indicates the infrared image at the 1st The shallow features of each stage Indicates the visible light image at the 1st Shallow features of each stage.

[0017] Step S3: Perform bi-branch feature decomposition on the shallow features of the infrared image and the shallow features of the visible light image obtained in step S2, and apply nonlinear feature decoupling constraints to the different types of features obtained by decomposition to reduce the statistical correlation between different types of features, so as to obtain basic structural features for characterizing the main structure of the scene and detailed features for characterizing modal-specific information, which are used for image reconstruction.

[0018] Furthermore, step S3 specifically includes the following steps:

[0019] Step 3-1: For the image saved in step S2, the obtained shallow features of the infrared image and the shallow features of the visible light image are input into the basic structure feature extraction branch and the texture detail feature extraction branch, respectively. Let the shallow features of any modality be... ,in Indicates infrared mode, Represents the visible light mode.

[0020] Step 3-2: Extract the shallow features through the basic structural feature extraction branch. Global dependency modeling and local structure awareness are performed to extract basic structural features that characterize the main outline, spatial layout, and low-frequency structural information of the scene, represented as follows:

[0021] ;

[0022] in, Indicates the first The basic structural features corresponding to each mode Represents the basic structural feature extraction mapping;

[0023] Step S3-3: Extract the shallow features through the texture detail feature extraction branch. To perform high-frequency texture enhancement processing, the high-frequency residual components are first obtained based on the difference between the shallow features and their low-frequency background.

[0024] ;

[0025] Then, texture gating coefficients are generated based on the high-frequency residual components:

[0026] ;

[0027] The gated and modulated high-frequency residual components are then injected into the shallow features to obtain enhanced texture detail features:

[0028] ;

[0029] in, Indicates the first The high-frequency residual components corresponding to each mode This indicates the average pooling operation. This indicates a batch normalization operation. This represents the activation function. express Convolution operation, This indicates element-wise multiplication, and 𝛼 represents the weighting coefficient for balancing the high-frequency enhancement intensity. Indicates the first Texture detail features corresponding to each modality;

[0030] Step S3-4: Apply the basic structural features obtained in step S3-2 and the texture detail features obtained in step S3-3 The output is used as the result of bi-branch feature decomposition and the model parameters are trained through image reconstruction to provide feature input for subsequent cross-modal complementary fusion.

[0031] Step S4: Perform cross-modal complementary fusion on the basic structural features and detail features obtained in step S3, so that the thermal radiation target information in the infrared image and the texture detail information in the visible light image can adaptively interact to obtain fused features.

[0032] Furthermore, step S4 specifically includes the following steps:

[0033] Step S4-1: Obtain the basic structural features and texture detail features corresponding to the infrared mode and the visible light mode respectively, and rearrange the basic structural features and texture detail features into sample matrix form, denoted as... ,in, Indicates the number of samples.

[0034] Step S4-2: Center the sample matrix and calculate the statistical dependence between features based on the centered sample matrix. The Hilbert-Schmidt independence measure, in linear approximation form, is expressed as:

[0035] ;

[0036] in, and These represent the centered sample matrices, express and The covariance matrix between them and These represent the corresponding autocovariance matrices, It is the Frobenius norm.

[0037] Step S4-3: Construct a nonlinear feature decoupling loss based on the statistical dependence, constraining the basic structural features and texture detail features between the infrared mode and the visible light mode. The nonlinear feature decoupling loss is expressed as:

[0038] ;

[0039] in, and These represent the basic structural features of the infrared mode and the visible light mode, respectively. and These represent the texture detail features of the infrared mode and the visible light mode, respectively. A statistical dependency measure representing the relationship between infrastructure features. This demonstrates a statistical dependency measure between texture detail features. This is a stable term used to prevent the denominator from being zero;

[0040] Step S4-4: Based on the nonlinear feature decoupling loss constraint feature extraction process, the basic structural features of the infrared mode and the visible light mode are kept highly consistent, and the redundant correlation between the texture detail features of the two modes is reduced, thereby obtaining mutually distinguishable and complementary basic structural features and texture detail features.

[0041] Step S5: Generate an image based on the fusion features obtained in step S4, generating a visible light-infrared fused image, and use the fused image for display, storage, or input into subsequent downstream visual processing flows such as target detection.

[0042] Furthermore, step S5 specifically includes the following steps:

[0043] Step S5-1: Perform cross-modal complementary gating interaction on the basic structural features and texture detail features obtained in step S4 to obtain the interacted basic structural features and texture detail features. The calculation method is as follows:

[0044] ;

[0045] ;

[0046] in, , Indicates basic structural features. Indicates texture detail features, This represents the normalization operator. This indicates element-wise multiplication. and These represent the infrared modal characteristics and visible light modal characteristics after complementary gating interaction, respectively;

[0047] Step S5-2: Fuse the basic structural features and texture detail features after interaction to obtain the basic structural fusion features and texture detail fusion features respectively:

[0048] ;

[0049] ;

[0050] in, Indicates texture detail blending mapping, Represents the infrastructure fusion mapping;

[0051] Step S5-3: Integrate the basic structural features Features that blend with texture details Context aggregation is performed to obtain fusion features for image generation, and the fused image is generated. Finally, the fused image is fed as input into the downstream detection network.

[0052] The visible-infrared image fusion system based on nonlinear feature decoupling and texture enhancement includes a dual-modal image acquisition device, a communication device, a computing platform, an image fusion processing module, and a downstream vision processing module. The dual-modal image acquisition device is used to acquire visible light and infrared images of the same scene. This dual-modal image acquisition device includes, but is not limited to, industrial cameras, network cameras, or combined vision acquisition devices composed of the above. The communication device is used to transmit the acquired visible light and infrared images to the computing platform, and supports wired or wireless communication. The computing platform is used to process the visible light and infrared images... The image and infrared image undergo registration, resizing, and normalization preprocessing. The image fusion processing module then performs feature extraction, bi-branch feature decomposition, nonlinear feature decoupling constraints, complementary gating fusion, and image reconstruction to generate a visible-infrared fused image. The downstream vision processing module receives the visible-infrared fused image and inputs it into at least one processing flow in target detection, instance segmentation, or semantic understanding tasks. The computing platform includes a server, an edge computing platform, or a combination of both. The server is equipped with a graphics processor, and the edge computing platform supports image preprocessing, fusion inference, result display, and data transmission.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] 1. This invention fuses visible light images and infrared images of the same scene, making full use of the texture details, edge contours and scene structure information contained in the visible light images, as well as the thermal radiation target information contained in the infrared images. This can effectively make up for the problem of insufficient information expression of single-modal images under low illumination, backlight, occlusion and complex background conditions.

[0055] 2. This invention reduces redundant coupling between features at different levels by performing hierarchical representation and nonlinear decoupling of shared basic structural information and unique texture detail information of modalities, thereby enhancing the distinguishability, independence and complementarity of basic structural features and texture detail features, and thus improving the effectiveness of fused feature expression.

[0056] 3. This invention uses a complementary gating fusion mechanism to adaptively and interactively fuse infrared thermal radiation target information and visible light texture detail information, so that the fused image can enhance the saliency of the target and the fidelity of texture details while maintaining the consistency of scene structure, and reduce the problems of target information loss, edge blurring and detail degradation in low-light complex environments.

[0057] 4. The visible light-infrared fusion image generated by this invention can provide more stable, clear and reliable image input for downstream visual perception tasks such as target detection, semantic segmentation, scene understanding and construction safety risk detection, thereby improving the robustness, accuracy and engineering applicability of multimodal visual perception tasks in complex environments. Attached Figure Description

[0058] Figure 1 is a flowchart of a visible light-infrared image fusion method based on nonlinear feature decoupling and texture enhancement;

[0059] Figure 2 is a diagram of the texture enhancement network;

[0060] Figure 3 is a diagram of a gated complementary fusion network. Detailed Implementation

[0061] To fully illustrate the visible light-infrared image fusion method and system based on nonlinear feature decoupling and texture enhancement proposed in this invention, a specific embodiment is introduced: for safety risk detection tasks in low-light building construction scenarios, the visible light-infrared image fusion method and system described in this invention are used to generate a fused image, and the fused image is used for subsequent safety risk detection. The invention will be further described in detail below with reference to the accompanying drawings.

[0062] This invention employs a dual-modal visual image acquisition device, consisting of a visible light camera and an infrared camera, to capture images of low-light construction sites. At the edge, an NVIDIA Jetson-Xavier edge computing device is used for image acquisition and deployment of image fusion and object detection algorithms. The edge computing device is connected to the monitoring camera via USB. On the server side, a high-performance server equipped with a single GTX 3090 GPU is used for dataset construction and model training. The edge and server sides are connected via a network to enable data uploading and model parameter updates.

[0063] Based on the above hardware system, such as Figure 1 As shown, the visible light-infrared image fusion method based on nonlinear feature decoupling and texture enhancement in this embodiment includes the following steps:

[0064] Step S1: Acquire infrared and visible light images of the same scene, and perform preprocessing such as registration and size adjustment on the infrared and visible light images to obtain source image pairs for image fusion.

[0065] Furthermore, step S1 specifically includes the following steps:

[0066] Step 1-1: Using a pre-deployed dual-modal visual image acquisition device, construction images of workers at the construction site are continuously acquired during the movement of inspection personnel. The target images include visible light images and infrared images, and initial cropping is performed based on the effective overlapping area of ​​the two types of images.

[0067] Steps 1-2: Based on the pre-configured homography matrix, the infrared image coordinates are mapped to the visible light image coordinate system. Cropping, correction, and registration error checks are then performed on the mapped results to obtain visible light-infrared source image pairs. Subsequently, the first 2000 source image pairs are uploaded to the server via HTTPS and REST API using a POST request, and a new visible light-infrared image dataset is constructed.

[0068] Step S2: Perform shallow feature extraction on the source image pair obtained in step S1 to obtain shallow features of the infrared image and shallow features of the visible light image, so as to retain the basic visual information in the infrared image and the visible light image.

[0069] Furthermore, step S2 specifically includes the following steps:

[0070] Step 2-1: For the image saved in step S1, use a feature extraction network to obtain the image's feature information. The feature extraction network includes, but is not limited to, convolutional neural networks and Transformer-based feature extraction networks.

[0071] The shallow features are extracted using the feature extraction processing network in step 2-1 for a given input infrared image. and visible light images Through shallow feature mapping function with shared parameters Shallow features were extracted from the infrared and visible light images respectively, yielding:

[0072] ;

[0073] ;

[0074] in, Indicates the first Shallow feature mapping function for each stage, Indicates the infrared image at the 1st The shallow features of each stage Indicates the visible light image at the 1st Shallow features of each stage.

[0075] Step S3: Perform bi-branch feature decomposition on the shallow features of the infrared image and the shallow features of the visible light image obtained in step S2, and apply nonlinear feature decoupling constraints to the different types of features obtained by decomposition to reduce the statistical correlation between different types of features, so as to obtain basic structural features for characterizing the main structure of the scene and detailed features for characterizing modal-specific information, which are used for image reconstruction.

[0076] Furthermore, step S3 specifically includes the following steps:

[0077] Step 3-1: For the image saved in step S2, the obtained shallow features of the infrared image and the shallow features of the visible light image are input into the basic structure feature extraction branch and the texture detail feature extraction branch, respectively. Let the shallow features of any modality be... ,in Indicates infrared mode, Represents the visible light mode.

[0078] Step 3-2: Extract the shallow features through the basic structural feature extraction branch. Global dependency modeling and local structure awareness are performed to extract basic structural features that characterize the main outline, spatial layout, and low-frequency structural information of the scene, represented as follows:

[0079] ;

[0080] in, Indicates the first The basic structural features corresponding to each mode Represents the basic structural feature extraction mapping;

[0081] Step S3-3: Texture enhancement network as follows Figure 2 As shown, the shallow features are extracted through the texture detail feature extraction branch. To perform high-frequency texture enhancement processing, the high-frequency residual components are first obtained based on the difference between the shallow features and their low-frequency background.

[0082] ;

[0083] Then, texture gating coefficients are generated based on the high-frequency residual components:

[0084] ;

[0085] The gated and modulated high-frequency residual components are then injected into the shallow features to obtain enhanced texture detail features:

[0086] ;

[0087] in, Indicates the first The high-frequency residual components corresponding to each mode This indicates the average pooling operation. This indicates a batch normalization operation. This represents the activation function. express Convolution operation, This indicates element-wise multiplication, and 𝛼 represents the weighting coefficient for balancing the high-frequency enhancement intensity. Indicates the first Texture detail features corresponding to each modality;

[0088] Step S3-4: Apply the basic structural features obtained in step S3-2 and the texture detail features obtained in step S3-3 The output of the bi-branch feature decomposition is used to train the model parameters through image reconstruction, providing feature input for subsequent cross-modal complementary fusion. During the image reconstruction training phase, the loss function is expressed as:

[0089] ;

[0090] in and Indicates the image reconstruction loss. This represents the feature decoupling loss. and The weights of each loss term are represented by hyperparameters; in this example, 1 and 2 are used respectively.

[0091] Step S4: Gated complementary fusion network such as Figure 3 As shown, the basic structural features and detail features obtained in step S3 are subjected to cross-modal complementary fusion, enabling adaptive interaction between the thermal radiation target information in the infrared image and the texture detail information in the visible light image to obtain fused features.

[0092] Furthermore, step S4 specifically includes the following steps:

[0093] Step S4-1: Obtain the basic structural features and texture detail features corresponding to the infrared mode and the visible light mode respectively, and rearrange the basic structural features and texture detail features into sample matrix form, denoted as... ,in, Indicates the number of samples.

[0094] Step S4-2: Center the sample matrix and calculate the statistical dependence between features based on the centered sample matrix. The Hilbert-Schmidt independence measure, in linear approximation form, is expressed as:

[0095] ;

[0096] in, and These represent the centered sample matrices, express and The covariance matrix between them and These represent the corresponding autocovariance matrices, It is the Frobenius norm.

[0097] Step S4-3: Construct a nonlinear feature decoupling loss based on the statistical dependence, constraining the basic structural features and texture detail features between the infrared mode and the visible light mode. The nonlinear feature decoupling loss is expressed as:

[0098] ;

[0099] in, and These represent the basic structural features of the infrared mode and the visible light mode, respectively. and These represent the texture detail features of the infrared mode and the visible light mode, respectively. A statistical dependency measure representing the relationship between infrastructure features. This demonstrates a statistical dependency measure between texture detail features. This is a stable term used to prevent the denominator from being zero;

[0100] Step S4-4: Based on the nonlinear feature decoupling loss constraint feature extraction process, the basic structural features of the infrared mode and the visible light mode are kept highly consistent, and the redundant correlation between the texture detail features of the two modes is reduced, thereby obtaining mutually distinguishable and complementary basic structural features and texture detail features.

[0101] Step S5: Generate an image based on the fusion features obtained in step S4, generating a visible light-infrared fused image, and use the fused image for display, storage, or input into subsequent downstream visual processing flows such as target detection.

[0102] Furthermore, step S5 specifically includes the following steps:

[0103] Step S5-1: Perform cross-modal complementary gating interaction on the basic structural features and texture detail features obtained in step S4 to obtain the interacted basic structural features and texture detail features. The calculation method is as follows:

[0104] ;

[0105] ;

[0106] in, , Indicates basic structural features. Indicates texture detail features, This represents the normalization operator. This indicates element-wise multiplication. and These represent the infrared modal characteristics and visible light modal characteristics after complementary gating interaction, respectively;

[0107] Step S5-2: Fuse the basic structural features and texture detail features after interaction to obtain the basic structural fusion features and texture detail fusion features respectively:

[0108] ;

[0109] ;

[0110] in, This indicates a texture detail blending map. Represents the infrastructure fusion mapping;

[0111] Step S5-3: Integrate the basic structural features Features that blend with texture details Context aggregation is performed to obtain fused features for image generation, and then the fused image is generated. After each sample dataset update, the model is trained on the server, with a total of 120 training iterations and a batch size of 8. The optimizer is Adam, and the initial learning rate is set to... The loss function is set to 0.5 times its current value every 20 rounds. During the image generation training phase, the loss function is expressed as:

[0112] ;

[0113] in, This represents an intensity preservation constraint, used to constrain the fused image to retain significant intensity information from the source image; This represents a gradient preservation constraint, used to constrain the fused image to retain edge contours and texture details. and To minimize the loss weights, this example uses 2 and 2 respectively. After training, the trained model weight parameters are saved for online updates of the inference model. Finally, the updated model parameters are used to perform image fusion processing on the acquired visible light-infrared source images, and the generated fused image is fed into the construction safety risk detection network.

[0114] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any modifications or equivalent changes made based on the technical essence of the present invention shall still fall within the scope of protection claimed by the present invention.

Claims

1. A visible light-infrared image fusion method based on nonlinear feature decoupling and texture enhancement, characterized in that, Includes the following steps: Step S1: Acquire infrared and visible light images of the same scene, and preprocess the infrared and visible light images to obtain source image pairs for image fusion; Step S2: Perform shallow feature extraction on the source image pair obtained in step S1 to obtain shallow features of infrared image and shallow features of visible light image, so as to retain the basic visual information in infrared image and visible light image; Step S3: Perform bi-branch feature decomposition on the shallow features of the infrared image and the shallow features of the visible light image obtained in step S2, and apply nonlinear feature decoupling constraints to the different types of features obtained by decomposition to reduce the statistical correlation between different types of features, so as to obtain the basic structural features used to characterize the main structure of the scene and the detailed features used to characterize the modality-specific information, for image reconstruction. Step S4: Perform cross-modal complementary fusion on the basic structural features and detail features obtained in step S3, so that the thermal radiation target information in the infrared image and the texture detail information in the visible light image can adaptively interact to obtain fused features; Step S5: Generate an image based on the fusion features obtained in step S4, generate a visible light-infrared fused image, and use the visible light-infrared fused image in the downstream visual processing flow.

2. The visible light-infrared image fusion method based on nonlinear feature decoupling and texture enhancement according to claim 1, characterized in that: Step S1 specifically includes the following steps: Step 1-1: Use a pre-deployed dual-modal vision image acquisition device to acquire target images in the same scene. The target images include visible light images and infrared images, and perform initial cropping based on the effective field-of-view overlap area of ​​the two types of images. Steps 1-2: Calculate the homography matrix based on the selected corresponding points, map the infrared image coordinates to the visible light image coordinate system, and perform cropping, correction, and registration error checks based on the mapping results to ensure that the source image pairs meet the requirements of subsequent fusion processing.

3. The visible light-infrared image fusion method based on nonlinear feature decoupling and texture enhancement according to claim 1, characterized in that: Step S2 specifically includes the following steps: For the image saved in step S1, a feature extraction network is used to obtain the feature information of the image. The feature extraction network includes a convolutional neural network and a Transformer-based feature extraction network.

4. The visible light-infrared image fusion method based on nonlinear feature decoupling and texture enhancement according to claim 1, characterized in that: When extracting and decomposing the infrared and visible light image features, a feature coding structure is used to perform shallow feature representation on the input image, and the basic structural information and texture detail information in the shallow features are separated and extracted. Specifically, the basic structural information is subjected to structured coding processing to obtain basic structural features for representing the main structure of the scene; the texture detail information is subjected to enhancement processing to obtain texture detail features for representing modality-specific high-frequency details; the basic structural features and the texture detail features are used for subsequent cross-modal complementary fusion.

5. The visible light-infrared image fusion method based on nonlinear feature decoupling and texture enhancement according to claim 1, characterized in that: The bi-branch feature decomposition and image reconstruction in step S3 specifically includes the following steps: Step 3-1: For the image saved in step S2, the obtained shallow features of the infrared image and the shallow features of the visible light image are input into the basic structure feature extraction branch and the texture detail feature extraction branch, respectively. Let the shallow features of any modality be... ,in Indicates infrared mode, Represents visible light modes; Step 3-2: Extract the shallow features through the basic structural feature extraction branch. Global dependency modeling and local structure awareness are performed to extract basic structural features that characterize the main outline, spatial layout, and low-frequency structural information of the scene, represented as follows: ； in, Indicates the first The basic structural features corresponding to each mode Represents the basic structural feature extraction mapping; Step S3-3: Extract the shallow features through the texture detail feature extraction branch. To perform high-frequency texture enhancement processing, the high-frequency residual components are first obtained based on the difference between the shallow features and their low-frequency background. ； Then, texture gating coefficients are generated based on the high-frequency residual components: ； The gated and modulated high-frequency residual components are then injected into the shallow features to obtain enhanced texture detail features: ； in, Indicates the first The high-frequency residual components corresponding to each mode This indicates the average pooling operation. This indicates a batch normalization operation. This represents the activation function. express Convolution operation, This indicates element-wise multiplication, and 𝛼 represents the weighting coefficient for balancing the high-frequency enhancement intensity. Indicates the first Texture detail features corresponding to each modality; Step S3-4: Apply the basic structural features obtained in step S3-2 and the texture detail features obtained in step S3-3 The output is used as the result of bi-branch feature decomposition and the model parameters are trained through image reconstruction to provide feature input for subsequent cross-modal complementary fusion.

6. The visible light-infrared image fusion method based on nonlinear feature decoupling and texture enhancement according to claim 1, characterized in that: Step S4 specifically includes the following steps: Step S4-1: Obtain the basic structural features and texture detail features corresponding to the infrared mode and the visible light mode respectively, and rearrange the basic structural features and texture detail features into sample matrix form, denoted as... ,in, Indicates the number of samples; Step S4-2: Center the sample matrix and calculate the statistical dependence between features based on the centered sample matrix. The Hilbert-Schmidt independence measure, in linear approximation form, is expressed as: ； in, and These represent the centered sample matrices, express and The covariance matrix between them and These represent the corresponding autocovariance matrices, It is the Frobenius norm; Step S4-3: Construct a nonlinear feature decoupling loss based on the statistical dependence, constraining the basic structural features and texture detail features between the infrared mode and the visible light mode. The nonlinear feature decoupling loss is expressed as: ； in, and These represent the basic structural features of the infrared mode and the visible light mode, respectively. and These represent the texture detail features of the infrared mode and the visible light mode, respectively. A statistical dependency measure representing the relationship between infrastructure features. This demonstrates a statistical dependency measure between texture detail features. This is a stable term used to prevent the denominator from being zero; Step S4-4: Based on the nonlinear feature decoupling loss, perform constrained feature extraction to maintain high consistency of the basic structural features of the infrared mode and the visible light mode, and reduce the redundant correlation between the texture detail features of the two modes, thereby obtaining mutually distinguishable and complementary basic structural features and texture detail features.

7. The visible light-infrared image fusion method based on nonlinear feature decoupling and texture enhancement according to claim 1, characterized in that: Step S5 specifically includes the following steps: Step S5-1: Perform cross-modal complementary gating interaction on the basic structural features and texture detail features obtained in step S4 to obtain the interacted basic structural features and texture detail features. The calculation method is as follows: ； ； in, , Indicates basic structural features. Indicates texture detail features, This represents the normalization operator. This indicates element-wise multiplication. and These represent the infrared modal characteristics and visible light modal characteristics after complementary gating interaction, respectively; Step S5-2: Fuse the basic structural features and texture detail features after interaction to obtain the basic structural fusion features and texture detail fusion features respectively: ； ； in, This indicates a texture detail blending map. Represents the infrastructure fusion mapping; Step S5-3: Integrate the basic structural features Features that blend with texture details Context aggregation is performed to obtain fusion features for image generation, and the fused image is generated. Finally, the fused image is fed as input into the downstream detection network.

8. The visible light-infrared image fusion method based on nonlinear feature decoupling and texture enhancement according to claim 1, characterized in that: The training process and downstream visual processing flow of the method specifically include the following steps: During the image reconstruction training phase, the loss function is expressed as: ； in, and Indicates the image reconstruction loss. This represents the feature decoupling loss. and The hyperparameters representing the weights of each loss term; During the image generation training phase, the loss function is expressed as: ； in, This represents an intensity preservation constraint, used to constrain the fused image to retain significant intensity information from the source image; This represents a gradient preservation constraint, used to constrain the fused image to retain edge contours and texture details. and These are the loss weighting coefficients; After the model training is completed, the visible light image and infrared image to be processed are input into the trained image fusion model to generate a visible light-infrared fused image. The visible light-infrared fused image is then input into any one of the downstream visual processing flows in the target detection, instance segmentation or semantic understanding tasks.

9. A visible light-infrared image fusion system based on nonlinear feature decoupling and texture enhancement, used to implement the method as described in any one of claims 1 to 8, characterized in that: The system includes a dual-modal image acquisition device, communication equipment, a computing processing platform, an image fusion processing module, and a downstream visual processing module; wherein... The dual-modal image acquisition device is used to acquire visible light images and infrared images in the same scene. The dual-modal image acquisition device includes an industrial camera, a network camera, or a combined visual acquisition device consisting of an industrial camera and a network camera. The communication device is used to transmit the acquired visible light images and infrared images to the computing processing platform. The communication device supports wired or wireless communication. The computing platform is used to perform registration, size adjustment and normalization preprocessing on the visible light image and the infrared image, and calls the image fusion processing module to perform feature extraction, bi-branch feature decomposition, nonlinear feature decoupling constraint, complementary gating fusion and image reconstruction processing to generate a visible light-infrared fused image. The downstream visual processing module is used to receive the visible light-infrared fused image and input it into at least one processing flow in target detection, instance segmentation or semantic understanding tasks. The computing platform includes a server, an edge computing platform, or a combination of both. The server is equipped with a graphics processor, and the edge computing platform supports image preprocessing, fusion inference, result display, and data transmission functions.

Images