A polarization image fusion method and system based on multi-scale brightness perception

Abstract

The present invention discloses a polarization image fusion method and system based on multi-scale brightness perception, the method comprising: obtaining a linear polarization image and an intensity image based on the Stokes vector method; constructing a multi-scale brightness perception polarization image fusion network model based on a brightness dynamic weight generation mechanism and a dual-modal feature interaction mechanism; performing feature fusion processing on the linear polarization image and the intensity image based on the multi-scale brightness perception polarization image fusion network model to obtain a polarization fusion image. The present invention can accurately extract special target textures of linear polarization under complex brightness conditions and maintain the details of the intensity image, thereby improving the quality of the fused image. As a polarization image fusion method and system based on multi-scale brightness perception, the present invention can be widely used in the field of image fusion technology.

Description

Technical Field

[0001] The present invention relates to the field of image fusion technology, and in particular to a polarization image fusion method and system based on multi-scale brightness perception. Background Art

[0002] Polarization, as an essential vector property of light waves, reflects the vibration direction of the electric field vector as it propagates through space. Polarization imaging technology analyzes the changes in polarization properties of light waves after reflection from an object, such as the degree of polarization and polarization angle, to obtain multidimensional information such as the object's shape, material, and roughness. This physical correlation opens up a new information dimension for optical imaging. Using the Stokes vector method, polarization degree and polarization angle information can be calculated from the source image, thereby expanding the amount of information from the commonly used three-dimensional information (amplitude, frequency, and phase) to multidimensional information, providing key visual information and breaking through the physical limitations of traditional optical imaging. Because polarization imaging can mine multidimensional information through a single image, it exhibits unique application value in the field of image fusion.

[0003] Related technologies based on deep learning have been widely applied. For example, a novel polarization image fusion model was designed based on target geometry and material polarization characteristics. An image quality assessment method was developed based on the target surface roughness coefficient, specular reflectance, and diffuse reflectance polarization parameters to drive feature information mining. Another approach addresses the problem of poor image quality in underwater optical imaging due to noise and scattering. A neural network combining a frequency decomposition strategy and a residual dense network was proposed, which may also be applied to foggy or nighttime images in the future. Another approach involves designing a dual-channel cross-fusion network that preserves texture information through a multi-attention module. Another approach involves a weak target imaging method based on a dual-discriminator generative adversarial network. This method uses an attention mechanism to guide the fusion of intensity and polarization information, addressing the challenge of weak target detection in bright light. A dual-transposed fusion Transformer was proposed, which exploits the complementary information between the linear polarization map and the intensity image through a cross-transposed attention mechanism. Its innovative gradient median enhancement loss function effectively constrains the fusion process. Furthermore, to address the challenge of weak target detection in SAR, a detection Transformer was combined with a polarization feature weighting module. This dual-channel attention mechanism enhances the representation of ship scattering characteristics, improving detection accuracy under low signal-to-noise ratio conditions. In addition, a color polarization image fusion method considering optical properties is proposed. Through customized loss function and lightweight Transformer architecture, texture details are enhanced while maintaining color fidelity.

[0004] Although the existing polarization image fusion field has achieved good fusion performance, these studies are designed to mine linear polarization information under complex brightness information, resulting in existing polarization image fusion being overly dependent on intensity images, reducing the contribution of the characteristic information of the linear polarization image to the fusion result, thereby reducing the quality of the final fusion result. Summary of the Invention

[0005] In order to solve the above technical problems, the purpose of the present invention is to provide a polarization image fusion method and system based on multi-scale brightness perception, which can accurately extract special target textures of linear polarization degree under complex brightness conditions and maintain the details of the intensity image, thereby improving the quality of the fused image.

[0006] The first technical solution adopted by the present invention is: a polarization image fusion method based on multi-scale brightness perception, comprising the following steps:

[0007] Obtain linear polarization degree image and intensity image based on Stokes vector method;

[0008] Based on the dynamic brightness weight generation mechanism and the dual-modal feature interaction mechanism, a multi-scale brightness-aware polarization image fusion network model is constructed.

[0009] Based on the multi-scale brightness-aware polarization image fusion network model, the linear polarization degree image and the intensity image are fused to obtain a polarization fused image.

[0010] Furthermore, the multi-scale brightness perception polarization image fusion network model specifically includes a splicing module, a texture fusion module, a brightness perception module, an encoder module, a bottleneck module, a decoder module, a convolution module and a brightness enhancement module. The output end of the splicing module is connected to the input end of the texture fusion module, the output end of the texture fusion module and the output end of the brightness perception module are connected to the input end of the encoder module, the output end of the encoder module is connected to the input end of the bottleneck module, the output end of the bottleneck module is connected to the input end of the decoder module, the output end of the decoder module is connected to the input end of the convolution module, and the output end of the convolution module is connected to the input end of the brightness enhancement module, wherein:

[0011] The texture fusion module includes a first dual convolution layer, a first batch normalization layer, a first non-linear activation function, a second dual convolution layer, a second batch normalization layer, a second non-linear activation function and a first convolution attention mechanism module;

[0012] The brightness perception module includes a third dual convolution layer, a third batch normalization layer, a third non-linear activation function, a first maximum pooling layer, a fourth dual convolution layer, a fourth batch normalization layer, a fourth non-linear activation function, a second maximum pooling layer, a fifth dual convolution layer, a fifth batch normalization layer and a fifth non-linear activation function;

[0013] The brightness enhancement module includes a fourth splicing layer, a tenth dual convolution layer, a first ReLU function, an eleventh dual convolution layer, a second ReLU function, a twelfth dual convolution layer and a Sigmoid function.

[0014] Furthermore, the step of performing feature fusion processing on the linear polarization degree image and the intensity image based on the multi-scale brightness perception polarization image fusion network model to obtain the polarization fused image specifically includes:

[0015] Inputting the linear polarization degree image and the intensity image into the multi-scale brightness-aware polarization image fusion network model;

[0016] Based on the splicing module of the multi-scale brightness-aware polarization image fusion network model, the linear polarization degree image and the intensity image are spliced ​​to obtain a preliminary polarization fusion image.

[0017] The texture fusion module based on the multi-scale brightness perception polarization image fusion network model performs texture feature information fusion processing on the preliminary polarization fusion image to obtain a texture feature fusion image;

[0018] The brightness perception module based on the multi-scale brightness perception polarization image fusion network model performs brightness perception calculation on the linear polarization image to obtain multi-level brightness weights;

[0019] The encoder module based on the multi-scale brightness perception polarization image fusion network model combines multi-level brightness weights to encode the texture feature fusion image to obtain the encoded texture feature fusion image;

[0020] Based on the bottleneck module of the multi-scale brightness-aware polarization image fusion network model, the encoded texture feature fusion image is lightweight processed to obtain a lightweight texture feature fusion image;

[0021] The decoder module based on the multi-scale brightness perception polarization image fusion network model decodes the encoded texture feature fusion image and the lightweight texture feature fusion image to obtain a decoded texture feature fusion image;

[0022] Based on the convolution module of the multi-scale brightness perception polarization image fusion network model, the decoded texture feature fusion image is convolved to obtain the convolved texture feature fusion image;

[0023] Based on the brightness enhancement module of the multi-scale brightness-aware polarization image fusion network model, the convolutional texture feature fusion image and the linear polarization degree image are brightness enhanced to obtain the polarization fusion image.

[0024] Furthermore, the texture fusion module based on the multi-scale brightness perception polarization image fusion network model performs texture feature information fusion processing on the preliminary polarization fusion image to obtain a texture feature fusion image, which specifically includes:

[0025] Input the preliminary polarization fusion image into the texture fusion module of the multi-scale brightness-aware polarization image fusion network model;

[0026] Based on the first double convolution layer, the first normalization layer and the first nonlinear activation function of the texture fusion module, texture feature extraction is performed on the preliminary polarization fusion image to obtain a first texture feature image;

[0027] Based on the second double convolution layer, the second batch normalization layer and the second nonlinear activation function of the texture fusion module, the first texture feature image is subjected to texture feature extraction processing to obtain a second texture feature image;

[0028] Adding the first texture feature image and the second texture feature image, and performing feature enhancement processing through the first convolutional attention mechanism module of the texture fusion module to obtain an enhanced texture feature image;

[0029] The enhanced texture feature image is added to the preliminary polarization fusion image and residual connection is performed through the activation function of the texture fusion module to obtain the texture feature fusion image.

[0030] Furthermore, the brightness perception module based on the multi-scale brightness perception polarization image fusion network model performs brightness perception calculation on the linear polarization image to obtain the multi-level brightness weight, which specifically includes:

[0031] Inputting the linear polarization degree image into the brightness perception module of the multi-scale brightness perception polarization image fusion network model;

[0032] Based on the third double convolution layer, the third batch normalization layer and the third nonlinear activation function of the brightness perception module, brightness perception calculation is performed on the linear polarization image to obtain the first-level brightness weight;

[0033] Based on the first maximum pooling layer of the brightness perception module, the first-level brightness weight is subjected to maximum pooling processing to obtain the brightness weight after the first pooling;

[0034] Based on the fourth double convolution layer, the fourth batch normalization layer and the fourth nonlinear activation function of the brightness perception module, the brightness weight after the first pooling is calculated to obtain the second-level brightness weight;

[0035] Based on the second maximum pooling layer of the brightness perception module, the second level brightness weight is subjected to maximum pooling processing to obtain the second pooled brightness weight;

[0036] Based on the fifth double convolution layer, the fifth batch normalization layer, and the fifth nonlinear activation function of the brightness perception module, brightness perception calculation is performed on the brightness weight after the second pooling to obtain the third level brightness weight;

[0037] The first-level brightness weight, the second-level brightness weight, and the third-level brightness weight are combined to obtain a multi-level brightness weight.

[0038] Furthermore, the encoder module based on the multi-scale brightness perception polarization image fusion network model, in combination with the multi-level brightness weight, encodes the texture feature fusion image to obtain the encoded texture feature fusion image, which specifically includes:

[0039] Input the texture feature fusion image into the encoder module of the multi-scale brightness-aware polarization image fusion network model;

[0040] Based on the sixth double convolutional layer of the encoder module and the second convolutional attention mechanism module, texture feature extraction and correction processing are performed on the texture feature fusion image to obtain a first corrected texture feature image;

[0041] The first corrected texture feature image is concatenated with the first-level brightness weight and downsampled through the third maximum pooling layer of the encoder module to obtain a first encoded texture feature fusion image;

[0042] Based on the seventh double convolutional layer and the third convolutional attention mechanism module of the encoder module, texture feature extraction and correction processing are performed on the first encoded texture feature fusion image to obtain a second corrected texture feature image;

[0043] The second corrected texture feature image is concatenated with the second level brightness weight and downsampled through the fourth maximum pooling layer of the encoder module to obtain a second encoded texture feature fusion image;

[0044] Based on the eighth double convolutional layer and the fourth convolutional attention mechanism module of the encoder module, texture feature extraction and correction processing are performed on the second encoded texture feature fusion image to obtain a third corrected texture feature image;

[0045] The third corrected texture feature image is concatenated with the third level brightness weight and downsampled through the fifth maximum pooling layer of the encoder module to obtain a third encoded texture feature fusion image;

[0046] The first encoded texture feature fusion image, the second encoded texture feature fusion image and the third encoded texture feature fusion image are combined to obtain an encoded texture feature fusion image.

[0047] Furthermore, the bottleneck module based on the multi-scale brightness perception polarization image fusion network model performs lightweight processing on the encoded texture feature fusion image to obtain a lightweight texture feature fusion image, which specifically includes:

[0048] The encoded texture feature fusion image is input into the bottleneck module of the multi-scale brightness-aware polarization image fusion network model;

[0049] Based on the ninth double convolution layer of the bottleneck module, feature extraction processing is performed on the encoded texture feature fusion image to obtain a texture feature fusion image after double convolution;

[0050] A lightweight attention mechanism module based on the bottleneck module performs lightweight processing on the texture feature fusion image after double convolution to obtain a preliminary lightweight texture feature fusion image;

[0051] The fifth convolutional attention mechanism module based on the bottleneck module performs feature enhancement processing on the preliminary lightweight texture feature fusion image to obtain a lightweight texture feature fusion image.

[0052] Furthermore, the decoder module based on the multi-scale brightness perception polarization image fusion network model decodes the encoded texture feature fusion image and the lightweight texture feature fusion image to obtain a decoded texture feature fusion image, which specifically includes:

[0053] Input the encoded texture feature fusion image and the lightweight texture feature fusion image into the decoder module of the multi-scale brightness perception polarization image fusion network model;

[0054] Based on the first transposed convolution module of the decoder module, a transposed convolution process is performed on the lightweight texture feature fusion image to obtain a first transposed convolution texture feature fusion image;

[0055] Based on the first splicing layer of the decoder module and the sixth convolutional attention mechanism module, the first transposed convolution texture feature fusion image and the third encoded texture feature fusion image are decoded to obtain a first decoded texture feature fusion image;

[0056] A second transposed convolution module based on the decoder module performs a transposed convolution process on the texture feature fusion image after the first decoding to obtain a second transposed convolution texture feature fusion image;

[0057] Based on the second splicing layer of the decoder module and the seventh convolutional attention mechanism module, the second transposed convolution texture feature fusion image and the second encoded texture feature fusion image are decoded to obtain a second decoded texture feature fusion image;

[0058] Based on the third transposed convolution module of the decoder module, transposed convolution processing is performed on the second decoded texture feature fusion image to obtain a third transposed convolution texture feature fusion image;

[0059] Based on the third splicing layer of the decoder module and the eighth convolutional attention mechanism module, the third transposed convolution texture feature fusion image and the first encoded texture feature fusion image are decoded to obtain a third decoded texture feature fusion image;

[0060] The first decoded texture feature fusion image, the second decoded texture feature fusion image and the third decoded texture feature fusion image are combined to obtain a decoded texture feature fusion image.

[0061] Furthermore, the brightness enhancement module based on the multi-scale brightness perception polarization image fusion network model performs brightness enhancement processing on the convolved texture feature fusion image and the linear polarization degree image to obtain the polarization fusion image, which specifically includes:

[0062] Obtain the channel mean of the linear polarization degree image and perform normalization processing to obtain a reference brightness image;

[0063] The reference brightness image and the convolved texture feature fusion image are input into the brightness enhancement module of the multi-scale brightness-aware polarization image fusion network model;

[0064] Based on the fourth splicing layer of the brightness enhancement module, the reference brightness image and the convolved texture feature fusion image are spliced ​​to obtain a spliced ​​texture feature fusion image;

[0065] Based on the tenth double convolution layer and the first ReLU function of the brightness enhancement module, the brightness attention value is calculated on the spliced ​​texture feature fusion image to obtain the first brightness attention mapping coefficient;

[0066] Based on the eleventh double convolution layer and the second ReLU function of the brightness enhancement module, a brightness attention value is calculated for the first brightness attention mapping coefficient to obtain a second brightness attention mapping coefficient;

[0067] Based on the twelfth double convolution layer and Sigmoid function of the brightness enhancement module, the brightness attention value is calculated for the second brightness attention mapping coefficient to obtain the third brightness attention mapping coefficient;

[0068] The third brightness attention map coefficient is combined with the spliced ​​texture feature fusion image to achieve brightness adaptive correction and obtain a polarization fusion image.

[0069] The second technical solution adopted by the present invention is: a polarization image fusion system based on multi-scale brightness perception, comprising:

[0070] The first module is used to obtain a linear polarization degree image and an intensity image based on a Stokes vector method;

[0071] The second module is used to build a multi-scale brightness-aware polarization image fusion network model based on the brightness dynamic weight generation mechanism and the bimodal feature interaction mechanism;

[0072] The third module is used to perform feature fusion processing on the linear polarization image and the intensity image based on the multi-scale brightness perception polarization image fusion network model to obtain a polarization fusion image.

[0073] The beneficial effects of the method and system of the present invention are as follows: the present invention obtains a linear polarization degree image and an intensity image through the Stokes vector method, and further based on a brightness dynamic weight generation mechanism and a dual-modal feature interaction mechanism, the brightness dynamic weight generation mechanism injects brightness information into the feature map in a dynamic weighted manner to solve the inherent contrast difference problem of the polarization image, and the dual-modal feature interaction mechanism realizes step-by-step guidance from local details to global semantics, constructs a multi-scale brightness-aware polarization image fusion network model, and finally performs feature fusion processing on the linear polarization degree image and the intensity image based on the multi-scale brightness-aware polarization image fusion network model to obtain a polarization fusion image. Through the multi-stage feature interaction mechanism, consistent fusion from local details to global illumination is achieved, and special target textures of the linear polarization degree can be accurately extracted under complex brightness conditions while maintaining the details of the intensity image, thereby improving the quality of the fused image. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 This is a flowchart of the steps of a polarization image fusion method based on multi-scale brightness perception of the present invention;

[0075] Figure 2 This is a structural block diagram of a polarization image fusion system based on multi-scale brightness perception of the present invention;

[0076] Figure 3 Schematic diagram of the structure of a multi-scale brightness-aware polarization image fusion network model provided by a specific embodiment of the present invention;

[0077] Figure 4 is a structural diagram of a brightness perception module provided by a specific embodiment of the present invention;

[0078] Figure 5 is a structural diagram of a texture fusion module provided by a specific embodiment of the present invention;

[0079] Figure 6 Schematic diagram of the structure of a double convolution module provided by a specific embodiment of the present invention;

[0080] Figure 7 is a structural diagram of a brightness enhancement module provided by a specific embodiment of the present invention;

[0081] Figure 8 It is a schematic diagram of simulation experiment results provided by a specific embodiment of the present invention. DETAILED DESCRIPTION

[0082] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. The step numbers in the following embodiments are provided for ease of description only and do not limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted based on the understanding of those skilled in the art.

[0083] The embodiment of the present invention proposes a polarization image fusion network framework based on multi-scale brightness perception, innovatively introduces brightness perception branches to generate multi-scale dynamic weights to guide feature enhancement, and realizes step-by-step guidance from local details to global semantics. It combines the windowed global attention mechanism of Swin-Transformer to optimize the bottleneck layer feature expression, and adopts the CBAM module to realize channel-space dual attention screening. In the encoding-decoding path, details are retained by hybrid upsampling of transposed convolution and bilinear interpolation. Finally, the brightness adaptive enhancement module is used to dynamically modulate the output, achieving a balance between high precision and strong robustness, and is good at processing detail recovery and cross-modal feature collaboration under complex lighting conditions.

[0084] Reference Figure 1 The present invention provides a polarization image fusion method based on multi-scale brightness perception, which includes the following steps:

[0085] S100, acquiring a linear polarization degree image and an intensity image based on a Stokes vector method;

[0086] In the embodiment of the present invention, a focal plane polarization camera is first used to shoot multiple scenes, and then the Stokes vector method is used to analyze the four-directional polarization image data ( ) to obtain the degree of linear polarization DOLP image and intensity I image, and the specific calculation is shown in the following formula:

[0087] ;

[0088] In the above formula, represents the linear polarization degree image, represents the intensity image, Represents the image of the linearly polarized light component in the x-axis direction, Represents the image of the linear polarized light component in the 45° direction, represents the right-handed circularly polarized light component image, express based on Normalized image, express based on Normalized image, express based on Normalized image.

[0089] S200, based on the brightness dynamic weight generation mechanism and the dual-modal feature interaction mechanism, a multi-scale brightness perception polarization image fusion network model is constructed;

[0090] In this embodiment, if Figure 3 As shown, the multi-scale brightness perception polarization image fusion network model specifically includes a splicing module, a texture fusion module, a brightness perception module, an encoder module, a bottleneck module, a decoder module, a convolution module and a brightness enhancement module. The output end of the splicing module is connected to the input end of the texture fusion module, the output end of the texture fusion module and the output end of the brightness perception module are connected to the input end of the encoder module, the output end of the encoder module is connected to the input end of the bottleneck module, the output end of the bottleneck module is connected to the input end of the decoder module, the output end of the decoder module is connected to the input end of the convolution module, and the output end of the convolution module is connected to the input end of the brightness enhancement module. Figure 5 As shown, the texture fusion module includes a first double convolution layer, a first batch of normalization layers, a first nonlinear activation function, a second double convolution layer, a second batch of normalization layers, a second nonlinear activation function and a first convolution attention mechanism module; Figure 4 As shown, the brightness perception module includes a third dual convolution layer, a third batch normalization layer, a third nonlinear activation function, a first maximum pooling layer, a fourth dual convolution layer, a fourth batch normalization layer, a fourth nonlinear activation function, a second maximum pooling layer, a fifth dual convolution layer, a fifth batch normalization layer and a fifth nonlinear activation function; as shown Figure 7 As shown, the brightness enhancement module includes a fourth splicing layer, a tenth dual convolution layer, a first ReLU function, an eleventh dual convolution layer, a second ReLU function, a twelfth dual convolution layer and a Sigmoid function.

[0091] It should also be noted that the double convolution layer in the embodiment of the present invention is as follows: Figure 6 As shown, it includes a first reflection filling layer, a first convolutional layer, a sixth batch of normalization layers, a first activation function, a second reflection filling layer, a second convolutional layer, a seventh batch of normalization layers and a second activation function.

[0092] Furthermore, it should be noted that the method of this embodiment is based on the characteristics of polarization data. In order to focus on retaining the main texture details and balancing the information contribution of different polarization characteristics, the embodiment of the present invention designs a multi-objective joint optimization loss function. , accurately balances structural similarity, pixel accuracy, directional texture, local contrast and model complexity in polarization image fusion, and is specifically defined as follows:

[0093] ;

[0094] in and are the hyperparameters that control the weights of the five loss functions.

[0095] It is a commonly used structural similarity loss function, which is used to measure the similarity between the fusion result and the source image, and helps to preserve the feature details of the source image to the maximum extent. It is defined as follows:

[0096] ;

[0097] in is the fusion result graph, The input source image, or , which represent S0 and DoLP images respectively, and the structural similarity of a single target The calculation is as follows:

[0098] ;

[0099] in Indicates that the image is obtained by Gaussian filtering The local mean of Representing an image The variance of It means the image The covariance of and It is a custom constant used to stabilize calculations.

[0100] It is a pixel-level absolute error loss, which is added to effectively constrain the S0 intensity map to be raised to the global brightness reference and suppress the amplification of DoLP dark noise, thereby reducing pixel-level differences and improving similarity. It is defined as follows:

[0101] ;

[0102] To prevent the detail information advantages of polarized images from being flattened and lost during the fusion process, the model is directly constrained to enhance the brightness and darkness differences in the fused image. The specific definitions are as follows:

[0103] ;

[0104] in represents the image mean of each channel, is a very small positive number used to stabilize the value, is the tensor of the input image. By comparing the horizontal and vertical gradient maps to show the penalty texture loss, the model is forced to generate a clearer and sharper fused image as possible. The specific implementation process is as follows:

[0105] ;

[0106] Two Sobel operators are defined To calculate the gradient of the target image in two directions, It means to find the mean of the absolute values ​​of the elements of XY.

[0107] Finally, in order to control the complexity of the model, prevent overfitting and improve generalization ability, the embodiment of the present invention adds , as shown below:

[0108] ;

[0109] Regularization is achieved by calculating the L2 norm of the model parameters, where Express request The Euclidean norm (L2 norm) of Indicates the model Layers have learnable parameters.

[0110] S300, performing feature fusion processing on the linear polarization degree image and the intensity image based on a multi-scale brightness perception polarization image fusion network model to obtain a polarization fused image.

[0111] S310, inputting the linear polarization degree image and the intensity image into a multi-scale brightness perception polarization image fusion network model;

[0112] S320, a stitching module based on a multi-scale brightness perception polarization image fusion network model, stitching the linear polarization degree image and the intensity image to obtain a preliminary polarization fusion image;

[0113] S330, a texture fusion module based on a multi-scale brightness perception polarization image fusion network model performs texture feature information fusion processing on the preliminary polarization fusion image to obtain a texture feature fusion image;

[0114] Specifically, a preliminary polarization fusion image is input into a texture fusion module of a multi-scale brightness-aware polarization image fusion network model; based on the first double convolution layer, the first batch of normalization layers and the first nonlinear activation function of the texture fusion module, texture feature extraction is performed on the preliminary polarization fusion image to obtain a first texture feature image; based on the second double convolution layer, the second batch of normalization layers and the second nonlinear activation function of the texture fusion module, texture feature extraction is performed on the first texture feature image to obtain a second texture feature image; the first texture feature image and the second texture feature image are added and feature enhancement is performed through the first convolutional attention mechanism module of the texture fusion module to obtain an enhanced texture feature image; the enhanced texture feature image is added to the preliminary polarization fusion image and residual connection is performed through the activation function of the texture fusion module to obtain a texture feature fusion image.

[0115] In this embodiment, the module first processes the data it inputs Perform Conv convolution operation, and then obtain the first layer feature information through batch normalization layer BN and nonlinear activation function ReLU , as shown below:

[0116] ;

[0117] Then thought As the second layer feature input, repeat the above operation to obtain the second layer feature information , and then and Add and generate final feature information through CBAM , and finally and Direct addition is performed and residual connection is realized through activation function ReLU to obtain the final information output , as shown below:

[0118] ;

[0119] Subsequently, in order to improve the sensitivity to the edge details of the final generated image, we use the dual convolution module commonly used in image segmentation tasks as the main way to extract features in Unet, because the dual convolution module mainly avoids the artifacts caused by filling the boundary through reflection filling Reflecpad. Subsequently, through convolution Conv, batch normalization BN, and nonlinear activation function, we achieve high-quality feature enhancement while protecting boundary information.

[0120] S340, a brightness perception module based on a multi-scale brightness perception polarization image fusion network model, performing brightness perception calculation on the linear polarization image to obtain a multi-level brightness weight;

[0121] Specifically, the linear polarization degree image is input into the brightness perception module of the multi-scale brightness perception polarization image fusion network model; based on the third double convolution layer, the third batch normalization layer and the third nonlinear activation function of the brightness perception module, the linear polarization degree image is subjected to brightness perception calculation to obtain the first-level brightness weight; based on the first maximum pooling layer of the brightness perception module, the first-level brightness weight is subjected to maximum pooling processing to obtain the first-pooled brightness weight; based on the fourth double convolution layer, the fourth batch normalization layer and the fourth nonlinear activation function of the brightness perception module, the first-pooled brightness weight is subjected to brightness perception calculation to obtain the second-level brightness weight; based on the second maximum pooling layer of the brightness perception module, the second-level brightness weight is subjected to maximum pooling processing to obtain the second-pooled brightness weight; based on the fifth double convolution layer, the fifth batch normalization layer and the fifth nonlinear activation function of the brightness perception module, the second-pooled brightness weight is subjected to brightness perception calculation to obtain the third-level brightness weight; and the first-level brightness weight, the second-level brightness weight and the third-level brightness weight are combined to obtain multi-level brightness weight.

[0122] In this embodiment, the module designs a three-layer convolutional network to extract brightness features at different scales and generates corresponding spatial attention weights to dynamically enhance brightness-sensitive area information, as shown in the following formula:

[0123] ;

[0124] The combination of the above two modules forms a complete brightness information guidance link. The residual enhancement mechanism ensures that the original information is preserved while avoiding over-enhancement, and the normalization operation improves the generalization ability.

[0125] S350, an encoder module based on a multi-scale brightness perception polarization image fusion network model, combining multi-level brightness weights, encoding the texture feature fusion image to obtain an encoded texture feature fusion image;

[0126] Specifically, the texture feature fusion image is input into the encoder module of the multi-scale brightness perception polarization image fusion network model; based on the sixth double convolution layer and the second convolution attention mechanism module of the encoder module, the texture feature fusion image is subjected to texture feature extraction and correction processing to obtain a first corrected texture feature image; the first corrected texture feature image is spliced ​​with the first-level brightness weight and down-sampled through the third maximum pooling layer of the encoder module to obtain a first encoded texture feature fusion image; based on the seventh double convolution layer and the third convolution attention mechanism module of the encoder module, the first encoded texture feature fusion image is subjected to texture feature extraction and correction processing to obtain a second corrected texture feature image; the second corrected texture feature image is spliced ​​with the first-level brightness weight and down-sampled through the third maximum pooling layer of the encoder module to obtain a first encoded texture feature fusion image; The image is spliced ​​with the second-level brightness weight and down-sampled through the fourth maximum pooling layer of the encoder module to obtain a second encoded texture feature fusion image; based on the eighth double convolution layer and the fourth convolution attention mechanism module of the encoder module, texture features are extracted and corrected on the second encoded texture feature fusion image to obtain a third corrected texture feature image; the third corrected texture feature image is spliced ​​with the third-level brightness weight and down-sampled through the fifth maximum pooling layer of the encoder module to obtain a third encoded texture feature fusion image; the first encoded texture feature fusion image, the second encoded texture feature fusion image and the third encoded texture feature fusion image are combined to obtain an encoded texture feature fusion image.

[0127] In this embodiment, the preliminary feature map then enters a multi-layer coding layer, where each coding layer extracts texture features through double convolution, combines the convolutional attention module (CBAM) to calibrate the feature weights, and uses the multi-level brightness weights obtained by calculating the brightness perception module on the linear polarization degree map to guide the generation of the feature map of this level. Finally, it is downsampled through the maximum pooling layer and input into the next level coding layer.

[0128] S360, a bottleneck module based on a multi-scale brightness perception polarization image fusion network model, performs lightweight processing on the encoded texture feature fusion image to obtain a lightweight texture feature fusion image;

[0129] Specifically, the encoded texture feature fusion image is input into the bottleneck module of the multi-scale brightness-aware polarization image fusion network model; based on the ninth double convolution layer of the bottleneck module, feature extraction processing is performed on the encoded texture feature fusion image to obtain a double-convolution texture feature fusion image; based on the lightweight attention mechanism module of the bottleneck module, lightweight processing is performed on the double-convolution texture feature fusion image to obtain a preliminary lightweight texture feature fusion image; based on the fifth convolution attention mechanism module of the bottleneck module, feature enhancement processing is performed on the preliminary lightweight texture feature fusion image to obtain a lightweight texture feature fusion image.

[0130] In this embodiment, after passing through the three-stage encoder, it enters the bottleneck layer and integrates the advantages of local and global attention through the lightweight SwinBlock attention mechanism and the dual attention mechanism of channel and spatial attention.

[0131] S370, a decoder module based on a multi-scale brightness perception polarization image fusion network model, decoding the encoded texture feature fusion image and the lightweight texture feature fusion image to obtain a decoded texture feature fusion image;

[0132] Specifically, the encoded texture feature fusion image and the lightweight texture feature fusion image are input into the decoder module of the multi-scale brightness perception polarization image fusion network model; based on the first transposed convolution module of the decoder module, the lightweight texture feature fusion image is transposed convolutionally processed to obtain a first transposed convolution texture feature fusion image; based on the first splicing layer and the sixth convolution attention mechanism module of the decoder module, the first transposed convolution texture feature fusion image and the third encoded texture feature fusion image are decoded to obtain a first decoded texture feature fusion image; based on the second transposed convolution module of the decoder module, the first decoded texture feature fusion image is transposed convolutionally processed to obtain a second transposed convolution texture feature fusion image; based on the second splicing layer of the decoder module Together with the seventh convolutional attention mechanism module, the second transposed convolution texture feature fusion image and the second encoded texture feature fusion image are decoded to obtain the second decoded texture feature fusion image; based on the third transposed convolution module of the decoder module, the second decoded texture feature fusion image is transposed convolutionally processed to obtain the third transposed convolution texture feature fusion image; based on the third splicing layer of the decoder module and the eighth convolutional attention mechanism module, the third transposed convolution texture feature fusion image and the first encoded texture feature fusion image are decoded to obtain the third decoded texture feature fusion image; the first decoded texture feature fusion image, the second decoded texture feature fusion image and the third decoded texture feature fusion image are combined to obtain the decoded texture feature fusion image.

[0133] In this embodiment, the third-stage luminance weight-guided feature map and the lightweight texture feature fusion image are used as inputs to the third-stage decoder. The decoder performs transposed convolution on the lightweight texture feature fusion image, then concatenates it with the third-stage luminance weight-guided feature map and outputs it to the next-stage encoder through CBAM.

[0134] S380, a convolution module based on a multi-scale brightness perception polarization image fusion network model, performing convolution processing on the decoded texture feature fusion image to obtain a convolved texture feature fusion image;

[0135] S390, a brightness enhancement module based on a multi-scale brightness perception polarization image fusion network model, performs brightness enhancement processing on the convolved texture feature fusion image and the linear polarization degree image to obtain a polarization fusion image.

[0136] Specifically, the channel mean of the linear polarization degree image is obtained and normalized to obtain a reference brightness map; the reference brightness map and the convolved texture feature fusion image are input into the brightness enhancement module of the multi-scale brightness-aware polarization image fusion network model; based on the fourth splicing layer of the brightness enhancement module, the reference brightness map and the convolved texture feature fusion image are spliced ​​to obtain a spliced ​​texture feature fusion image; based on the tenth dual convolution layer and the first ReLU function of the brightness enhancement module, the brightness attention value of the spliced ​​texture feature fusion image is calculated to obtain a first brightness attention mapping coefficient; based on the eleventh dual convolution layer and the second ReLU function of the brightness enhancement module, the brightness attention value of the first brightness attention mapping coefficient is calculated to obtain a second brightness attention mapping coefficient; based on the twelfth dual convolution layer and the Sigmoid function of the brightness enhancement module, the brightness attention value of the second brightness attention mapping coefficient is calculated to obtain a third brightness attention mapping coefficient; the third brightness attention mapping coefficient is combined with the spliced ​​texture feature fusion image to realize brightness adaptive correction to obtain a polarization fusion image.

[0137] In this embodiment, the decoder upsamples the image layer by layer to obtain a further fused image, which is then convolved with the brightness reference image generated by the linear polarization image and enters the brightness enhancement module, and finally passes through the Sigmoid function to obtain the final fused image.

[0138] Because the linear polarization image can reflect the unique characteristics and information advantages of the object surface, it becomes the main source of advantage of polarization image fusion compared with other image fusions. Therefore, the brightness perception module first calculates the given reference brightness image. (channel mean of the linear polarization image) is normalized to obtain , as shown below:

[0139] ;

[0140] in ; Then the feature map (Feature map after fusion) and Splicing in the channel dimension , and then generate its brightness attention mapping coefficient through the following formula , whose expression is:

[0141] ;

[0142] in For After convolution, pass through the ReLU function. is the Sigmoid function;

[0143] Finally, the following output , realizes adaptive brightness correction of feature maps, where It represents element-by-element multiplication, and its expression is:

[0144] ;

[0145] Finally, in order to further illustrate the effectiveness of the embodiment of the present invention, this embodiment Figure 8 Figure 2 shows the results of two sets of linear polarization image and intensity image fusion comparison methods. In the first set of images, our method achieves the closest color fidelity to the intensity image, while also preserving the enhanced details of the pillar's side texture and highlighting the iron railings from the linear polarization image to the greatest extent possible. In the second set of images, while preserving the detailed textures of both the intensity image and the linear polarization image to the greatest extent possible, it also reduces the interference of noise on the enhanced dark areas, demonstrating the effectiveness of our method in realistic, complex brightness environments.

[0146] As shown in Table 1 for the specific quantitative indicators of the seven methods, it can be seen that the SSIM, VIF, SD, and MS-SSIM of the method in the embodiment of the present invention are all the best, among which QMI far exceeds the indicator of the second best method, indicating that the method in the embodiment of the present invention provides the highest detail richness and the highest consistency with the source image information while being most consistent with the human visual system, further confirming the effectiveness and authenticity of the method. In summary, the method proposed in this embodiment achieves the best effect in balancing subjective vision and objective indicators.

[0147] Table 1 Comparison results between this application and seven image fusion methods

[0148]

[0149] In summary, an embodiment of the present invention proposes a polarization image fusion method based on multi-scale brightness perception, which includes: a brightness dynamic weight generation mechanism, a bimodal feature interaction mechanism, a fusion of local and global attention mechanism, and a brightness adaptive mechanism. In the encoder stage, a multi-scale spatial weight matrix is ​​generated through a brightness branch network, and the brightness information is injected into the feature map in a dynamic weighted manner to solve the inherent contrast difference problem of the polarization image; the bottleneck layer designs a global-local feature fusion mechanism, which maintains compatibility with the convolution feature through lightweight self-attention calculation, and balances the global context and local details through residual connection in the feature dimension reorganization stage; the decoder part establishes a mapping relationship between brightness distribution and texture features through a brightness enhancement module, and uses an adaptive enhancement coefficient to achieve nonlinear brightness correction of the fusion result. This architecture achieves consistent fusion from local details to global illumination through a multi-stage feature interaction mechanism. The present invention can accurately extract special target textures of linear polarization degree under complex brightness conditions and maintain the details of the intensity image, thereby improving the quality of the fused image.

[0150] Reference Figure 2 , a polarization image fusion system based on multi-scale brightness perception, comprising:

[0151] The first module 201 is used to obtain a linear polarization degree image and an intensity image based on a Stokes vector method;

[0152] The second module 202 is used to build a multi-scale brightness perception polarization image fusion network model based on the brightness dynamic weight generation mechanism and the dual-modal feature interaction mechanism;

[0153] The third module 203 is used to perform feature fusion processing on the linear polarization degree image and the intensity image based on the multi-scale brightness perception polarization image fusion network model to obtain a polarization fused image.

[0154] The contents of the above method embodiments are all applicable to the present system embodiments. The functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0155] The above is a specific description of the preferred implementation of the present invention, but the invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A polarization image fusion method based on multi-scale brightness perception, characterized in that: The following steps are involved: Obtain linear polarization degree image and intensity image based on Stokes vector method; Based on the dynamic brightness weight generation mechanism and the dual-modal feature interaction mechanism, a multi-scale brightness-aware polarization image fusion network model is constructed. Based on the multi-scale brightness-aware polarization image fusion network model, the linear polarization image and the intensity image are fused to obtain a polarization fusion image, which includes: Inputting the linear polarization degree image and the intensity image into the multi-scale brightness-aware polarization image fusion network model; Based on the splicing module of the multi-scale brightness-aware polarization image fusion network model, the linear polarization degree image and the intensity image are spliced ​​to obtain a preliminary polarization fusion image. The texture fusion module based on the multi-scale brightness perception polarization image fusion network model performs texture feature information fusion processing on the preliminary polarization fusion image to obtain a texture feature fusion image; The brightness perception module based on the multi-scale brightness perception polarization image fusion network model performs brightness perception calculation on the linear polarization image to obtain multi-level brightness weights; The encoder module based on the multi-scale brightness perception polarization image fusion network model combines multi-level brightness weights to encode the texture feature fusion image to obtain the encoded texture feature fusion image; Based on the bottleneck module of the multi-scale brightness-aware polarization image fusion network model, the encoded texture feature fusion image is lightweight processed to obtain a lightweight texture feature fusion image; The decoder module based on the multi-scale brightness perception polarization image fusion network model decodes the encoded texture feature fusion image and the lightweight texture feature fusion image to obtain a decoded texture feature fusion image; Based on the convolution module of the multi-scale brightness perception polarization image fusion network model, the decoded texture feature fusion image is convolved to obtain the convolved texture feature fusion image; The brightness enhancement module based on the multi-scale brightness perception polarization image fusion network model performs brightness enhancement processing on the convolved texture feature fusion image and the linear polarization degree image to obtain a polarization fusion image; The brightness perception module based on the multi-scale brightness perception polarization image fusion network model performs brightness perception calculation on the linear polarization image to obtain the multi-level brightness weight. Specifically, the step includes: Inputting the linear polarization degree image into the brightness perception module of the multi-scale brightness perception polarization image fusion network model; Based on the third double convolution layer, the third batch normalization layer and the third nonlinear activation function of the brightness perception module, brightness perception calculation is performed on the linear polarization image to obtain the first-level brightness weight; Based on the first maximum pooling layer of the brightness perception module, the first-level brightness weight is subjected to maximum pooling processing to obtain the brightness weight after the first pooling; Based on the fourth double convolution layer, the fourth batch normalization layer and the fourth nonlinear activation function of the brightness perception module, the brightness weight after the first pooling is calculated to obtain the second-level brightness weight; Based on the second maximum pooling layer of the brightness perception module, the second level brightness weight is subjected to maximum pooling processing to obtain the second pooled brightness weight; Based on the fifth double convolution layer, the fifth batch normalization layer, and the fifth nonlinear activation function of the brightness perception module, brightness perception calculation is performed on the brightness weight after the second pooling to obtain the third level brightness weight; Combining the first-level brightness weight, the second-level brightness weight and the third-level brightness weight, a multi-level brightness weight is obtained; The encoder module based on the multi-scale brightness perception polarization image fusion network model encodes the texture feature fusion image in combination with the multi-level brightness weight to obtain the encoded texture feature fusion image. This step specifically includes: Input the texture feature fusion image into the encoder module of the multi-scale brightness-aware polarization image fusion network model; Based on the sixth double convolutional layer of the encoder module and the second convolutional attention mechanism module, texture feature extraction and correction processing are performed on the texture feature fusion image to obtain a first corrected texture feature image; The first corrected texture feature image is concatenated with the first-level brightness weight and downsampled through the third maximum pooling layer of the encoder module to obtain a first encoded texture feature fusion image; Based on the seventh double convolutional layer and the third convolutional attention mechanism module of the encoder module, texture feature extraction and correction processing are performed on the first encoded texture feature fusion image to obtain a second corrected texture feature image; The second corrected texture feature image is concatenated with the second level brightness weight and downsampled through the fourth maximum pooling layer of the encoder module to obtain a second encoded texture feature fusion image; Based on the eighth double convolutional layer and the fourth convolutional attention mechanism module of the encoder module, texture feature extraction and correction processing are performed on the second encoded texture feature fusion image to obtain a third corrected texture feature image; The third corrected texture feature image is concatenated with the third level brightness weight and downsampled through the fifth maximum pooling layer of the encoder module to obtain a third encoded texture feature fusion image; The first encoded texture feature fusion image, the second encoded texture feature fusion image and the third encoded texture feature fusion image are combined to obtain an encoded texture feature fusion image.

2. The polarization image fusion method based on multi-scale brightness perception according to claim 1, characterized in that: The texture fusion module includes a first dual convolution layer, a first batch normalization layer, a first non-linear activation function, a second dual convolution layer, a second batch normalization layer, a second non-linear activation function and a first convolution attention mechanism module; The brightness enhancement module includes a fourth splicing layer, a tenth dual convolution layer, a first ReLU function, an eleventh dual convolution layer, a second ReLU function, a twelfth dual convolution layer and a Sigmoid function.

3. The polarization image fusion method based on multi-scale brightness perception according to claim 2, characterized in that: The texture fusion module based on the multi-scale brightness perception polarization image fusion network model performs texture feature information fusion processing on the preliminary polarization fusion image to obtain the texture feature fusion image. This step specifically includes: Input the preliminary polarization fusion image into the texture fusion module of the multi-scale brightness-aware polarization image fusion network model; Based on the first double convolution layer, the first normalization layer and the first nonlinear activation function of the texture fusion module, texture feature extraction is performed on the preliminary polarization fusion image to obtain a first texture feature image; Based on the second double convolution layer, the second batch normalization layer and the second nonlinear activation function of the texture fusion module, the first texture feature image is subjected to texture feature extraction processing to obtain a second texture feature image; Adding the first texture feature image and the second texture feature image, and performing feature enhancement processing through the first convolutional attention mechanism module of the texture fusion module to obtain an enhanced texture feature image; The enhanced texture feature image is added to the preliminary polarization fusion image and residual connection is performed through the activation function of the texture fusion module to obtain the texture feature fusion image.

4. The polarization image fusion method based on multi-scale brightness perception according to claim 3, characterized in that: The bottleneck module based on the multi-scale brightness perception polarization image fusion network model performs lightweight processing on the encoded texture feature fusion image to obtain a lightweight texture feature fusion image. This step specifically includes: The encoded texture feature fusion image is input into the bottleneck module of the multi-scale brightness-aware polarization image fusion network model; Based on the ninth double convolution layer of the bottleneck module, feature extraction processing is performed on the encoded texture feature fusion image to obtain a texture feature fusion image after double convolution; A lightweight attention mechanism module based on the bottleneck module performs lightweight processing on the texture feature fusion image after double convolution to obtain a preliminary lightweight texture feature fusion image; The fifth convolutional attention mechanism module based on the bottleneck module performs feature enhancement processing on the preliminary lightweight texture feature fusion image to obtain a lightweight texture feature fusion image.

5. The polarization image fusion method based on multi-scale brightness perception according to claim 4, characterized in that: The decoder module based on the multi-scale brightness perception polarization image fusion network model decodes the encoded texture feature fusion image and the lightweight texture feature fusion image to obtain the decoded texture feature fusion image. This step specifically includes: Input the encoded texture feature fusion image and the lightweight texture feature fusion image into the decoder module of the multi-scale brightness perception polarization image fusion network model; Based on the first transposed convolution module of the decoder module, a transposed convolution process is performed on the lightweight texture feature fusion image to obtain a first transposed convolution texture feature fusion image; Based on the first splicing layer of the decoder module and the sixth convolutional attention mechanism module, the first transposed convolution texture feature fusion image and the third encoded texture feature fusion image are decoded to obtain a first decoded texture feature fusion image; A second transposed convolution module based on the decoder module performs a transposed convolution process on the texture feature fusion image after the first decoding to obtain a second transposed convolution texture feature fusion image; Based on the second splicing layer of the decoder module and the seventh convolutional attention mechanism module, the second transposed convolution texture feature fusion image and the second encoded texture feature fusion image are decoded to obtain a second decoded texture feature fusion image; Based on the third transposed convolution module of the decoder module, transposed convolution processing is performed on the second decoded texture feature fusion image to obtain a third transposed convolution texture feature fusion image; Based on the third splicing layer of the decoder module and the eighth convolutional attention mechanism module, the third transposed convolution texture feature fusion image and the first encoded texture feature fusion image are decoded to obtain a third decoded texture feature fusion image; The first decoded texture feature fusion image, the second decoded texture feature fusion image and the third decoded texture feature fusion image are combined to obtain a decoded texture feature fusion image.

6. The polarization image fusion method based on multi-scale brightness perception according to claim 5, characterized in that: The brightness enhancement module based on the multi-scale brightness perception polarization image fusion network model performs brightness enhancement processing on the convolved texture feature fusion image and the linear polarization degree image to obtain the polarization fusion image. This step specifically includes: Obtain the channel mean of the linear polarization degree image and perform normalization processing to obtain a reference brightness image; The reference brightness image and the convolved texture feature fusion image are input into the brightness enhancement module of the multi-scale brightness-aware polarization image fusion network model; Based on the fourth splicing layer of the brightness enhancement module, the reference brightness image and the convolved texture feature fusion image are spliced ​​to obtain a spliced ​​texture feature fusion image; Based on the tenth double convolution layer and the first ReLU function of the brightness enhancement module, the brightness attention value is calculated on the spliced ​​texture feature fusion image to obtain the first brightness attention mapping coefficient; Based on the eleventh double convolution layer and the second ReLU function of the brightness enhancement module, a brightness attention value is calculated for the first brightness attention mapping coefficient to obtain a second brightness attention mapping coefficient; Based on the twelfth double convolution layer and Sigmoid function of the brightness enhancement module, the brightness attention value is calculated for the second brightness attention mapping coefficient to obtain the third brightness attention mapping coefficient; The third brightness attention map coefficient is combined with the spliced ​​texture feature fusion image to achieve brightness adaptive correction and obtain a polarization fusion image.

7. A polarization image fusion system based on multi-scale brightness perception, characterized in that: Includes the following modules: The first module is used to obtain a linear polarization degree image and an intensity image based on a Stokes vector method; The second module is used to build a multi-scale brightness-aware polarization image fusion network model based on the brightness dynamic weight generation mechanism and the bimodal feature interaction mechanism; The third module is used to perform feature fusion processing on the linear polarization degree image and the intensity image based on the multi-scale brightness perception polarization image fusion network model to obtain a polarization fused image, which includes: Inputting the linear polarization degree image and the intensity image into the multi-scale brightness-aware polarization image fusion network model; Based on the splicing module of the multi-scale brightness-aware polarization image fusion network model, the linear polarization degree image and the intensity image are spliced ​​to obtain a preliminary polarization fusion image. The texture fusion module based on the multi-scale brightness perception polarization image fusion network model performs texture feature information fusion processing on the preliminary polarization fusion image to obtain a texture feature fusion image; The brightness perception module based on the multi-scale brightness perception polarization image fusion network model performs brightness perception calculation on the linear polarization image to obtain multi-level brightness weights; The encoder module based on the multi-scale brightness perception polarization image fusion network model combines multi-level brightness weights to encode the texture feature fusion image to obtain the encoded texture feature fusion image; Based on the bottleneck module of the multi-scale brightness-aware polarization image fusion network model, the encoded texture feature fusion image is lightweight processed to obtain a lightweight texture feature fusion image; The decoder module based on the multi-scale brightness perception polarization image fusion network model decodes the encoded texture feature fusion image and the lightweight texture feature fusion image to obtain a decoded texture feature fusion image; Based on the convolution module of the multi-scale brightness perception polarization image fusion network model, the decoded texture feature fusion image is convolved to obtain the convolved texture feature fusion image; The brightness enhancement module based on the multi-scale brightness perception polarization image fusion network model performs brightness enhancement processing on the convolved texture feature fusion image and the linear polarization degree image to obtain a polarization fusion image; The brightness perception module based on the multi-scale brightness perception polarization image fusion network model performs brightness perception calculation on the linear polarization image to obtain multi-level brightness weights, specifically including: Inputting the linear polarization degree image into the brightness perception module of the multi-scale brightness perception polarization image fusion network model; Based on the third double convolution layer, the third batch normalization layer and the third nonlinear activation function of the brightness perception module, brightness perception calculation is performed on the linear polarization image to obtain the first-level brightness weight; Based on the first maximum pooling layer of the brightness perception module, the first-level brightness weight is subjected to maximum pooling processing to obtain the brightness weight after the first pooling; Based on the fourth double convolution layer, the fourth batch normalization layer and the fourth nonlinear activation function of the brightness perception module, the brightness weight after the first pooling is calculated to obtain the second-level brightness weight; Based on the second maximum pooling layer of the brightness perception module, the second level brightness weight is subjected to maximum pooling processing to obtain the second pooled brightness weight; Based on the fifth double convolution layer, the fifth batch normalization layer, and the fifth nonlinear activation function of the brightness perception module, brightness perception calculation is performed on the brightness weight after the second pooling to obtain the third level brightness weight; Combining the first-level brightness weight, the second-level brightness weight and the third-level brightness weight, a multi-level brightness weight is obtained; The encoder module based on the multi-scale brightness perception polarization image fusion network model, combined with multi-level brightness weights, encodes the texture feature fusion image to obtain the encoded texture feature fusion image, specifically including: Input the texture feature fusion image into the encoder module of the multi-scale brightness-aware polarization image fusion network model; Based on the sixth double convolutional layer of the encoder module and the second convolutional attention mechanism module, texture feature extraction and correction processing are performed on the texture feature fusion image to obtain a first corrected texture feature image; The first corrected texture feature image is concatenated with the first-level brightness weight and downsampled through the third maximum pooling layer of the encoder module to obtain a first encoded texture feature fusion image; Based on the seventh double convolutional layer and the third convolutional attention mechanism module of the encoder module, texture feature extraction and correction processing are performed on the first encoded texture feature fusion image to obtain a second corrected texture feature image; The second corrected texture feature image is concatenated with the second level brightness weight and downsampled through the fourth maximum pooling layer of the encoder module to obtain a second encoded texture feature fusion image; Based on the eighth double convolutional layer and the fourth convolutional attention mechanism module of the encoder module, texture feature extraction and correction processing are performed on the second encoded texture feature fusion image to obtain a third corrected texture feature image; The third corrected texture feature image is concatenated with the third level brightness weight and downsampled through the fifth maximum pooling layer of the encoder module to obtain a third encoded texture feature fusion image; The first encoded texture feature fusion image, the second encoded texture feature fusion image and the third encoded texture feature fusion image are combined to obtain an encoded texture feature fusion image.

Images