Pseudo-supervised learning traffic image quality enhancement method based on edge information guidance

Abstract

The invention discloses a pseudo supervised learning traffic image quality enhancement method based on edge information guidance, and aims to solve the problem of target detection precision reduction caused by underexposure and overexposure phenomena. The method comprises the following steps: firstly, designing a visual content generator based on a visible light image-SAM mask; respectively extracting image and mask features and carrying out deep fusion to generate a corresponding enhanced matrix gamma; secondly, designing a pseudo label generator, randomly generating 2N reference images by using the input image, and aiming at providing rich reference data for the model and enhancing generalization and robustness of the model; thirdly, constructing a global-local image quality evaluator to evaluate the image quality of the reference image, and selecting the image with the best quality as a pseudo tag; and finally, L1-SSIM is introduced as the structural reconstruction loss of the model, the detail and structural information of the image is effectively captured, and the visual continuity of the image is ensured while the image quality is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and deep learning technology, and in particular to a pseudo-supervised learning traffic image quality enhancement method based on edge information guidance, which aims to solve the problem of decreased target detection accuracy caused by "underexposure" and "overexposure" phenomena in visible light images in the field of smart transportation. Background Art

[0002] The field of smart transportation has developed rapidly in recent years, and computer vision technology has played an important role in it. Real-time acquisition of images by cameras and processing and analysis using computer vision technology can help optimize traffic management, improve traffic safety and increase traffic efficiency. However, due to the complex and changeable real-world traffic environment, weather problems, lighting changes and other factors, the visible light images acquired by the camera have serious quality problems, among which "overexposure" and "underexposure" are the most common.

[0003] "Overexposure" will cause the image to be too bright, with some areas losing details and white areas or spots appearing, making it impossible to read and recognize information, thus affecting subsequent visual processing and decision-making; while "underexposure" will cause the visible light image to appear dim, with lost details, blurred outlines of objects, and distorted colors, making the image difficult to identify and analyze. This phenomenon will seriously affect the detection accuracy of the target detection algorithm. Therefore, it is necessary to enhance the quality of visible light images to suppress "overexposure" and "underexposure" phenomena and provide a good data foundation for downstream tasks such as target detection.

[0004] Visible light image quality enhancement technology is mainly divided into traditional visible light image enhancement algorithms and image enhancement algorithms based on deep learning. Traditional image enhancement algorithms directly adjust the pixel values ​​of the image to improve the brightness and contrast of the image, thereby obtaining better visual effects. The deep learning-based methods are mainly based on convolutional neural networks and generative adversarial networks. By constructing deep learning network models and using a large amount of data training, they can effectively enhance low-quality visible light images. Although relevant research has made certain progress, traditional methods are sensitive to parameter settings, lack automation and adaptability, and are difficult to adapt to changing lighting environments and different application scenarios; deep learning methods require a large amount of paired training data, which is extremely difficult to obtain in the existing smart transportation field, which makes it difficult for deep learning methods to fit, and then the edges of the image enhancement results are unclear, making it difficult to effectively distinguish the foreground and background. The above limitations make it difficult for mainstream image enhancement methods to guarantee the visual effects of generated images in smart transportation applications. Therefore, it is still a huge challenge to achieve efficient and stable image quality enhancement in the field of smart transportation.

[0005] In response to the above problems, the present invention proposes an edge-guided traffic image enhancement method based on pseudo-supervised learning, which generates image enhancement results with excellent visual effects under different input lighting conditions without the need for paired training data. Specifically, this method cleverly combines the advantages of deep learning technology and traditional image processing methods, and generates corresponding enhancement matrices for different input images through deep learning models, thereby effectively responding to input conditions under various lighting conditions and achieving significant image quality improvement. This technology provides strong data support for downstream tasks such as target detection in the field of smart transportation.

[0006] The technical solution of the present invention not only includes the design and optimization of the algorithm model, but also covers the testing and verification of the algorithm, forming a complete set of technical solutions, which effectively improves the generalization of the image enhancement algorithm in the field of smart transportation, suppresses the "overexposure" and "underexposure" phenomena in traffic scenes, provides a good data basis for downstream tasks such as target detection, and improves the detection accuracy of the detection algorithm. It has important practical application value and broad market prospects. Summary of the invention

[0007] In view of the problems that current methods in the field of intelligent transportation are difficult to train and the difference between foreground and background of generated results is not obvious, the present invention proposes a pseudo-supervised learning traffic image quality enhancement method based on edge information guidance.

[0008] In order to ensure that the model can achieve effective traffic image quality enhancement under the premise of lack of paired datasets, firstly, the Segment Anything Model (SAM) model is used to obtain the SAM mask corresponding to the image, and a visual content generator based on visible light image-SAM mask, RGB-SAMBased Visual Content Generator (R-SVCG), is designed. The image and mask features are extracted respectively using a double-end encoder structure. Then, the Hybrid Attention Fusion Module (HAFM) is used to achieve deep fusion of image features and mask features at the channel and spatial levels, using the edge information provided by the SAM mask as a guide, and the corresponding enhancement matrix γ is generated. Secondly, a pseudo-label generator is designed to randomly generate 2N random pseudo-labels based on the visible light image, which correspond to N brighter and N dimmer images respectively. This design aims to provide the model with rich pseudo-label reference data when facing unknown input lighting conditions, thereby enhancing the generalization and robustness of the model. Then, a global-local image quality evaluator is constructed to comprehensively evaluate the visual effects of the image from three aspects: global image spatial quality, local pixel contrast, and local color saturation, so as to select the image with the best visual effect in the reference image set as the pseudo label for training; finally, L1-SSIM is introduced as the structural reconstruction loss of the model, and the difference between the enhanced result and the pseudo label is minimized by considering the image structural similarity and global similarity at the same time. This loss function can effectively capture the details and structural information of the image, and ensure the visual coherence of the image while improving the image quality enhancement effect.

[0009] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0010] Step 1: Construction and expansion of the data set. For the existing common data sets in the field of smart transportation, data from multiple different shooting positions are selected to enrich the scenes of the data set for the training of deep learning models. Since the images of the current data set are too monotonous and lack sufficient positive and negative samples, the Retinex method is used to process the original image data set to simulate visible light images under different lighting conditions and complete the expansion of the original data set, adding positive and negative samples to the data set. After the data set expansion is completed, the original data is divided into training samples and verification samples in a ratio of 7:3.

[0011] Step 2: Generate an enhancement matrix for image enhancement through a visual content generator based on visible light image-SAM mask image pairs. First, a dual-end encoder is used to obtain the feature information of the SAM mask image and the original image. Secondly, from the perspective of both channels and space, the edge information contained in the SAM features is used as a guide to perform deep fusion with the original image features. Finally, the fused features are input into the decoder to generate an enhancement matrix and used for subsequent gamma transformation to achieve image enhancement.

[0012] Step 3: Use the pseudo-label generator to obtain a set of pseudo-label reference images. The pseudo-label generator performs a gamma transform of random parameters on the original image. The range of the gamma value γ is [0.1, 1) and [1, 4), and obtains N reference labels respectively. Assuming that the image is "underexposed", γ should be less than 1 to enhance the brightness of the dark part of the image; assuming that the image is "overexposed", γ should be greater than 1 to reduce the brightness of the bright part. Finally, 2N reference images are generated, that is, N images that are brighter or darker than the original image are generated.

[0013] Step 4: Use the global-local image quality evaluator to judge the image quality in the pseudo-label reference image set and obtain the pseudo-label. Use the Blind / Referenceless Image Spatial Quality Evaluator (BRISQUE) to calculate the image quality from a global perspective, use the support vector machine (SVM) to learn the mapping relationship between image quality and image features, and calculate the global quality score of the image. In addition, the visual effect of the pseudo-label reference image is evaluated from two levels: local pixel contrast and local color saturation.

[0014] Step 5: Calculate the reconstruction loss function between the pseudo label and the output result, and back-propagate the model parameters of the visual content generator. Train the model through iterative enhancement training strategy until the model fits.

[0015] Step 6: Input the test sample into the trained model to obtain the result after image quality enhancement, and verify the effect of image quality enhancement through human preference evaluation and target detection accuracy.

[0016] Compared with the prior art, the present invention has the following advantages:

[0017] (1) To address the problem of lack of paired datasets in the current smart transportation field, a pseudo-supervised learning method for traffic image quality enhancement based on edge information is proposed. By combining the advantages of deep learning and traditional methods, the deep learning model is used to generate an image enhancement matrix, which is adaptively adjusted for different input images, effectively improving the generalization performance of the model.

[0018] (2) The current deep learning methods are difficult to fit, which makes it difficult to ensure the difference between the foreground and the background after enhancement. That is, the edges of the images after enhancement by the existing methods are not clear. The SAM mask is introduced as edge guidance information, and the features of the SAM mask and the visible light image are extracted by a double-end encoder. The features of the two are fully integrated to effectively enhance the model's ability to distinguish the foreground and background of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is the overall process of the pseudo-supervised learning traffic image quality enhancement method based on edge information guidance;

[0020] Figure 2 It is a schematic diagram of the architecture of the pseudo-supervised traffic image quality enhancement method guided by edge information based on pseudo-supervised learning;

[0021] Figure 3 It is a schematic diagram of the structure of the visible light image-SAM mask based visual content generator (R-SVCG);

[0022] Figure 4 It is a schematic diagram of the structure of the hybrid attention fusion module (HAFM); DETAILED DESCRIPTION

[0023] The present invention is described in detail below in conjunction with a specific implementation case, and the implementation case is named edge-guided image enhancement implementation based on pseudo-supervised learning on a FLIR and LLVIP mixed data set. The following implementation case is for the purpose of enabling technical personnel to understand the specific process of the present invention, but does not limit the present invention in any form. In addition, changes and improvements can be made on the basis of the present invention without departing from the concept of the present invention.

[0024] In related research, deep learning-based image enhancement algorithms aim to achieve image enhancement by directly inputting high-quality images or learning specific parameters of parameter models from paired datasets. However, since collecting paired datasets is extremely time-consuming and difficult to achieve in the field of smart transportation, this method proposes an image enhancement framework based on pseudo-supervised learning to cope with the current situation of lack of paired training data in the field of smart transportation. At the same time, the SAM mask is introduced to guide the network model to distinguish the foreground and background information of the image to generate more accurate image enhancement results.

[0025] The overall process of an edge-guided image enhancement method based on pseudo-supervised learning in this implementation case is as follows: Figure 1 , the method comprises the following steps:

[0026] Step 1: Construct a FLIR and LLVIP mixed dataset.

[0027] In this implementation case, the mainstream datasets FLIR and LLVIP in the field of smart transportation are selected as the main body of the dataset. Images are obtained from the original datasets at a ratio of 2:1 to construct the dataset of this implementation case, and a total of 12,384 images are obtained. The obtained data is uniformly scaled to 640×640 images and the Retinex method is used to obtain images under different lighting conditions. The specific expression is shown in the formula:

[0028] L(x,y)=I(x,y)*G(k,σ) (1)

[0029] E(x,y)=Log(I(x,y))-Log(L(x,y)) (2)

[0030] Where I(x,y) represents the input image data, G represents Gaussian blur, k is the convolution kernel size corresponding to Gaussian blur, σ is the standard deviation of Gaussian blur, L(x,y) represents the result of Gaussian blurring of the input image, which is used to approximate the incident component in the single-scale Retinex algorithm, and E(x,y) represents the result after image enhancement. By setting different Gaussian kernel sizes and adjusting code parameters to simulate "underexposure" and "overexposure" phenomena, in this implementation case, two types of Gaussian convolution kernels, 3×3 and 7×7, are used to process the original image, with a standard deviation of 0.

[0031] The expanded dataset is divided into a ratio of 7:3, with 26,007 training samples and 11,145 validation samples.

[0032] Step 2: Build a pseudo-supervised learning traffic image quality enhancement model framework based on edge information guidance and complete training

[0033] The framework structure of the edge-guided traffic image quality enhancement model based on pseudo-supervised learning is as follows: Figure 2 As shown, it mainly includes a visible light-mask image based visual content generator (R-SVCG), a pseudo label generator and an image quality assessor.

[0034] The visible light-mask image-based visual content generator (R-SVCG) is the main body of the overall model, and its specific structure is as follows: Figure 3 As shown. First, the Segment Anything Model (SAM) model is used to obtain the SAM mask corresponding to the image, and the feature information of the visible light image and the SAM mask is extracted respectively through the dual-encoder. The SAM feature information and the image feature are fused with each other through the hybrid attention fusion module (HAFM) to fully integrate the complementary information of the two. Finally, after obtaining the fused features, the decoder generates the image enhancement matrix γ and performs a gamma transform on the visible light image. The specific expression is shown in the formula:

[0035] F r ,F s =Dual-Encoder(I r ,I s ) (3)

[0036] F fus =HAFM(F r ,F s ) (4)

[0037]

[0038] Among them I r ,I s Represent the input image and SAM mask result respectively, F r ,F s The image features and mask features obtained by the dual-encoder are fused by the hybrid attention fusion module (HAFM) to obtain the fusion feature F fus , and use the decoder to gradually predict the enhancement matrix γ from the features, and perform gamma transformation on the original visible light image based on the enhancement matrix γ to obtain the enhanced image I e .

[0039] The Dual-Encoder is composed of five layers of sequentially iterative deep learning modules. The construction order of the dual-end encoder is 1×1 convolution layer 1×1Conv, deep separable convolution DWConv, activation function Hswish, 3×3 convolution kernel 3×3Conv and compression-excitation module SE. 1×1Conv and 3×3Conv are used together to extract features of different dimensions to effectively enhance the nonlinear expression ability of the network; DWConv separable convolution performs channel and point-by-point convolution on the feature map to retain the ability of feature extraction while improving computational efficiency; SE module enhances the expression ability of channel features by introducing global information. The number of output channels of each layer of the 5-layer network is set to (16, 16, 32, 32, 16). By changing the number of channels layer by layer, image features are effectively extracted and represented.

[0040] After feature extraction, the two features are fused using a hybrid attention fusion module (HAFM). The structure of the hybrid attention fusion module (HAFM) is as follows: Figure 4As shown in the figure. The module first performs maximum pooling and average pooling on the channel dimension to obtain the feature components of the image feature and mask feature respectively, and then learns the feature components through the linear layer. Through learning, the model can adaptively determine which channels in the SAM feature are more important to the current task, thereby providing weights for subsequent fusion. Then, the SAM feature and the fused feature are input into a weight-sharing convolutional layer to complete the feature fusion from the spatial level to obtain the final fused feature F. fus The specific expression is shown in the formula:

[0041] F fc =Tanh(Linear(Linear(Adapt-avgpool c (F s )+Adapt-maxpool c (F s )) (6)

[0042] +Linear(Adapt-avgpool c (F r )+Adapt-maxpool c (F r ))))·F r

[0043] F fus =Tanh(Concat(7×7Conv(F s ),7×7Conv(F fc )))·F fc (7)

[0044] where F r ,F s are the image features and mask features obtained by the dual-encoder, F fc Represents the features after channel attention fusion, Adapt-avgpool c ,Adapt-maxpool c It is an adaptive pooling layer in the Pytorch framework. It performs maximum pooling and average pooling on the feature map based on the channel level and ensures that the size of the feature map before and after the output is consistent. It calculates the maximum value and average value of each fixed-size grid on the feature channel dimension and adds the two to obtain the corresponding feature component. The linear layer Linear is used to perform linear transformation on the original data, and the generalization of the model is enhanced through learning. The Tanh activation function is applied to obtain the final fusion weight and complete the channel fusion to obtain F fc , the Tanh activation function formula is:

[0045]

[0046] After channel fusion is completed, F fc and the original SAM feature F s In the 7×7Conv with shared input weights, the convolution kernel step size is 1 and the padding is 3. The spatial feature components of visible light and SAM mask are obtained respectively and the feature components are fused through the Concat dimension splicing method. The spatial fusion weight is obtained through the Tanh activation function to complete the feature level fusion and obtain the fusion feature F fus .

[0047] The decoder structure is similar to the encoder structure, generating an enhancement matrix γ and performing a gamma transform on the original image, which is specifically expressed as shown in the formula:

[0048] γ=MLP(Up(Concat(F fus ,3×3Conv(Up(F fus ))))) (9)

[0049] The decoder is composed of a 3×3 convolution kernel 3×3Conv with a step size of 1, an upsampling Up, and a multi-layer perceptron MLP. The upsampling Up adopts a bilinear interpolation method to generate new pixels by performing weighted calculations between existing pixels to realize the fusion feature F fus The fusion feature is input into 3×3Conv for information extraction. The Concat channel dimension concatenation method is used to concatenate the feature F fus The information is connected with the initial decoded features to effectively retain the information, and the spliced ​​information is input into the multi-layer perceptron MLP to build a complex function mapping to complete the feature decoding and obtain the enhancement matrix γ. Based on the enhancement matrix, the original visible light image is gamma transformed to obtain a single training result.

[0050] The pseudo-label generator is used to generate photos of the original image that may appear under different lighting conditions. The gamma transform is used, and the gamma value γ′ ranges from [0.1, 1) to [1, 4). It is assumed that when the image is "underexposed", γ′ should be less than 1 to enhance the brightness of the dark part of the image; and when the image is "overexposed", γ′ should be greater than 1 to reduce the brightness of the bright part. In this implementation case, 2 γ′ are randomly selected from two value ranges to obtain pseudo-label images. In this example, 0.3, 0.7, 1.5, and 3.5 are selected for pseudo-label generation. The specific expression is shown in the formula:

[0051]

[0052] in, represents the generated pseudo-label reference image set, which contains a total of 2N reference pseudo-labels, I rrepresents the input visible light image, represents the nth processed pseudo-label reference image; rand(i,(a,b)) represents the i-th gamma value γ′ obtained from the range (a,b).

[0053] The global-local image quality evaluator is used to calculate the quality scores of images in the pseudo-label reference image set, and select the image with the highest quality score as the pseudo-label of the model to complete the training. The image quality evaluator is mainly divided into two parts. The global image quality evaluation is implemented using the reference-free spatial image quality evaluator (BRISQUE). By extracting features such as local contrast and brightness of the image, the natural scene statistics (NSS) model is used to capture the natural visual characteristics of the image, and then the quality score of the image is calculated through the trained support vector regression (SVM) model. In addition, the local image quality evaluation is used to measure the vividness and visual effect of the image by evaluating the local pixel contrast and local color saturation of the image. The specific steps are shown in the formula:

[0054]

[0055] Among them, P x,y represents a patch centered at the (x, y) pixel, whose size is K×K; I(i, j) represents the pixel value corresponding to the i, j position, and μ is the pixel mean of the patch; calculate L c To represent the local pixel contrast of the image. In addition, C s Indicates the local color saturation of the image, using the saturation channel of the HSV color space to measure the color saturation of the pixel, where R, G, and B correspond to the pixel values ​​of the three channels of the visible light image at the (x, y) position.

[0056] In this implementation case, after the deep learning framework is built, the specific training steps of the pseudo-supervised edge-guided image enhancement model can be divided into the following steps:

[0057] 2.1 Use the SAM model to obtain the SAM mask corresponding to the visible light image.

[0058] 2.2 The visible light image and SAM mask image are input into the visual content generator based on visible light image-SAM mask image pair (R-SVCG). After extracting features respectively, the feature information of the two is fused using the hybrid attention fusion module (HAFM). The fused feature map is input into the decoder to generate an image enhancement matrix and the visible light image is gamma transformed according to the matrix.

[0059] 2.3 Input the visible light image into the pseudo-label generator to obtain the pseudo-label reference image set, and add the n-1th training result to the pseudo-label reference image set. Use the global-local image quality evaluator to calculate the quality score of the image in the reference set, and select the highest score as the pseudo-label for model training.

[0060] 2.4 Calculate the loss function between the pseudo-label and the output result, and back-propagate the entire model. Use the iterative enhancement training strategy to continuously iterate the training to enhance the image enhancement effect of the model and retain the corresponding model weights.

[0061] In this embodiment, the image size input into the leakage condition recognition model is 640×640, the initial learning rate during model training is set to 0.01, the momentum parameter is 0.937, the decay rate is set to 0.0005, and the stochastic gradient descent algorithm is used to train the recognition model for 100 rounds.

[0062] The model uses L1-SSIM as the structured reconstruction loss. The L1-SSIM loss function calculates the structural difference between the output image and the pseudo label, as shown in the formula:

[0063]

[0064] L c =α·SSIM(I o ,I p )+(1-α)·L1(I o ,I p ) (14)

[0065] SSIM operates on a pixel window of fixed size, x and y represent two different input images, μ x ,μ y Represents the average value of pixels in the window at the same x,y position. Represents the corresponding window standard deviation, σ xy is the covariance of the window at the same position of x and y, c1 and c2 are set as fixed proportions of the input pixel value range. This parameter is used to maintain the stability of the calculation results and is set to 6.5025 and 58.5025 respectively. The L1 loss function calculates the absolute difference between the pseudo label and the output image, where I o Represents the image output of the model, I p Represents the pseudo-label image. α is a learnable parameter obtained through model iterative learning and is used to balance the loss function. The initial value is set to 0.5. When the training round reaches 20, the value tends to be stable, about 0.275.

[0066] Step 3: Input the test sample into the trained model for image enhancement, obtain the enhanced image and evaluate its image quality. In this implementation case, due to the lack of paired data sets in the current smart transportation field to effectively evaluate the experimental results, a no-reference image quality assessment method is used to judge the image quality. The natural image quality assessment method (NIQE) based on pixel discrimination and the no-reference image quality assessment method (BIECON) based on convolutional neural network are selected to calculate the image quality score for quantifying the quality of the image, where the lower the value of the natural image quality assessment method (NIQE), the better the image quality, and the higher the score of the no-reference image quality assessment method (BIECON), the better the image quality. Experimental data show that the method of the present invention is used to enhance 8948 verification samples in the mainstream LFIR and LLVIP datasets, calculate the mean of all picture quality scores and compare them with other advanced methods. In the FLIR dataset, the NIQE score is 9.968 and the BIECON score is 0.945, both of which are better than other advanced image enhancement methods. These results not only demonstrate the effectiveness of our proposed method, but also indicate the potential of image enhancement through deep learning technology in the field of smart transportation.

[0067]

[0068] The above describes the specific implementation cases of the present invention. It should be noted that the present invention is not limited to the above specific implementations, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essential content of the present invention. In the absence of conflict, the implementation cases of the present invention application and the features in the implementation cases can be arbitrarily combined with each other.

Claims

1. A pseudo-supervised learning traffic image quality enhancement method based on edge information guidance, characterized by: Step 1: Construct a visible light traffic image dataset and complete data expansion. Randomly sample and merge the dataset to construct a visible light dataset and use the Retinex image enhancement method to perform data expansion. Step 2: Construct a visual content generator based on visible light image-SAM mask image to obtain the enhancement matrix and image quality enhancement results, use the Segment Anything Model (SAM) large model to obtain the corresponding SAM mask and input the visible light image and SAM mask into the visual content generator to generate the image enhancement matrix γ and obtain the enhanced image, i.e., the single output result; Step 3: Use the pseudo-label generator to construct a pseudo-label reference image set. The pseudo-label generator randomly generates multiple pseudo-label images with different exposure levels to cope with different input environments, and merges them with the training results of the previous round to construct a pseudo-label reference image set. Step 4: Build a global-local image quality evaluator and evaluate the quality of images in the reference set, calculate the quality scores of images in the reference image set, and select the image with the highest score as the pseudo label to complete the model training; Step 5: Calculate the reconstruction loss function between the pseudo label and the output result, implement backpropagation of the visual content generator and update the model parameters, and retain the model weights after model fitting; Step 6: Input the test sample into the trained model to obtain the image quality enhanced result, and use the no-reference image quality assessment method to verify the image quality enhancement effect.

2. The method according to claim 1, characterized in that: Step 1 is to use the Retinex algorithm to expand the original image data set, adjust the brightness of the original visible light image to simulate the "overexposure" and "underexposure" phenomena, and add positive and negative samples to the data set to expand the data set.

3. The method according to claim 1, characterized in that: Step 2 is specifically based on the visual content generator (R-SVCG) based on the visible light image-SAM mask image pair. It uses a double-end encoder to extract the feature information of the visible light image and the SAM mask image respectively, and fuses the SAM features with the visible light features through a hybrid attention fusion module (HAFM); after obtaining the fused features, the decoder generates the image enhancement matrix γ and performs a gamma transform on the image to achieve image quality enhancement. The specific expression is shown in the formula: F r ,F s =Dual-Encoder(I r ,I s )F fus =HAFM(F r ,F s )I e =I r γ =I r Decoder(Ffus) Among them I r ,I s Represent the input image and SAM mask result respectively, F r ,F s The image features and mask features obtained by the dual-encoder are fused by the hybrid attention fusion module (HAFM) to obtain the fusion feature F fus , and use the decoder to gradually predict the enhancement matrix γ from the features, and perform gamma transformation on the original visible light image based on the enhancement matrix γ to obtain the enhanced image I e ; The dual-end encoder is composed of 5 layers of sequentially iterative deep learning modules. The dual-end encoder is constructed in the order of 1×1 convolution layer 1×1Conv, depthwise separable convolution DWConv, activation function Hswish, 3×3 convolution kernel 3×3Conv and compression-excitation module SE; 1×1Conv and 3×3Conv are used together to extract features of different dimensions to effectively enhance the nonlinear expression ability of the network; the number of output channels of each layer of the 5-layer network is set to (16, 16, 32, 32, 16); After feature extraction, the hybrid attention fusion module (HAFM) is used to fuse the two features. The module first performs maximum pooling and average pooling on the channel dimension to obtain the feature components of the image feature and the mask feature respectively, and then learns the feature components through the linear layer; Then, the SAM feature and the fusion feature are input into a weight-sharing convolutional layer to complete the feature fusion from the spatial level to obtain the final fusion feature F fus , specifically expressed as shown in the formula: F fc =Tanh(Linear(Linear(Adapt-avgpool c (F s )+Adapt-maxpool c (F s )) +Linear(Adapt-avgpool c (F r )+Adapt-maxpool c (F r ))))·F r F fus =Tanh(Concat(7×7Conv(F s ),7×7Conv(F fc )))·F fc where F r ,F s is the image feature and mask feature obtained by the dual-encoder, F fc Represents the features after channel attention fusion, Adapt-avgpool c ,Adapt-maxpool c It is an adaptive pooling layer in the Pytorch framework. It performs maximum pooling and average pooling on the feature map based on the channel level and ensures that the size of the feature map before and after the output is consistent. It calculates the maximum value and average value of each fixed-size grid on the feature channel dimension and adds the two to obtain the corresponding feature component. It uses the linear layer Linear to perform linear transformation on the original data and enhances the generalization of the model through learning. It applies the Tanh activation function to obtain the final fusion weight and complete the channel fusion to obtain F. fc , the Tanh activation function formula is: After channel fusion is completed, F fc and the original SAM feature F s In the 7×7Conv with shared input weights, the step size is 1 and the padding is 3. The feature components of visible light and SAM mask are obtained respectively and the channel dimension is spliced ​​through Concat. The spatial fusion weight is obtained through the Tanh activation function to complete the feature level fusion and obtain the fusion feature F fus ; The decoder performs gamma transformation on the original image to suppress the "underexposure" and "overexposure" phenomena. The specific expression is shown in the formula: γ=MLP(Up(Concat(F fus ,3×3Conv(Up(F fus ))))) The decoder consists of a 3×3 convolution kernel with a step size of 1, an upsampling Up, and a multi-layer perceptron MLP. The upsampling uses a bilinear interpolation method to generate new pixels by performing weighted calculations between existing pixels to achieve the fusion of the features F. fus The fusion feature is input into 3×3Conv for information extraction; Concat is used to extract the feature F fus The information is concatenated with the decoded features to effectively retain the information, and the information is input into the multi-layer perceptron MLP to build a complex function mapping to complete feature decoding and obtain the enhanced matrix γ.

4. The method according to claim 1, characterized in that: The pseudo-label generator generates reference images of various lighting conditions by adjusting the gamma value γ′ of the gamma transform; the value range of γ′ is [0.1, 1), [1, 4). It is assumed that when the image is "underexposed", γ′ should be less than 1 to enhance the brightness of the dark part of the image, and when the image is "overexposed", γ′ should be greater than 1 to reduce the brightness of the bright part; in this implementation case, N γ′ values ​​are randomly selected in the two value ranges to obtain pseudo-label images, that is, N brighter or darker images are generated in the above two ranges, and the specific expression is shown in the formula: in, represents the generated pseudo-label reference image set, which contains a total of 2N reference pseudo-labels, I r represents the input visible light image, represents the nth processed pseudo-label reference image; rand(i,(a,b)) represents the i-th gamma value γ′ randomly obtained from the range (a,b).

5. The method according to claim 1, characterized in that: The global-local image quality evaluator is used to calculate the quality scores of images in the pseudo-label reference image set, and selects the image with the highest quality score as the pseudo-label of the model to complete the training; the global image quality evaluation is implemented using the reference-free spatial image quality evaluator (BRISQUE), which extracts the local contrast and brightness of the image, uses the natural scene statistics (NSS) model to capture the natural visual characteristics of the image, and then calculates the image quality score through the trained support vector regression (SVM) model; in addition, the local image quality evaluation is used to measure the vividness and visual effect of the image by evaluating the local pixel contrast and local color saturation of the image. The specific steps are shown in the formula: Among them, P x,y represents a patch centered at the (x, y) pixel, whose size is K×K; I(i, j) represents the pixel value corresponding to the i, j position, and μ is the pixel mean of the patch; calculate L c To represent the local pixel contrast of the image; in addition, C s Indicates the local color saturation of the image, using the saturation channel of the HSV color space to measure the color saturation of the pixel, where R, G, and B correspond to the pixel values ​​of the three channels of the visible light image at the (x, y) position.

6. The method according to claim 1, characterized in that: L1-SSIM is proposed as the reconstruction loss function of the model to calculate the structural difference between the output image and the pseudo label, as shown in the formula; L c =α·SSIM(I o ,I p )+(1-α)·L1(I o ,I p ) SSIM operates on a pixel window of fixed size, x and y represent two different input images, μ x ,μ y Represents the average pixel value of the same position window in two different input images x and y. Represents the corresponding window standard deviation, σ xy is the covariance of the window at the same position of x and y, c1 and c2 are set as fixed proportions of the input pixel value range. This parameter is used to maintain the stability of the calculation results and is set to 6.5025 and 58.5025 respectively; the L1 loss function calculates the absolute difference between the pseudo label and the output image, where I o Represents the image output of the model, I p Represents the pseudo-label image, and α is a learnable parameter obtained through model iterative learning to balance the loss function.

Images