Dark light image enhancement method based on traffic scene

Abstract

The invention provides a dark light image enhancement method based on a traffic scene, and relates to the technical field of image enhancement, and the method comprises the steps: obtaining a visible light image and a thermal imaging image which are synchronously collected in the same traffic scene; inputting the visible light image and the thermal imaging image into a dark light image enhancement model to obtain an enhanced image; wherein the dark light image enhancement model comprises a double-branch illumination estimator module, a self-attention feature extraction module and a cross attention feature fusion module. The technical problem that the accuracy and reliability of target detection and recognition are affected due to poor image quality in a dark light traffic scene in the prior art is solved. The technical effects of realizing dark light image enhancement by fusing the visible light and the thermal imaging image, improving the image quality and further improving the accuracy and reliability of target detection and recognition are achieved.

Description

A Low-Light Image Enhancement Method for Traffic Scenes Technical Field

[0001] This invention relates to the field of data processing technology, and specifically to a method for enhancing low-light images based on traffic scenes. Background Technology

[0002] In recent years, high-position video technology has developed rapidly. By installing high-position video cameras at roadside locations and intersections to collect data, visual algorithms can be used to perform various tasks such as vehicle detection, license plate recognition, pedestrian detection, and parking line detection. This facilitates more refined and accurate management of roadside parking. However, in low-light scenarios such as nighttime or cloudy days, image quality deteriorates, and the accuracy of image-based visual algorithms decreases. Although installing supplementary lighting can improve visibility, it is not very effective at certain angles and in certain environments. Visual algorithms based on visible light images in low-light environments still need to be improved.

[0003] Existing technologies suffer from poor image quality in low-light traffic scenarios, which affects the accuracy and reliability of target detection and recognition. Summary of the Invention

[0004] The purpose of this application is to provide a low-light image enhancement method based on traffic scenes, which solves the technical problem that existing technologies have poor image quality in low-light traffic scenes, affecting the accuracy and reliability of target detection and recognition.

[0005] In view of the above problems, this application provides a low-light image enhancement method based on traffic scenes. The method includes: acquiring visible light images and thermal imaging images simultaneously acquired under the same traffic scene; inputting the visible light images and thermal imaging images into a low-light image enhancement model to obtain an enhanced image; wherein, the low-light image enhancement model includes: a dual-branch illumination estimator module, constructed based on Retinex theory, including a visible light image branch and a thermal imaging image branch; a self-attention feature extraction module, used to extract self-attention features from the illumination-corrected visible light image and thermal imaging image; and a cross-attention feature fusion module, used to fuse the self-attention features of the visible light and thermal imaging, perform fused feature reconstruction, and obtain the enhanced image.

[0006] Optionally, the visible light image branch and the thermal imaging image branch embed an illumination estimator, which is constructed based on the Transformer network structure.

[0007] Optionally, the processing procedure of the low-light image enhancement model includes: processing the visible light image and the thermal imaging image separately through a dual-branch illumination estimator module; wherein, the visible light image branch receives the visible light image, extracts a first illumination feature and generates an illumination map by performing illumination estimation; and performs pixel multiplication on the visible light image and the illumination map to obtain an illumination-corrected visible light image; wherein, the thermal imaging image branch receives the thermal imaging image and extracts a second illumination feature by performing illumination estimation.

[0008] Optionally, the self-attention feature extraction module performs self-attention encoding on the illumination-corrected visible light image and the first illumination feature to obtain visible light self-attention features; and performs self-attention encoding on the thermal imaging image and the second illumination feature to obtain thermal imaging self-attention features.

[0009] Optionally, the illumination-corrected visible light image is stitched together and linearly projected with the first illumination feature to obtain a query vector, a key vector, and a value vector; attention weights are obtained by calculating and normalizing the similarity between the query vector and all key vectors; and the value vectors are weighted and summed based on the attention weights to obtain the visible light self-attention feature.

[0010] Optionally, the visible light self-attention features and the thermal imaging self-attention features are fused across modes using a cross-attention feature fusion module to generate fused features; the fused features are then reconstructed into the enhanced image using a reconstruction network, wherein the reconstruction network employs a multilayer perceptron network.

[0011] Optionally, the cross-modal fusion adopts a multi-head cross-attention mechanism; wherein, the processing method based on the multi-head cross-attention mechanism includes: taking the self-attention feature of any modality among visible light self-attention features or thermal imaging self-attention features as a query vector, taking the self-attention feature of another modality as a key vector and a value vector, calculating the cross-attention in multiple attention heads respectively, and obtaining the fused feature by concatenating and linearly transforming the outputs of multiple attention heads.

[0012] Optionally, in the training of the low-light image enhancement model, the weighted sum of L1 loss and SSIM loss is used as the loss function to train the low-light image enhancement model until convergence.

[0013] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0014] The method provided in this application acquires visible light images and thermal imaging images simultaneously collected under the same traffic scene; the visible light images and thermal imaging images are input into a low-light image enhancement model to obtain an enhanced image; wherein, the low-light image enhancement model includes: a dual-branch illumination estimator module, constructed based on Retinex theory, including a visible light image branch and a thermal imaging image branch; a self-attention feature extraction module, used to extract self-attention features from the illumination-corrected visible light image and thermal imaging image; and a cross-attention feature fusion module, used to fuse the self-attention features of visible light and thermal imaging, perform fused feature reconstruction, and obtain the enhanced image. This achieves the technical effect of enhancing low-light images by fusing visible light and thermal imaging images, improving image quality, and thus improving the accuracy and reliability of target detection and recognition.

[0015] The above description is merely an overview of the technical solution of this application. To enable a clearer understanding of the technical means of this application and to facilitate its implementation according to the description, and to make the above and other objects, features, and advantages of this application more apparent, specific embodiments of this application are described below. It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent through the following description. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely exemplary. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0017] Figure 1 is a flowchart illustrating a low-light image enhancement method based on a traffic scene provided in this application.

[0018] Figure 2 is a flowchart illustrating a low-light image enhancement method based on a traffic scene provided in this application. Detailed Implementation

[0019] This application provides a low-light image enhancement method based on traffic scenes, addressing the technical problem of poor image quality in low-light traffic scenes, which affects the accuracy and reliability of target detection and recognition. It achieves the technical effect of enhancing low-light images by fusing visible light and thermal imaging images, thereby improving image quality and ultimately enhancing the accuracy and reliability of target detection and recognition.

[0020] The technical solutions of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. It should be understood that the present invention is not limited to the exemplary embodiments described herein. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention. It should also be noted that, for ease of description, only the parts related to the present invention are shown in the accompanying drawings, not all of them.

[0021] As shown in Figure 1, this application provides a low-light image enhancement method based on traffic scenes, which includes:

[0022] Acquire visible light images and thermal imaging images simultaneously in the same traffic scenario.

[0023] Specifically, by installing both visible light and thermal infrared cameras within a unified device, the two cameras can be placed horizontally or vertically. By adjusting the viewing angles of the two cameras, they can capture both visible light and thermal video images of the same traffic scene area at the same time. The visible light images can reveal rich visual information such as the color and texture of objects in the traffic scene. The thermal images, generated by detecting the infrared radiation emitted by objects, can reflect the temperature distribution of objects. Even in low-light environments with insufficient visible light, thermal images can still clearly show the outline and position of objects, unaffected by lighting conditions.

[0024] By simultaneously acquiring visible light and thermal images, which complement each other, comprehensive and reliable data input is provided for subsequent low-light image enhancement models, thereby improving the overall quality and interpretability of low-light images.

[0025] The visible light image and the thermal imaging image are input into the low-light image enhancement model to obtain the enhanced image.

[0026] The low-light image enhancement model includes: a dual-branch illumination estimator module, constructed based on Retinex theory, comprising a visible light image branch and a thermal imaging image branch; a self-attention feature extraction module, used to extract self-attention features from the illumination-corrected visible light image and thermal imaging image; and a cross-attention feature fusion module, used to fuse the self-attention features of visible light and thermal imaging, perform fused feature reconstruction, and obtain the enhanced image.

[0027] Specifically, the synchronously acquired visible light images and thermal imaging images are input into the low-light image enhancement model for image enhancement processing, and the enhanced image is output to improve the image quality in low-light environments, thereby improving the usability of images in traffic scenarios.

[0028] The low-light image enhancement model includes a two-branch illumination estimator module, a self-attention feature extraction module, and a cross-attention feature fusion module. The two-branch illumination estimator module is built based on Retinex theory. Retinex theory posits that an image (I) perceived by the human eye consists of two parts: a reflection component (R) and an illumination component (M). The reflection component (R) represents the essential attributes of an object, such as color and texture, and is independent of illumination. The illumination component (M) describes the lighting conditions of a traffic scene, specifically as follows: ,in, Represents an image. This indicates pixel-by-pixel multiplication.

[0029] The dual-branch illumination estimator module includes a visible light image branch and a thermal imaging image branch. The two branches work in parallel. The visible light image branch receives the visible light image and performs illumination estimation on the visible light image according to Retinex theory. The thermal imaging image branch receives the thermal imaging image and performs illumination estimation in the same way.

[0030] The self-attention feature extraction module employs a self-attention mechanism to extract self-attention features from illumination-corrected visible light images and thermal imaging images. This mechanism allows the low-light image enhancement model to focus on important parts of the input data. In this module, the illumination-corrected visible light image and the illumination features estimated by the dual-branch illumination estimator are concatenated and linearly projected to obtain a query vector, key vector, and value vector. Attention weights are obtained by calculating and normalizing the similarity between the query vector and all key vectors, reflecting the importance of information at different locations. The value vectors are then weighted and summed based on these attention weights to extract visible light self-attention features. These features highlight the most critical information in the visible light image for enhancing image quality. Similarly, the same operation is performed on the thermal imaging image and its corresponding illumination features to extract thermal imaging self-attention features.

[0031] The cross-attention feature fusion module is used to fuse the self-attention features of visible light and thermal imaging, perform fused feature reconstruction, and obtain the enhanced image. The cross-attention mechanism is a cross-modal information interaction method that allows features of visible light and thermal imaging images to mutually focus on and fuse. In the cross-attention feature fusion module, a multi-head cross-attention mechanism is adopted, using the self-attention feature of either the visible light or thermal imaging self-attention feature as a query vector, and the self-attention feature of the other modality as a key vector and value vector. Then, cross-attention is calculated in multiple attention heads, each focusing on different aspects of information. By concatenating and linearly transforming the outputs of multiple attention heads, deep fusion of visible light and thermal imaging self-attention features is achieved. Furthermore, a reconstruction network reconstructs the fused features into the final enhanced image, resulting in significant improvements in brightness, contrast, and sharpness, making it closer to a normal lighting image in a real scene.

[0032] By working together through multiple modules in the low-light image enhancement model, the visible light image and thermal imaging image information of two different modalities are effectively integrated and analyzed, and finally high-quality enhanced images are output. The enhanced images provide a clear and reliable image foundation for subsequent applications such as traffic monitoring, target detection, and behavior analysis, and greatly improve the visual perception capability and decision-making accuracy in low-light traffic scenarios.

[0033] Furthermore, the visible light image branch and the thermal imaging image branch are embedded with illumination estimators, which are constructed based on the Transformer network structure.

[0034] Specifically, the visible light image branch and the thermal imaging image branch embed illumination estimators. These illumination estimators are constructed using a transformer-based network structure, consisting of multiple transformer network layers (n layers, which can be set to 5). Each layer has a feature dimension of 512. Each transformer layer is divided into two sub-layers: the first sub-layer is a multi-head attention module, where the multi-head attention can be set to 8 heads; the second sub-layer is a standard feedforward neural network. Within each transformer layer, the output of each multi-head attention module is residually connected to the input of the current encoder layer. The output is then fed into a normalization layer, which employs a layer-level normalization layer. The normalized result is then fed into the feedforward neural network layer, where residual connection calculations and further layer-level normalization calculations are performed. The result is then fed into the next transformer layer for the process to repeat.

[0035] Based on the illumination estimator, illumination maps and illumination features are predicted for visible light images and thermal imaging images, specifically as follows: Where I represents a visible light image or a thermal image, M represents the lighting characteristics, and M represents the lighting diagram. This represents the illumination estimator.

[0036] In the visible light image branch, the illumination estimator is used to obtain an illumination map and illumination features. Then, the visible light image is multiplied by the illumination map to obtain a partially illuminated image, highlighting areas of uneven illumination in the image. In the thermal imaging image branch, the illumination estimator is used to obtain illumination features to provide structural information. An illumination map is not required because thermal imaging can capture scene structural information without visible light intensity.

[0037] In the dual-branch illumination estimator module, illumination estimators based on a Transformer network structure are embedded in the visible light image branch and the thermal imaging image branch. This enables more accurate extraction of illumination features and generation of illumination maps, providing a reliable basis for subsequent illumination correction of visible light images. It effectively eliminates uneven illumination interference, allowing the low-light image enhancement model to further extract effective features based on clearer and more uniformly illuminated traffic scene images. This improves the adaptability and enhancement effect of the entire low-light image enhancement model to images under different illumination conditions.

[0038] In one embodiment, as shown in Figure 2, the processing procedure of the low-light image enhancement model includes: processing the visible light image and the thermal imaging image respectively through a dual-branch illumination estimator module; wherein, the visible light image branch receives the visible light image, extracts a first illumination feature and generates an illumination map by performing illumination estimation; and performs pixel multiplication on the visible light image and the illumination map to obtain an illumination-corrected visible light image; wherein, the thermal imaging image branch receives the thermal imaging image and extracts a second illumination feature by performing illumination estimation.

[0039] Specifically, in the process of processing visible light images and thermal imaging images using a low-light image enhancement model, the dual-branch illumination estimator module is first activated. The visible light image branch of the dual-branch illumination estimator module receives the visible light image and uses the illumination estimator to estimate the illumination, analyze the illumination distribution in the image, and extract the first illumination feature. This first illumination feature reflects information such as the intensity and direction of light in the visible light image, and generates an illumination map. The illumination map visually presents the illumination intensity of each region of the visible light image. A pixel-wise multiplication operation is then performed between the visible light image and the illumination map to highlight areas of uneven illumination in the image. Illumination correction is then performed on the visible light image to eliminate the influence of uneven illumination, resulting in a more uniformly illuminated, illumination-corrected visible light image.

[0040] The thermal imaging image branch receives thermal imaging images and performs illumination estimation on the received thermal imaging images based on the embedded illumination estimator, extracting second illumination features to provide structural information. Since the thermal imaging images themselves are less affected by illumination, scene structural information can be captured without generating an illumination map.

[0041] By processing traffic scene images of two different modalities separately, the illumination features of each modality were accurately extracted, and the illumination of the visible light image was effectively corrected. This not only solved the problem of uneven illumination in visible light images under low light conditions, but also made full use of the advantage of thermal imaging images being unaffected by illumination. This provided high-quality input data for subsequent steps such as self-attention feature extraction and cross-attention feature fusion, thereby ensuring that the entire low-light image enhancement model can accurately enhance low-light images, improve the quality and usability of traffic scene images, and provide clear and reliable image support for applications such as traffic monitoring and target detection.

[0042] Furthermore, the processing of the low-light image enhancement model also includes: using a self-attention feature extraction module to perform self-attention encoding on the illumination-corrected visible light image and the first illumination feature to obtain visible light self-attention features; and performing self-attention encoding on the thermal imaging image and the second illumination feature to obtain thermal imaging self-attention features.

[0043] Specifically, a self-attention feature extraction module is used to perform self-attention encoding on the illumination-corrected visible light image and the first illumination feature to obtain visible light self-attention features. Simultaneously, self-attention encoding is performed on the thermal imaging image and the second illumination feature to obtain thermal imaging self-attention features.

[0044] Specifically, the self-attention module is used to extract features for self-attention encoding, which is represented as follows: ; ,in, Represents thermal imaging images. This represents the second illumination feature of a thermal imaging image. This represents the self-attention feature of thermal imaging. This represents the visible light image after illumination correction. The first illumination feature of a visible light image, This represents the self-attention characteristic of visible light.

[0045] The thermal imaging image and the visible light image are used as paired data. The thermal imaging image is a single-channel image, represented as follows: Visible light images are three-channel images, represented as H and W represent the height and width of the image, and the images in both modalities have the same height and width dimensions.

[0046] By automatically focusing on the most important features for image enhancement through the self-attention feature extraction module, redundant information is removed, and the representativeness and effectiveness of features are improved. This enables the low-light image enhancement model to better fuse feature information from two different modalities of images, thereby generating higher-quality and more detailed enhanced images. This significantly improves the overall quality and interpretability of images in low-light environments, which is beneficial for improving the accuracy of visual algorithms for tasks such as object detection, semantic segmentation, and person re-identification when applied based on fused images.

[0047] Furthermore, the self-attention encoding of the illumination-corrected visible light image and the first illumination feature includes: concatenating and linearly projecting the illumination-corrected visible light image and the first illumination feature to obtain a query vector, a key vector, and a value vector; calculating and normalizing the similarity between the query vector and all key vectors to obtain attention weights; and performing a weighted summation of the value vectors based on the attention weights to obtain the visible light self-attention feature.

[0048] Specifically, the illuminated visible light image after illumination correction is stitched together with the first illumination feature, and the stitched composite feature information is linearly projected. The linear projection is an operation that maps data from the original feature space to a new feature space through linear transformation. Through linear projection, the stitched composite feature information is mapped to three different feature subspaces, thereby obtaining a query vector, a key vector, and a value vector. The query vector is used to actively search for related information in the feature space, the key vector provides clues about the relationships between features, and the value vector contains the actual information content.

[0049] The similarity between the query vector and each key vector is calculated. Similarity calculation includes, but is not limited to, using similarity functions, dot product calculations, and perceptrons. For example, using dot product, a larger dot product indicates a higher similarity between the two vectors, and a stronger association of the characteristic information they represent. The obtained similarity values ​​are then transformed into a probability distribution using the softmax function to obtain attention weights. These attention weights reflect the importance of feature information at different locations to the current query; a larger weight indicates that the feature information at that location is more critical.

[0050] Based on the calculated attention weights, the corresponding value vectors are weighted and summed to obtain the visible light self-attention features. The same steps are then used to perform self-attention encoding on the thermal imaging image and the second illumination feature to obtain the thermal imaging self-attention features.

[0051] By using a self-attention module for analysis and processing, redundant information is effectively removed, the representativeness and effectiveness of image features are improved, and clearer and more accurate enhanced images are generated, thereby improving the quality and usability of the enhanced images.

[0052] Furthermore, the processing of the low-light image enhancement model also includes: performing cross-modal fusion of the visible light self-attention features and the thermal imaging self-attention features through a cross-attention feature fusion module to generate fused features; and reconstructing the fused features into the enhanced image through a reconstruction network, wherein the reconstruction network adopts a multilayer perceptron network.

[0053] Furthermore, the cross-modal fusion adopts a multi-head cross-attention mechanism; wherein, the processing method based on the multi-head cross-attention mechanism includes: taking the self-attention feature of any modality among visible light self-attention features or thermal imaging self-attention features as the query vector, taking the self-attention feature of the other modality as the key vector and value vector, calculating the cross-attention in multiple attention heads respectively, and obtaining the fused feature by concatenating and linearly transforming the outputs of multiple attention heads.

[0054] Specifically, the function of the cross-attention feature fusion module is to utilize thermal imaging features to optimize and guide the visible light feature space, thereby enhancing the quality of low-light images. Through the cross-attention feature fusion module, visible light self-attention features and thermal imaging self-attention features are fused across modally. This cross-modal fusion employs a multi-head cross-attention mechanism, specifically expressed as follows: =MCA( || ),in, These represent the self-attention features of thermal imaging and visible light, respectively. MCA() represents the multi-head cross-attention operation, which utilizes the features after cross-attention fusion.

[0055] Specifically, the multi-head cross-attention operation is set to 8 heads, and by computing 8 attention subspaces in parallel, different dependencies between sequences are captured, specifically expressed as: MultiHead(Q, K, V) = Concat( W, where MultiHead(Q, K, V) represents multi-head cross-attention, and Concat() represents concatenation along the channel dimension. This represents multiple attention heads, h=8, indicating an 8-head attention configuration, and W represents the learned weights; Q, K, and V represent the key vector, value vector, and query vector, respectively. The query vector is the self-attention feature of either visible light or thermal imaging self-attention features, while the key and value vectors are the self-attention features of the other modality. Cross-attention is calculated across multiple attention heads, and the outputs of these heads are concatenated and linearly transformed to achieve the fusion of two different modalities, resulting in a fused feature. For each... The calculation process is as follows: calculate the similarity between the query vector and each key vector, normalize the similarity value using the softmax function to obtain the attention weight, and then sum the attention weight and the corresponding value vector to obtain the final attention value.

[0056] A multilayer perceptron network is used as the reconstruction network to reconstruct an enhanced image from fused features. Specifically, the multilayer perceptron network consists of multiple fully connected layers, each containing a certain number of neurons. These neurons are connected to the layers before and after them via weights and biases. The input fused features first enter the first fully connected layer, which performs a linear transformation on the fused features. This transformation maps the fused features to a new feature space by multiplying the input features with the weight matrix and adding the bias vector. An activation function, such as the ReLU function, is then applied after each fully connected layer. Introducing nonlinearity through the activation function enhances the expressive power of the reconstruction network, enabling it to learn more complex feature relationships. The ReLU function sets negative values ​​to 0 while keeping positive values ​​unchanged, effectively mitigating the vanishing gradient problem and accelerating the training process of the reconstruction network.

[0057] Through alternating processing with multiple fully connected layers and activation functions, the fused features are continuously refined and adjusted within the reconstruction network. Finally, the output layer of the reconstruction network generates the pixel values ​​of the enhanced image based on the processing results. The number of neurons in the output layer matches the pixel dimension of the enhanced image, and a linear transformation maps the fused features to the pixel space to obtain the final enhanced image.

[0058] Cross-modal fusion fully leverages the complementarity of visible light and thermal imaging images, overcoming the limitations of single-modal approaches in low-light environments and enhancing the global representation capability of features. A reconstruction network using a multilayer perceptron transforms the fused features into an intuitive image, achieving a conversion from feature space to image space. This enables the low-light image enhancement model to output high-quality enhanced images, providing a reliable foundation for subsequent tasks such as traffic scene image analysis and object detection, significantly improving the usability and practicality of low-light images.

[0059] Furthermore, in the training of the low-light image enhancement model, the weighted sum of L1 loss and SSIM loss is used as the loss function to train the low-light image enhancement model until convergence.

[0060] Specifically, a loss function is constructed to train and iteratively optimize the parameters of the low-light image enhancement model. The training loss function of the low-light image enhancement model consists of a weighted sum of L1 loss and SSIM loss. L1 loss is the mean absolute error loss, and SSIM loss is constructed based on structural similarity theory to evaluate the similarity between images from three aspects: brightness, contrast, and structure. The closer the value is to 1, the more similar the enhanced image and the actual image are.

[0061] The specific form of the loss function is: ,in, , This represents the weighting parameter, which can be dynamically set according to actual conditions. and As a pixel fidelity loss, it can be expressed as: ,in, This represents two vectors that need to be calculated, namely the model's estimated value and the target value, respectively. m represents the total amount of data. The ultimate goal of this loss function is to minimize the absolute difference between the total estimated value and the target value. =1-SSIM(y, SSIM represents structural similarity of images, which has an open-source calculation formula, and will not be explained here.

[0062] During training, the low-light image enhancement model calculates the loss value based on the weighted loss function, calculates the gradient using the backpropagation algorithm, and updates the parameters of the low-light image enhancement model along the opposite direction of the gradient. Through continuous iterative training, it gradually adjusts its parameters, making the difference between the generated enhanced image and the real image smaller and smaller, that is, the loss value continuously decreases. When the loss value continues to decrease in multiple consecutive training iterations and the decrease is extremely small, basically stabilizing within a small range and no longer changing significantly, it indicates that the low-light image enhancement model has reached the convergence state. The trained low-light image enhancement model is then used to enhance low-light images of traffic scenes.

[0063] By employing a weighted sum of L1 and SSIM losses as the loss function, pixel-level accuracy and image structural similarity are considered, resulting in a more comprehensive and accurate measurement of the quality of the model-generated images. Through training to convergence, the model learns the mapping relationship from low-light images to high-quality enhanced images, thereby generating clearer, more realistic, and more detailed enhanced images in practical applications. This significantly improves the usability and practicality of low-light images, enhancing the accuracy of visual algorithms in tasks such as object detection, semantic segmentation, and person re-identification when applied to fused images.

[0064] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0065] Obviously, those skilled in the art can make several improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A low-light image enhancement method based on traffic scenes, characterized in that, The method includes: acquiring visible light images and thermal imaging images simultaneously collected under the same traffic scene; inputting the visible light images and thermal imaging images into a low-light image enhancement model to obtain an enhanced image; wherein, the low-light image enhancement model includes: a dual-branch illumination estimator module, constructed based on Retinex theory, including a visible light image branch and a thermal imaging image branch; a self-attention feature extraction module, used to extract self-attention features from the illumination-corrected visible light images and thermal imaging images; and a cross-attention feature fusion module, used to fuse the self-attention features of visible light and thermal imaging, perform fused feature reconstruction, and obtain the enhanced image.

2. The low-light image enhancement method based on traffic scenes as described in claim 1, characterized in that, The visible light image branch and the thermal imaging image branch have embedded illumination estimators, which are built based on the Transformer network structure.

3. The low-light image enhancement method based on traffic scenes as described in claim 1, characterized in that, The processing steps of the low-light image enhancement model include: processing the visible light image and the thermal imaging image separately through a dual-branch illumination estimator module; wherein, the visible light image branch receives the visible light image, extracts a first illumination feature and generates an illumination map by performing illumination estimation; and performs pixel multiplication on the visible light image and the illumination map to obtain an illumination-corrected visible light image; wherein, the thermal imaging image branch receives the thermal imaging image and extracts a second illumination feature by performing illumination estimation.

4. The low-light image enhancement method based on traffic scenes as described in claim 3, characterized in that, The self-attention feature extraction module performs self-attention encoding on the illumination-corrected visible light image and the first illumination feature to obtain visible light self-attention features; and performs self-attention encoding on the thermal imaging image and the second illumination feature to obtain thermal imaging self-attention features.

5. The low-light image enhancement method based on traffic scenes as described in claim 4, characterized in that, The self-attention encoding of the illumination-corrected visible light image and the first illumination feature includes: concatenating and linearly projecting the illumination-corrected visible light image and the first illumination feature to obtain a query vector, a key vector, and a value vector; calculating and normalizing the similarity between the query vector and all key vectors to obtain attention weights; and performing a weighted summation of the value vectors based on the attention weights to obtain the visible light self-attention feature.

6. The low-light image enhancement method based on traffic scenes as described in claim 5, characterized in that, The visible light self-attention features and the thermal imaging self-attention features are fused across modes using a cross-attention feature fusion module to generate fused features. The fused features are then reconstructed into the enhanced image using a reconstruction network, wherein the reconstruction network employs a multilayer perceptron network.

7. The low-light image enhancement method based on traffic scenes as described in claim 6, characterized in that, The cross-modal fusion adopts a multi-head cross-attention mechanism; wherein, the processing method based on the multi-head cross-attention mechanism includes: taking the self-attention feature of any modality among visible light self-attention features or thermal imaging self-attention features as the query vector, taking the self-attention feature of the other modality as the key vector and value vector, calculating the cross-attention in multiple attention heads respectively, and obtaining the fused feature by concatenating and linearly transforming the outputs of multiple attention heads.

8. The low-light image enhancement method based on traffic scenes as described in claim 1, characterized in that, In the training of the low-light image enhancement model, the weighted sum of L1 loss and SSIM loss is used as the loss function to train the low-light image enhancement model until convergence.

Images