Method for enhancing low-illumination image in motor vehicle based on multi-scale calibration network

Abstract

The invention discloses a method for enhancing a low-illumination image in a motor vehicle based on a multi-scale calibration network, which comprises an illumination enhancement network, a detail enhancement network, a color correction network and a gating fusion network, and is characterized in that the illumination enhancement network designs an illumination enhancement module of a cascade space attention enhancement mechanism through a pre-lightening module; a self-calibration module is introduced to enhance the overall exposure capability control of the image; designing a multi-level residual network structure in the detail enhancement network, and designing a color correction network to decouple a color image into a color histogram for the color distortion phenomenon of the image and in the enhancement process; the images output by the illumination enhancement network, the detail enhancement network and the color correction network are organically fused through the gating fusion network, the optimal fusion weight output by the three networks is learned in an end-to-end mode, and parallel weighted combination output is carried out. According to the method, good balance among brightness enhancement, detail recovery and accurate color correction is realized, and high-quality and reliable image input is provided for a hybrid electric vehicle visual system.

Description

Technical Field

[0001] This invention relates to the field of image enhancement methods, specifically a method for enhancing low-light images inside a power vehicle using a multi-scale calibration network. Background Technology

[0002] With the continuous advancement of hybrid vehicle technology and the constant improvement of intelligence and automation, accurate perception of the internal and external environment of the vehicle and timely inspection of in-vehicle equipment have become key aspects of ensuring the safe and stable operation of vehicles.

[0003] In-vehicle equipment inspection is of great significance. Due to the relatively enclosed space inside a vehicle, lighting conditions are extremely complex and unstable, primarily influenced by a combination of factors such as the layout of interior lighting and external light reflection. Images of in-vehicle equipment taken under such lighting conditions often suffer from blurriness and loss of detail. Subtle anomalies such as cracks, wear, and loose connections on the equipment surface are difficult to accurately identify. For example, if even tiny cracks on the battery pack's surface are not detected in time, they can pose serious safety hazards during vehicle operation, directly threatening the vehicle's safety and reliability. Therefore, enhancing image quality through improved lighting can help inspection systems quickly and accurately detect equipment anomalies, effectively improving the accuracy and efficiency of detection, thereby ensuring the normal operation of in-vehicle equipment.

[0004] External environmental perception is equally crucial. When hybrid vehicles operate in adverse weather conditions such as nighttime, heavy fog, or heavy rain, external images face numerous challenges, including insufficient lighting, color distortion, and low contrast. These issues significantly impair the vehicle's ability to perceive its surroundings, making it difficult to accurately identify road signs, obstacles, and other vehicles, thus increasing the risk of traffic accidents. For example, in low-light conditions at night, images captured by cameras may not clearly show the outline and position of pedestrians ahead, causing the driver assistance system to fail to react in time, seriously threatening driving safety. Therefore, enhancing the brightness and clarity of low-light external images and improving environmental perception are of paramount importance for ensuring vehicle safety and improving the performance of driver assistance systems.

[0005] Currently, numerous algorithms have emerged for low-light image enhancement, but all have certain limitations. Traditional image enhancement algorithms mainly focus on histogram equalization (HEM) and Retinex-based enhancement algorithms. Histogram equalization (HEM) algorithms change the overall brightness and contrast of an image by expanding the gray-level distribution range of the gray-level histogram. Tu Yihan et al. proposed a low-light image enhancement method based on multi-scale local histogram equalization to address the under-enhancement and over-enhancement problems in low-light image enhancement. After preprocessing the image, local histogram equalization is performed in multi-scale blocks, but the enhancement process does not consider color distortion and loss of detail information. Zhang Liya et al. proposed an image enhancement method based on an improved HSV-space fusion of the Retinex algorithm to address image halo and edge blurring phenomena during image enhancement. By improving the bilateral filtering algorithm and fusing it with the multi-scale Retinex algorithm, the brightness component can be effectively enhanced and the saturation component corrected. However, it is insufficient for enhancing local dark areas, and the Retinex algorithm is quite sensitive to changes in illumination during the enhancement process and the adjustment parameters are fixed, which limits its applicability to enhancing illumination in multiple scenarios inside and outside hybrid vehicles.

[0006] Another category is image enhancement methods based on deep learning. These methods train deep neural networks to learn features from a large number of low-light images to enhance image brightness and detail. Ge Junchao et al. proposed an algorithm combining adaptive Retinex and UNet. After decomposing the image into illumination and reflection maps, they used the UNet network to extract multi-scale features and designed a loss function to address issues such as low brightness and insufficient contrast in the acquired images. However, due to its simple convolution and pooling operations, the UNet network may experience problems such as blurred edges and information loss when extracting features from complex environments with multiple scenes and dark areas. For example, when processing images of equipment inside hybrid vehicles, it may not accurately capture the subtle textures and edge features of the equipment surface, affecting the detection accuracy of equipment anomalies.

[0007] In addition, Wang Manli et al. proposed an image enhancement algorithm based on deep neural networks to address the problems of insufficient contrast and poor texture details in low-light images. They designed an image decomposition network, an illumination adjustment network, and a reflection reconstruction network, respectively, and introduced depthwise separable convolution and MobileNet networks to improve the network's feature extraction and image detail recovery capabilities. Finally, based on Retinex theory, they fused the illumination and reflection components to enhance the illumination. However, they did not consider the color distortion inherent in low-light images and the possibility of over-enhancing certain colors during the enhancement process, leading to unnatural visual effects. For example, when enhancing low-light images outside a vehicle, the sky might become oversaturated, inconsistent with the actual scene, and affecting the accuracy of environmental perception.

[0008] More importantly, most existing low-light image enhancement algorithms are designed for single scenes or specific datasets. However, the internal and external environments of hybrid vehicles are complex and diverse, encompassing varying light intensities, angles, and weather conditions, while multi-scene datasets are scarce. Algorithms designed for single datasets often exhibit poor enhancement results when handling other vehicle scenes. For example, algorithms trained in laboratory environments may fail to adapt to the complex and changing lighting and environmental conditions encountered in real-world driving, leading to misjudgments or omissions during in-vehicle equipment inspections or external environment perception, thus failing to meet the actual image enhancement needs of hybrid vehicles.

[0009] In summary, existing low-light image enhancement algorithms have many limitations when applied to various scenarios inside and outside hybrid vehicles, making it difficult to effectively solve the image quality problems of hybrid vehicles in complex lighting environments. Therefore, this invention proposes a low-light image enhancement method for hybrid vehicles using a multi-scale calibration network. Summary of the Invention

[0010] The purpose of this invention is to overcome the defects and shortcomings of the existing technology and provide a method for enhancing low-light images inside electric vehicles using a multi-scale calibration network. This method solves the problems of low illumination, insufficient contrast, color distortion, and insufficient texture details in images acquired by video monitoring equipment in hybrid electric vehicles due to the harsh low-light environment.

[0011] To achieve the above objectives, the present invention provides the following technical solution:

[0012] A method for enhancing low-light images inside a power vehicle using a multi-scale calibration network, specifically including the following steps:

[0013] Step S1. Establish an illumination enhancement network, which specifically includes: inputting low-light images of internal devices and external environment, firstly compensating for illumination information through a pre-illumination module, then designing an illumination enhancement module based on Retinex theory and introducing a spatial enhancement attention module to enhance the feature capture capability of occluded dark areas and local dark areas inside the vehicle body, and finally controlling the overall exposure of the image through a self-calibration module to improve image brightness;

[0014] Step S2. Establish a detail enhancement network. The image after brightness enhancement in step S1 is processed using a multi-level residual network structure to enrich the image texture features and edge detail information.

[0015] Step S3. Establish a color correction network. Based on the encoding and decoding structure technology, the color distortion phenomenon in the image itself and the enhancement process in step S2 is considered. The color balance of the color histogram in the input color image is used, and a color loss function is designed to guide the network to achieve accurate color correction.

[0016] Step S4. Establish a gated fusion network. Adjust the network sharing parameters of steps S1-S3 through the gated unit to automatically adjust the weights of the output images of each sub-network, so as to achieve optimal fusion of brightness, detail and color of low-light images in multiple scenes and generate the final enhanced image.

[0017] Furthermore, the expression for the pre-lighting module in step S1 is as follows:

[0018] (I pre ,F pre )=E(I,L p (1)

[0019] L p =mean c (I) (2)

[0020] E represents the pre-lighting module, I represents the input low-light image, and mean c I represents the operation of calculating the average value of each pixel along the channel dimension. pre and F pre These are the pre-lit image and features output by the module, respectively, L p This is a mean feature map of a low-light image.

[0021] First, a 1×1 convolution with a kernel of 1 is used to fuse I and L. p The cascading of regions; considering that high-brightness areas in the image can provide contextual semantic information for underexposed areas, a 9×9 depthwise separable convolution is used to extract semantic information between regions, generating illumination features F. pre Finally, a 1×1 convolution with a kernel of 1 is used to focus the feature map, generating the illumination map I. pre The pre-lit output is obtained by multiplying the original image by the dot product.

[0022] Furthermore, in step S1, the illumination enhancement module in the low-light enhancement algorithm based on Retinex theory establishes a certain relationship between the low-light image and the desired sharp image, as shown in the formula.

[0023] S(x,y)=R(x,y)×I(x,y) (3)

[0024] S(x, y) is the original image, x and y are the pixel positions in the image; R is the desired clear image, and I represents the illumination component. The desired image can be recovered by using the illumination component in formula (3).

[0025] To accurately obtain the illumination components in the image, illumination estimation is performed using the LIME algorithm in a stepwise manner. The calculation formula is as follows:

[0026] A n+1 =H θ (An )+A n (4)

[0027] Where H θ It is a light enhancement network, A n It is the input low-light feature image, A n+1 This is the low-light feature image output in the next stage.

[0028] Furthermore, in step S1, a self-calibration module is designed to link the input of each stage with the input of the first stage, thereby presenting the input differences between each stage and the first stage. The difference between the output of each stage and the original input is integrated as input. A parameter learning network composed of multiple convolutional blocks is designed to learn and adjust the weights of the shared network, thereby reducing and calibrating the error between the output and the target stage by stage, improving the illumination enhancement effect and structural similarity of the enhanced image; the formula is as follows:

[0029]

[0030] x t Let K represent the input feature map at stage t, y be the input at stage 1, and K be the input at stage t. θ It is an introduced parameter learning network, with learnable parameters θ and v. t It is the output feature of stage t, and also the transformation input of stage t+1. G is the self-calibrating learning network, s t Let z be the characteristic parameters of stage t. t This is the input feature map after enhancement at stage t.

[0031] Furthermore, in step S2, the detail enhancement network is input from the lighting feature F output by the pre-lighting module in step S1. pre The network is constructed using a combination of Conv and GELU, and the use of small-sized 3×3 convolutional kernels can reduce the number of network parameters;

[0032] When using a standard ReLU as the activation function, as shown in Equation 6:

[0033] ReLU(x) = max(0, x) (6)

[0034] When the pixel values ​​of the low-light image to be enhanced are low, the weights of the neuron may become negative when convolved with the convolution kernel. The corresponding gradient of ReLU will all become 0, which will cause the corresponding weights not to be updated. However, when the smoother GELU is used as the activation function, as shown in Equation 7:

[0035] GELU(x)=x*P(X,,x)=x*Φ(x) (7)

[0036] Where Φ(x) refers to the cumulative function of the Gaussian normal distribution of x, which is further expressed by the following formula.

[0037]

[0038] The parameters in formulas (6-8) are mathematical formulas and have no explicit definition. Furthermore, in step S3, in the color correction network, an encoder-decoder structure is designed to enable the neural network to learn the color feature distribution, transforming the input raw image into a learnable color histogram, as shown in the following formula:

[0039] C pre =CNet(S low (9)

[0040] Where C pre It is an N×256 matrix, S low The input is a low-light image, where N=3 corresponds to the three color channels R, G, and B, and 256 is consistent with the range of pixel values; CNet represents the encoder-decoder structure of the color correction network;

[0041] In the color correction network, to improve the color distribution characteristics of the histogram, a color correction loss function is designed based on the difference in color histogram characteristics before and after enhancement, facilitating color restoration. The formula for the color correction loss function is as follows:

[0042]

[0043] C high It is the histogram feature map of the enhanced image, C pre It is the histogram feature map of the input image, L c To compensate for the color correction loss, the color correction network learns to decouple the differences in color histogram features before and after the image through the encoding and decoding structure. Then, by using the designed correction loss, it constrains the histogram features of the input low-light image to move closer to the enhanced-light image, thereby ensuring that the image color restoration and enhancement are not distorted and the visual effect is more outstanding.

[0044] Furthermore, in step S4, the gating fusion network consists of a visual encoder and three gating units;

[0045] The visual encoder consists of a VGG16 algorithm and a nonlinear function. The VGG16 algorithm, through the stacking of multiple convolutional layers, can better capture the local details of the input image. Each gated unit receives an enhanced image from the output of a different network, as shown in the following formula:

[0046]

[0047] Where x i These are three network output features, where z represents the fused enhanced image, fi (x i ) represents the scalar output encoded by the visual decoder, w i This represents the weight learned by each gate, and N represents the normalization operation;

[0048] The outputs of the aforementioned illumination enhancement network, detail enhancement network, and illumination correction network are optimally adjusted using the backpropagation algorithm to achieve optimal fusion of brightness, detail, and color in low-light images across multiple scenes, ensuring image enhancement effects.

[0049] Compared with the prior art, the beneficial effects of the present invention are:

[0050] This invention consists of an illumination enhancement network, a detail enhancement network, a color correction network, and a gated fusion network. The illumination enhancement network uses a pre-illumination module and a cascaded spatial enhancement attention mechanism to significantly improve the brightness of low-light images inside and outside the vehicle, making details in dimly lit areas clearly visible. The detail enhancement network enriches texture features with a multi-level residual structure to avoid loss of detail. The color correction network accurately restores true colors by using an encoding and decoding structure and a color loss function, solving the problem of color distortion.

[0051] It has strong adaptability to multiple scenarios. It is applicable to various low-light image enhancement in the internal and external perception environments of hybrid vehicles, and can effectively handle different road scenarios and diverse weather conditions. It also has strong generalization performance, does not depend on specific scenarios or datasets, and can work stably in unknown low-light scenarios, overcoming the shortcomings of existing algorithms in terms of scenario limitations.

[0052] It can quickly complete image enhancement processing, meet the real-time image processing needs of vehicles, provide clear and accurate image information for vehicle external environment perception and in-vehicle equipment inspection in a timely manner, support the operation of intelligent systems in real time, and ensure the safe operation of vehicles and equipment maintenance.

[0053] This invention successfully achieves a good balance between brightness enhancement, detail restoration, and accurate color correction, which can well meet the needs of in-vehicle low-light image enhancement and the perception needs of low-light environments around the vehicle. When improving image brightness, it does not excessively sacrifice detail and color information; when enriching image details, it ensures uniform brightness enhancement and accurate color reproduction; when correcting color distortion, it takes into account both brightness and detail performance. This comprehensive balance performance comprehensively improves the quality of the enhanced image, provides high-quality and reliable image input for the vision system of hybrid vehicles, and effectively solves common problems such as insufficient brightness, low contrast, color distortion, and loss of detail information in low-light images. Attached Figure Description

[0054] Figure 1 This is a network structure diagram of the present invention;

[0055] Figure 2This is a structural diagram of the pre-lighting module in an embodiment of the present invention;

[0056] Figure 3 This is a structural diagram of the light enhancement module in an embodiment of the present invention;

[0057] Figure 4 This is a flowchart illustrating the overall self-calibration process in an embodiment of the present invention.

[0058] Figure 5 This is a structural diagram of the parameter learning network in an embodiment of the present invention;

[0059] Figure 6 This is a structural diagram of the detail enhancement network in an embodiment of the present invention;

[0060] Figure 7 This is a comparison diagram of activation functions in embodiments of the present invention;

[0061] Figure 8 This is a comparison chart of color histograms in an embodiment of the present invention;

[0062] Figure 9 This is a diagram of the color correction network structure.

[0063] Figure 10 This is a diagram of a gating fusion network. Detailed Implementation

[0064] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0065] A method for enhancing low-light images inside a power vehicle using a multi-scale calibration network, specifically including the following steps:

[0066] Step S1. Establish an illumination enhancement network, which specifically includes: inputting low-light images of the internal equipment of the vehicle and the external environment, firstly compensating for illumination information through a pre-illumination module, then designing an illumination enhancement module based on Retinex theory and introducing a spatial enhancement attention module to strengthen the feature capture capability of the dark areas occluded by the internal equipment and the local dark areas, and finally controlling the overall exposure of the image through a self-calibration module to improve the image brightness;

[0067] Step S2. Establish a detail enhancement network. The image after brightness enhancement in step S1 is processed using a multi-level residual network structure to enrich the image texture features and edge detail information.

[0068] Step S3. Establish a color correction network. Based on the encoding and decoding structure technology, the color distortion phenomenon in the image itself and the enhancement process in step S2 is considered. The color balance of the color histogram in the input color image is used, and a color loss function is designed to guide the network to achieve accurate color correction.

[0069] Step S4. Establish a gated fusion network. By adjusting the network-shared parameters from steps S1-S3 through the gated unit, the weights of the output images from each sub-network are automatically adjusted to achieve optimal fusion of brightness, detail, and color in low-light images from multiple scenes, generating the final enhanced image. The network structure diagram is shown below. Figure 1 As shown.

[0070] In step S1:

[0071] Due to lighting conditions limiting the interior of the vehicle, the dynamic range of images is typically narrow. Furthermore, the difference between bright and dark areas in low-light images is significant, and most areas may appear too dark or blurry. Directly enhancing these images would inevitably amplify noise and artifacts. Therefore, the expression for the pre-brightening module is as follows:

[0072] (I pre F pre )=E(I,L p (1)

[0073] L p =mean c (I) (2)

[0074] E represents the pre-lighting module, I represents the input low-light image, and mean c I represents the operation of calculating the average value of each pixel along the channel dimension. pre and F pre These are the pre-lit image and features output by the module, respectively, L p This is a mean feature map of a low-light image.

[0075] The structural diagram of the pre-lighting module is as follows: Figure 2 First, a 1×1 convolution with a kernel of 1 is used to fuse I and L. p The cascading of regions; considering that high-brightness areas in the image can provide contextual semantic information for underexposed areas, a 9×9 depthwise separable convolution is used to extract semantic information between regions, generating illumination features F. pre Finally, a 1×1 convolution with a kernel of 1 is used to focus the feature map, generating the illumination map I. pre The pre-brightening output is obtained by multiplying the pre-brightening output with the original image. The pre-brightening module learns the semantic information of the brightness distribution between regions, initially compensates for the illumination in weak light areas, preserves details, and controls noise, providing a better input for subsequent enhancement.

[0076] In the low-light enhancement algorithm based on Retinex theory, the illumination enhancement module establishes a certain relationship between the low-light image and the desired sharp image, as shown in the formula:

[0077] S(x,y)=R(x,y)×I(x,y)(3)

[0078] S(x,y) is the original image, where x and y are the pixel positions in the image; R is the desired clear image, and I represents the illumination component. The desired image can be recovered using the illumination component in formula (3).

[0079] To accurately obtain the illumination components in the image, illumination estimation is performed using the LIME algorithm in a stepwise manner. The calculation formula is as follows:

[0080] A n+1 =H θ (A n )+A n (4)

[0081] Where H θ It is a light enhancement network, A n It is the input low-light feature image, A n+1 This is the low-light feature image output in the next stage. The enhancement module structure design is as follows: Figure 3 As shown, this paper studies the Retinex-Net image enhancement algorithm based on adaptive illumination estimation. Each stage consists of a 3×3 convolutional layer with ReLU activation, a 3×3 convolutional layer with Sigmoid activation, a 3×3 convolutional layer with BatchNorm2d and ReLU activation, and a SEAM attention mechanism.

[0082] The SEAM attention mechanism was introduced to address the issue of excessive cover within the vehicle's interior environment, resulting in partial light source occlusion and large dark areas. Even after image enhancement, artifacts and uneven brightness persist, leading to unnatural images. The SEAM module uses a combination of depthwise separable convolutions and residual connections to achieve semantic exchange between occluded dark areas and normally exposed areas. However, while depthwise separable convolutions operate on a channel-by-channel basis, learning the importance of different channels and reducing the number of parameters, they neglect the information relationships between channels. To compensate for this, the outputs of convolutions at different depths are combined through point-to-point (1x1) convolutions, and then two fully connected layers are used to fuse the information from each channel, enhancing the connections between all channels. This model compensates for brightness loss under occlusion by learning the relationship between occluded and unoccluded surfaces.

[0083] By designing a self-calibration module, the input of each stage is linked to the input of the first stage to present the input differences between each stage and the first stage. The difference between the output of each stage and the original input is integrated as input. A parameter learning network composed of multiple convolutional blocks is designed to learn and adjust the shared network weights, thereby reducing and calibrating the error between the output and the target stage by stage, improving the illumination enhancement effect and structural similarity of the enhanced image; the formula is as follows:

[0084]

[0085] x t Let K represent the input feature map at stage t, y be the input at stage 1, and K be the input at stage t. θ It is an introduced parameter learning network, with learnable parameters θ and v. t It is the output feature of stage t, and also the transformation input of stage t+1. G is the self-calibrating learning network, s t Let z be the characteristic parameters of stage t. t Let t be the input feature map after enhancement at stage t. The self-calibration module integrates the difference between the output of each stage and the original input as input. By designing a parameter learning network composed of multiple convolutional blocks, it learns and adjusts the shared network weights to reduce and calibrate the error between the output and the target stage by stage, thereby improving the illumination enhancement effect and structural similarity of the enhanced image and accelerating the convergence between each stage, thus reducing inference cost.

[0086] Due to insufficient external light and numerous obstructions to the internal lighting in hybrid vehicles, images acquired by the visual processor often lose significant detail due to insufficient light, resulting in blurred image texture and edge features, and a decline in visual quality. The detail enhancement network is designed with a multi-level residual network to collect edge and texture information from the pre-illuminated feature map. In step S2, the detail enhancement network is input from the illuminated feature F output from the pre-illumination module in step S1. pre By incorporating residual layer design into the linear convolution structure, the network can retain shallow information even after the number of convolutional layers reaches a certain level. Shallow information mainly consists of image details, thereby reducing the loss of details and texture information during transmission and promoting the flow of information between deep and shallow layers. The network is constructed using a combination of convolutional Conv and Gaussian error linear units (GELUs). Using small-sized 3×3 convolutional kernels can reduce the number of network parameters.

[0087] When using a standard ReLU as the activation function, as shown in Equation 6:

[0088] ReLU(x) = max(0, x) (6)

[0089] When the pixel values ​​of the low-light image to be enhanced are low, the weights of the neuron may become negative when convolved with the convolution kernel. The corresponding gradient of ReLU will all become 0, which will cause the corresponding weights not to be updated. However, when the smoother GELU is used as the activation function, as shown in Equation 7:

[0090] GELU(x)=x*P(X,,x)=x*Φ(x) (7)

[0091] Where Φ(x) refers to the cumulative function of the Gaussian normal distribution of x, which is further expressed by the following formula.

[0092]

[0093] The parameters in formula (6-7) are not explicitly defined.

[0094] Comparison chart of the two is as follows Figure 7 As shown, the GELU activation function is smoother than the ordinary ReLU (e.g., Figure 7 (As shown in a and 7b). When x is greater than 0, the GELU function is similar to ReLU; when the input is much less than 0, the output of GELU is 0; when the input x is less than 0 but close to 0, GELU has a non-linear output with a certain degree of continuity. This ensures that subtle details can be well preserved.

[0095] Insufficient ambient light inside the vehicle and at night prevents visual devices from accurately measuring the intensity of the three colors of light, thus hindering accurate color reconstruction of the image. Furthermore, most current algorithms do not focus on enhancing the color consistency and naturalness of the image.

[15] Therefore, two common problems often occur: color distortion in the original acquired image and unnatural colors in the enhanced image. To address these issues, this paper designs a color correction network that learns the color distribution based on the color histogram of low-light images and adjusts the R, G, and B channel pixels to maintain the color accuracy of the enhanced output image.

[0096] A color histogram primarily describes the proportion of different colors within the entire image, without concern for the spatial location of the colors. For example... Figure 8 As shown ( Figure 8 a color histogram of a low-light image. Figure 8 (b) Color histogram of the image after enhancement by our algorithm. The figure shows a significant difference between the color histograms of the low-light image and the image after enhancement. In the low-light image, the RGB channels' display pixels are mainly concentrated in the lower brightness range, with smaller overall fluctuations and a smoother surface. Peaks are more concentrated, and the color channels are more tightly distributed. In contrast, the normal-light image has a more uniform pixel distribution, covering a wider brightness range, and different brightness areas show obvious differences and distribution.

[0097] In step S3, within the color correction network, an encoder-decoder structure is designed to enable the neural network to learn the color feature distribution. This structure transforms the original input image into a learnable color histogram, as shown in the following formula:

[0098] C pre =CNet(S low (9)

[0099] Where C pre It is an N×256 matrix, S low The input is a low-light image, where N=3 corresponds to the three color channels R, G, and B, and 256 is consistent with the range of pixel values; CNet represents the encoder-decoder structure of the color correction network;

[0100] In the color correction network, to improve the color distribution characteristics of the histogram, a color correction loss function is designed based on the difference in color histogram characteristics before and after enhancement, facilitating color restoration. The formula for the color correction loss function is as follows:

[0101]

[0102] C high It is the histogram feature map of the enhanced image, C pre It is the histogram feature map of the input image, L c To compensate for the color correction loss, the color correction network learns to decouple the differences in color histogram features before and after the image through the encoding and decoding structure. Then, by using the designed correction loss, it constrains the histogram features of the input low-light image to move closer to the enhanced-light image, thereby ensuring that the image color restoration and enhancement are not distorted and the visual effect is more outstanding.

[0103] Nighttime vehicle exterior scenes vary, with uneven light intensity. For example, parking lots often have numerous obstructions and interference factors such as dust, fog, and black light-absorbing substances. This means that current low-light enhancement algorithms are only applicable to a single scene and lack universality when switching to different vehicle exterior scenes, or the enhancement effect does not meet the actual generation requirements. In step S4, the gating fusion network consists of one visual encoder and three gating units; its structure is as follows... Figure 10 As shown.

[0104] The visual encoder consists of a VGG16 algorithm and a nonlinear function. The VGG16 algorithm, through the stacking of multiple convolutional layers, can better capture the local details of the input image. Each gated unit receives an enhanced image from the output of a different network, as shown in the following formula:

[0105]

[0106] Where x iThese are three network output features, where z represents the fused enhanced image, f i (x i ) represents the scalar output encoded by the visual decoder, w i This represents the weight learned by each gate, and N represents the normalization operation;

[0107] The outputs of the aforementioned illumination enhancement network, detail enhancement network, and illumination correction network are optimally adjusted using the backpropagation algorithm to achieve optimal fusion of brightness, detail, and color in low-light images across multiple scenes, ensuring image enhancement effects.

[0108] The loss function used in the multi-scale calibration gating network consists of fidelity loss, smoothing loss, and color correction loss. The total loss function is shown in Equation 8.

[0109] L total =αL f +βL s +γL c (12)

[0110]

[0111] L f L s L c These represent fidelity loss, smoothing loss, and color correction loss, respectively. α, β, and γ are the weighting coefficients for the corresponding losses, set to 1.5, 1, and 10 in this paper. Fidelity loss is used to ensure pixel-level consistency between the illumination estimate and the input of a single stage. In the formula, T is the total number of stages, and x... t The output illumination is s at stage t. t-1 The illumination estimated for stage t-1 is used to continuously constrain the output of each stage using fidelity loss. N is the total number of pixels. (i) It represents the pixel value within a fixed-size area. By continuously accumulating the differences between pixels in adjacent areas, it constrains the information transition between pixels, thereby making the overall image visual effect natural.

[0112] The above method has been verified in actual use as follows:

[0113] The experimental results were quantitatively analyzed using evaluation indicators. The algorithm used in this invention was compared with other algorithms, and the experimental results are shown in Tables 1-4.

[0114] Table 1. Low-light dataset of interior equipment in hybrid vehicles.

[0115]

[0116] in:

[0117] MSRCR: Multiscale Retina Enhancement Algorithm with Color Restoration

[0118] EnlightenGAN: Lighting up a generative adversarial network

[0119] RRDNet: Robust Retinal Decomposition Network

[0120] RetinexNet: Retinex Network

[0121] SCI: Self-calibrating illumination network

[0122] URetinexNet: URetinex Network

[0123] PairL1E: Paired Low-Light Enhancement Network

[0124] index:

[0125] PNSR: Peak Signal-to-Noise Ratio

[0126] SSIM: Structural Similarity

[0127] IE: Information Entropy

[0128] LOE: Luminance Step Difference

[0129] NIQE: Natural Image Quality Evaluator

[0130] According to Table 1, our algorithm leads other algorithms in PSNR, SSIM, IE, and LOE metrics, which are 15.84, 0.57, 7.55, and 2814, respectively. NIQE is second only to SCI and RetinexNet.

[0131] Data shows that the algorithm presented in this paper enhances low-light images with low distortion and high clarity in this scenario, while preserving details well and exhibiting smooth and natural brightness changes. Although the SCI algorithm ranks first in NIQE, it suffers from significant loss of image detail, resulting in blurred edges, textures, and shapes, leading to an overall unclear image.

[0132] Table 2 Dataset of Hybrid Vehicles in Low-Light Environment

[0133]

[0134] In Table 2, our algorithm ranks first in four metrics—PSNR, SSIM, IE, and LOE—compared to other algorithms, with scores of 13.43, 0.68, 7.83, and 2219, respectively. Its NIQE score of 5.11 ranks second only to EnlightenGAN.

[0135] Data shows that after the algorithm in this paper enhances the image in this conveyor belt scenario, the image exhibits high definition, well-preserved structure, natural brightness order, and characteristics consistent with natural statistical properties, resulting in a natural and high-quality overall visual effect. The image enhanced by the EnlightenGAN algorithm has relatively high PSNR and SSIM, but the enhancement process disrupts the original brightness change order of the image, leading to uneven brightness distribution and distortion in some dark areas.

[0136] To evaluate the efficiency of the eight image enhancement algorithms, each algorithm was applied to 20 low-light test images of 256×256 pixels, and the average time taken to enhance a single low-light image was calculated. Detailed time consumption data are shown in Table 3 for comparative analysis; Table 3:

[0137]

[0138] As shown in the table, our proposed algorithm ranks second among the eight algorithms, requiring only 0.291 seconds to infer a single image. Experiments demonstrate that our algorithm performs excellently in time-consuming tests, fully verifying that it can not only rapidly process images in both vehicle interior and exterior environments, but also effectively improve the brightness and overall quality of low-light images.

[0139] In summary, this paper proposes an MCGN-based low-light image enhancement algorithm for video surveillance images of vehicle interior equipment and complex external environments, which aims to address issues such as insufficient brightness, low contrast, color distortion, and loss of detail information.

[0140] The algorithm consists of four networks: illumination enhancement, detail enhancement, color correction, and gated fusion. The illumination enhancement network improves image brightness through a pre-illumination module and a cascaded spatial enhancement attention mechanism. The detail enhancement network enriches image texture and edge details using a multi-level residual network structure. The color correction network restores the true colors of the image using an encoding / decoding structure and a color loss function. The gated fusion network achieves optimal fusion of the outputs of the three networks through a novel gating mechanism, ensuring effective image enhancement. Experimental results show that the algorithm performs exceptionally well in significantly improving the quality of nighttime vehicle exterior images. It successfully enhances brightness while also achieving detail restoration and accurate color correction, achieving a good balance among the three aspects. It demonstrates strong multi-scene adaptability and broad generalization performance, exhibiting excellent results in various low-light scenarios.

[0141] Although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0142] Therefore, the above description is only a preferred embodiment of this application and is not intended to limit the scope of this application; that is, all equivalent modifications made in accordance with the scope of the claims of this application shall be within the protection scope of the claims of this application.

Claims

1. A method for enhancing low-light images inside a power vehicle using a multi-scale calibration network, characterized in that, Specifically, the steps include the following: Step S1. Establish an illumination enhancement network, which specifically includes: inputting low-light images of the internal equipment of the hybrid vehicle and the external environment, firstly compensating for illumination information through a pre-illumination module, addressing the brightness and detail loss in local areas of the pre-illumination image, then designing an illumination enhancement module based on Retinex theory and introducing a spatial enhancement attention module to strengthen the feature capture capability of dark areas occluded by the internal equipment of the hybrid vehicle and local dark areas of the external environment, and finally controlling the overall exposure of the image through a self-calibration module to improve the image brightness; Step S2. Establish a detail enhancement network. The image after brightness enhancement in step S1 is processed using a multi-level residual network structure to enrich the image texture features and edge detail information. Step S3. Establish a color correction network. Based on the encoding and decoding structure technology, the color distortion phenomenon in the image itself and the enhancement process in step S2 is considered. The color balance of the color histogram in the input color image is used, and a color loss function is designed to guide the network to achieve accurate color correction. Step S4. Establish a gated fusion network. Adjust the network sharing parameters of steps S1-S3 through the gated unit to automatically adjust the weights of the output images of each sub-network, so as to achieve optimal fusion of brightness, detail and color of low-light images in multiple scenes and generate the final enhanced image.

2. The method for enhancing low-light images of hybrid vehicles in multiple scenarios based on a multi-scale calibration gating network according to claim 1, characterized in that, The expression for the pre-lighting module in step S1 is as follows: (I pre ,F pre )=E(I,L p ) (1) L p =mean c (I) (2) E represents the pre-lighting module, I represents the input low-light image, and mean c I represents the operation of calculating the average value of each pixel along the channel dimension. pre and F pre These are the pre-lit image and features output by the module, respectively, L p This is a mean feature map of a low-light image. First, a 1×1 convolution with a kernel of 1 is used to fuse I and L. p Cascade; Considering that high-brightness areas in an image can provide contextual semantic information for underexposed areas, a 9×9 depthwise separable convolution is used to extract semantic information between regions and generate illumination features F. pre Finally, a 1×1 convolution with a kernel of 1 is used to focus the feature map, generating the illumination map I. pre The pre-lit output is obtained by multiplying the original image by the dot product.

3. The method for enhancing low-light images inside a power vehicle using a multi-scale calibration network according to claim 1, characterized in that, In step S1, the illumination enhancement module, based on the Retinex theory-based low-light enhancement algorithm, establishes a relationship between the low-light image and the desired sharp image, as shown in the formula: S(x,y)=R(x,y)×I(x,y) (3) S(x,y) is the original image, where x and y are the pixel positions in the image; R is the desired clear image, and I represents the illumination component. The desired image can be recovered using the illumination component in formula (3). To accurately obtain the illumination components in the image, illumination estimation is performed using the LIME algorithm in a stepwise manner. The calculation formula is as follows: A n+1 =H θ (A n )+A n (4) Where H θ It is a light enhancement network, A n It is the input low-light feature image, A n+1 This is the low-light feature image output in the next stage.

4. The method for enhancing low-light images inside a power vehicle using a multi-scale calibration network according to claim 1, characterized in that, In step S1, a self-calibration module is designed to link the input of each stage with the input of the first stage, thus presenting the input differences between each stage and the first stage. The difference between the output of each stage and the original input is integrated as input. A parameter learning network composed of multiple convolutional blocks is designed to learn and adjust the shared network weights, progressively reducing and calibrating the error between the output and the target, thereby improving the illumination enhancement effect and structural similarity of the enhanced image. The formula is as follows: x t Let K represent the input feature map at stage t, y be the input at stage 1, and K be the input at stage t. θ It is an introduced parameter learning network, with learnable parameters θ and v. t It is the output feature of stage t, and also the transformation input of stage t+1. G is the self-calibrating learning network, s t Let z be the characteristic parameters of stage t. t This is the input feature map after enhancement in stage t.

5. The method for enhancing low-light images inside a power vehicle using a multi-scale calibration network according to claim 1, characterized in that, In step S2, the detail enhancement network is input from the lighting feature F output by the pre-lighting module in step S1. pre The network is constructed using a combination of Conv and GELU, and the use of small-sized 3×3 convolutional kernels can reduce the number of network parameters; When using a standard ReLU as the activation function, as shown in Equation 6: ReLU(x) = max(0, x) (6) When the pixel values ​​of the low-light image to be enhanced are low, the weights of the neuron may become negative when convolved with the convolution kernel, and the corresponding gradient of GELU will all become 0. This will cause the corresponding weights not to be updated. However, when the smoother GELU is used as the activation function, as shown in Equation 7: GEIU(x)=x*P(X,,x)=x*Φ(x) (7) Where Φ(x) refers to the cumulative function of the Gaussian normal distribution of x, which is further expressed by the following formula. In the formula: P represents the probability that X is less than or equal to x; X is a random variable; dX represents the integral of the variable X; x represents the value of the random variable; Φ refers to the cumulative function of the Gaussian normal distribution; μ represents the mean; σ represents the standard deviation; e is the natural constant, with the meaning of 2.71828.

6. The method for enhancing low-light images inside a power vehicle using a multi-scale calibration network according to claim 1, characterized in that, In step S3, within the color correction network, an encoder-decoder structure is designed to enable the neural network to learn the color feature distribution. This structure transforms the original input image into a learnable color histogram, as shown in the following formula: C pre =CNet(S low ) (9) Where C pre It is an N×256 matrix, S low The input is a low-light image, where N=3 corresponds to the three color channels R, G, and B, and 256 is consistent with the range of pixel values; CNet represents the encoder-decoder structure of the color correction network; In the color correction network, to improve the color distribution characteristics of the histogram, a color correction loss function is designed based on the difference in color histogram characteristics before and after enhancement, facilitating color restoration. The formula for the color correction loss function is as follows: C high It is the histogram feature map of the enhanced image, C pre It is the histogram feature map of the input image, L c To compensate for the color correction loss, the color correction network learns to decouple the differences in color histogram features before and after the image through the encoding and decoding structure. Then, by using the designed correction loss, it constrains the histogram features of the input low-light image to move closer to the enhanced-light image, thereby ensuring that the image color restoration and enhancement are not distorted and the visual effect is more outstanding.

7. The method for enhancing low-light images inside a power vehicle using a multi-scale calibration network according to claim 1, characterized in that, In step S4, the gating fusion network consists of one visual encoder and three gating units; The visual encoder consists of a VGG16 algorithm and a nonlinear function. The VGG16 algorithm, through the stacking of multiple convolutional layers, can better capture the local details of the input image. Each gated unit receives an enhanced image from the output of a different network, as shown in the following formula: Where x i These are three network output features, where z represents the fused enhanced image, f i (x i ) represents the scalar output encoded by the visual decoder, w i This represents the weight learned by each gate, and N represents the normalization operation; The outputs of the aforementioned illumination enhancement network, detail enhancement network, and illumination correction network are optimally adjusted using the backpropagation algorithm to achieve optimal fusion of brightness, detail, and color in low-light images across multiple scenes, ensuring image enhancement effects.

Images