An image enhancement method for night-time fatigue driving detection

Abstract

The application discloses an image enhancement method for night fatigue driving detection, comprising the following steps: decomposing a night image under a visible light source and a corresponding normal light reference image into a reflection component and an illumination component by using a decomposition network, to obtain a night image reflection component, a night image illumination component, a reference image reflection component and a reference image illumination component; enhancing the reflection component by using the night image reflection component and the reference image reflection component through an enhancement network; enhancing the illumination component by using the night image illumination component and the reference image illumination component through the enhancement network; fusing the enhanced reflection component and the illumination component to obtain an enhanced image under the visible light source; fusing an infrared light source image and the enhanced image under the visible light source to obtain a final image, which is used for judging a driver fatigue state. The method effectively enhances the night image of the driver, makes the image clearer and brighter, and is beneficial to improving the accuracy of the fatigue detection system.

Description

Technical Field

[0001] The present invention relates to an image enhancement method, in particular to an image enhancement method for nighttime fatigue driving detection, and belongs to the technical field of image processing. Background Art

[0002] With the rapid development of the global economy, motor vehicles are becoming increasingly prevalent in our lives. Convenient transportation has led to an increase in traffic accidents. According to statistics, fatigue driving is one of the main causes of traffic accidents, accounting for approximately 10% to 20% of all accidents. Nighttime driving is particularly prone to neurological and visual fatigue, leading to slow reactions and inattention, which in turn increases the risk of accidents. Traditional fatigue detection methods are prone to generating excessive noise when light intensity is insufficient. This results in poor image quality, unclear feature extraction, reduced recognition accuracy, and delayed warnings, posing a significant safety hazard to nighttime driving.

[0003] Therefore, low-light image processing is necessary to improve image quality. Prior art methods, such as those disclosed in publication number CN117455822A, use deep Retinex theory to enhance low-light images. These methods include: decomposing low-light images and reference images in a dataset into corresponding reflectance maps and illumination maps; establishing an optimization objective function, introducing constraints on the structure and texture of the two sub-images, and iteratively solving the optimal solution to the optimization objective function using an alternating direction minimization method; and employing a deep neural network prior estimation model to guide the iterative optimization process. The illumination map and reflectance map, obtained through multiple iterations, are then multiplied to produce the final enhanced image. Traditional low-light image enhancement methods include histogram equalization, gamma transform, dark channel prior, denoising, and multi-scale Retinex. Histogram equalization and gamma correction can enhance image contrast and brightness, but they can also lead to over-enhancement. Dark channel verification and multi-scale Retinex are computationally complex, hindering the speed of real-time fatigue detection. Denoising inevitably results in detail loss and uneven processing. Summary of the Invention

[0004] The purpose of the present invention is to provide an image enhancement method for nighttime fatigue driving detection in order to solve at least one of the above-mentioned technical problems, which can effectively enhance the driver's nighttime image, make the image clearer and brighter, and facilitate the fatigue detection system to make real-time judgment on the driver's fatigue status.

[0005] The present invention achieves the above-mentioned object through the following technical solution: an image enhancement method for nighttime fatigue driving detection, the image enhancement method comprising the following steps:

[0006] S1, real-time image acquisition, using the vehicle-mounted camera to shoot the driver's video in real time and extract the driver's image from the video;

[0007] S2, image quality judgment, judges the brightness of the image and divides the image into night image and normal light image;

[0008] S3, normal light image processing, uses the image denoising module to process the collected driver image to improve image quality and reduce interference; uses the image enhancement module to enhance the driver image to enhance the image's brightness, color, and clarity.

[0009] S4. Low-light image processing: First, perform low-light image enhancement based on Retinex theory; build and train a Retinex model to decompose the image into reflectance and illumination components; use the night image reflectance component, night image illumination component, reference image reflectance component, and reference image illumination component to train the enhancement network model, input the collected low-light image, and obtain the enhanced image;

[0010] S5. Build an infrared and visible light fusion network model, use infrared image and normal light image datasets to train the fusion network, input the enhanced image and infrared image into the fusion network to obtain the final image for driver fatigue detection;

[0011] S6, face region segmentation, dividing the fused face region and extracting key features;

[0012] S7, Fatigue identification, detects the fatigue state. If the detection result is fatigue, intelligent reminders are given, such as tightening the seat belt, voice prompts and vibrating seats.

[0013] As a further solution of the present invention: in S2, when the average brightness of the image is higher than 0.45, the image is divided into a normal light image and conventional preprocessing of the normal light image is performed; when the average brightness of the image is lower than 0.45, the image is divided into a night image and low-light image enhancement based on Retinex theory and image enhancement by fusion of infrared and visible light are performed.

[0014] As a further solution of the present invention: in S4, constructing and training a Retinex model to perform low-light image enhancement based on the Retinex theory specifically includes:

[0015] S41. Build a data set, collect nighttime images under visible light sources and corresponding normal light reference images, divide them into training sets and test sets in proportion, annotate the images, and generate corresponding enhancement targets;

[0016] S42. Build a Retinex night image enhancement model based on the U-Net structure;

[0017] S43, using the data set created in S41 to perform multiple iterative training on the decomposition network until convergence is achieved or the set maximum number of iterations is reached, and the performance of the trained neural network model is evaluated using the test set. If the performance requirements are not met, the training is continued;

[0018] S44, using the data set created in S41 to perform multiple iterative training on the enhanced network until convergence is achieved or the set maximum number of iterations is reached, and the performance of the trained neural network model is evaluated using the test set. If the performance requirements are not met, the training is continued;

[0019] S45. Use the trained low-light image enhancement network to enhance the night image to be enhanced.

[0020] As a further solution of the present invention, the Retinex nighttime image enhancement model specifically includes: an input layer of a decomposition network receives image data to be processed, with a size of 256×256×3; downsampling is performed through successive convolutional layers and maximum pooling layers, so that the feature map can extract low-level and high-level features; the intermediate layer uses multiple convolutional blocks and ReLU activation functions to enhance nonlinear feature representation; the output layer generates decomposed image data, ensuring that the output is consistent with the input image in size, and using a mean square error loss function to optimize model performance;

[0021] The input layer of the enhancement network receives the reflection and illumination components from the decomposition network. The CBAM attention mechanism is added during downsampling to enhance the network's focus on important features. The downsampling process gradually restores the image resolution through transposed convolution layers, and uses jump connections to fuse the feature maps in the encoder with those in the decoder to ensure that detailed information is preserved. The output layer generates an enhanced image of the same size as the input image through 1x1 convolution. The smoothness loss function is used to optimize model performance, making the generated result closer to the real image in terms of visual quality.

[0022] The principles of the Retinex network model are as follows:

[0023] I(x,y)=L(x,y)*R(x,y)

[0024] Where I(x,y) represents the image information of the image observed or captured by the camera at the (x,y) coordinate, which is the input of the network model. L(x,y) represents the illumination component of the light source. R(x,y) represents the reflection component of the target object in the image for the long, medium and short wavelengths of the light. * represents pixel multiplication.

[0025] The loss function used by the decomposition network is the mean square error loss function, and the formula is as follows:

[0026]

[0027] Where N is the number of samples, which represents the total number of pixels in the image, and y (i,j) is the true color value of the pixel at position (i, j) of the input original image, It is the color value predicted by the pixel at the (i, j) position predicted by the decomposition network;

[0028] The loss function used by the enhanced network is the smoothness loss function, and the formula is as follows:

[0029]

[0030] Among them, I(x i,j ) represents the pixel value of the image at position (x, y), |I(x i,j )-I(x i+1,j )| represents the absolute difference between adjacent pixels in the horizontal direction of the target image; |I(x i,j )-I(x i,j+1 )| represents the absolute difference between adjacent pixels in the vertical direction of the target image;

[0031] The decomposition network is used to transform the night image S1 under visible light source and the corresponding normal light reference image S h Decomposed into reflection component and illumination component respectively, the night image reflection component R1, night image illumination component I1, reference image reflection component R h and the reference image illumination component I h ; Using the night image reflection component R1 and the reference image reflection component R h , enhance the reflection component through the enhancement network; use the night image illumination component I1 and the reference image illumination component I h , the illumination component is enhanced by the enhancement network; the enhanced reflection component and illumination component are fused to obtain the enhanced image under visible light source.

[0032] As a further solution of the present invention: S5 specifically includes:

[0033] S51. Build a data set, collect infrared and visible light images of night scenes, perform preprocessing, take a part of them to build a sample training set, and the other part to build a test set;

[0034] S52. Build an infrared and visible light fusion network model;

[0035] S53. Design a loss function by combining pixel loss function and gradient loss function;

[0036] S54, using the data set created in S51 to perform multiple iterations of training on the infrared and visible light fusion network until convergence is achieved or the set maximum number of iterations is reached, and the performance of the trained neural network model is evaluated using the test set. If the performance requirements are not met, the training is continued;

[0037] S55. Fusing the infrared light source image and the enhanced visible light source image using the trained network model to obtain a final image for determining the driver's fatigue state.

[0038] As a further solution of the present invention: the network model specifically includes: an input layer receives the infrared image and visible light image data to be processed, with a size of 256×256×3; feature extraction is performed through multiple convolutional layers, so that the feature map can effectively obtain key information in the image; each convolutional layer is followed by a ReLU activation function to enhance the nonlinear feature representation; the fusion layer uses the SE module to weight the extracted features to improve the expressive power of important features; finally, the output layer generates a fused image of the same size as the input image through 1x1 convolution, and uses perceptual loss to optimize the model performance, so that the generated result is closer to the real image in visual quality.

[0039] As a further solution of the present invention: the pixel loss function formula is as follows:

[0040]

[0041] Among them, y (i,j) is the true color value of the pixel at position (i, j) in the input image, is the color value of the pixel at position (i, j) predicted by the fusion network; δ is a threshold parameter that determines the switching point where the loss function changes from quadratic loss to linear loss;

[0042] The gradient loss function formula is as follows:

[0043]

[0044] in, Represents the gradient operator, which is used to measure the gradient difference between the output image and the true image;

[0045] The formula of the combined loss function is as follows:

[0046]

[0047] Among them, λ1 and λ2 are weight coefficients used to balance the impact of the two loss terms.

[0048] As a further solution of the present invention: in S6, the division of the face area specifically includes:

[0049] S61, in the process of processing the facial image, firstly divide the facial region and fuse different image information to better define and identify the various components of the face;

[0050] S62. Extract key features from the divided face regions; these features include but are not limited to important facial landmarks such as eyes, nose, and mouth, as well as the overall contour and texture information of the face.

[0051] As a further solution of the present invention: in S7, fatigue driving detection includes eye width ratio detection, specifically including:

[0052] Eye width ratio detection is performed using the EAR algorithm, which identifies the eye contour, calculates the distance between the positioning points, and derives the corresponding aspect ratio based on this.

[0053] The calculation formula is as follows:

[0054]

[0055] Among them, ‖p2-p6‖ and ‖p3-p5‖ represent the vertical distances between the upper and lower eyelids.

[0056] ‖p1-p4‖ represents the calculated horizontal width of the eye;

[0057] When the driver is awake, the EAR value will vary within a specific range; when the driver is fatigued, causing the eyes to close, the EAR value will drop to 0;

[0058] During the test, the presence of blinking behavior is determined by comparing the fluctuations of the EAR value in a short period of time. If the blinking frequency within 1 minute is significantly higher than normal, it is determined to be fatigue driving;

[0059] The driver's fatigue status is detected in night-time images; if the detection result is fatigue, intelligent reminders are given, such as tightening the seat belt, voice prompts, and seat vibration.

[0060] The beneficial effects of the present invention are as follows: the method of introducing deep learning into low-light image enhancement based on Retinex theory performs well in visual tasks such as denoising and feature enhancement. Compared with traditional technologies, it also shows higher accuracy, robustness and processing speed in the field of night image enhancement. At the same time, it has more outstanding effects than traditional methods in improving image details and texture quality. It can enhance the driver's night image, making the image clearer and brighter, which is conducive to the fatigue detection system to detect the driver's fatigue state in real time. The present invention adopts the fusion of infrared light and visible light for nighttime fatigue driving detection. The unique acquisition principle of infrared image enables it to work in complete darkness or low light conditions without relying on external light sources, which makes it unaffected by changes in lighting and can effectively highlight significant targets. The fusion of infrared and visible light images can make full use of the rich detailed information contained in the visible light image, thereby improving the clarity and recognizability of the overall image. This cross-modal fusion not only enhances the accuracy of nighttime fatigue detection, but also significantly improves the efficiency of real-time monitoring, enabling the system to quickly and accurately identify potential safety hazards, thereby effectively preventing accidents caused by fatigue driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Fig. 1 This is a flowchart of nighttime image enhancement and fatigue detection according to the present invention;

[0062] Fig. 2 This is a schematic diagram of the nighttime image enhancement principle of the present invention;

[0063] Fig. 3 This is a structural diagram of the night image enhancement network model based on Retinex theory of the present invention;

[0064] Fig. 4 This is a structural diagram of the infrared and visible light fusion network model of the present invention. DETAILED DESCRIPTION

[0065] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0066] Example 1, as Figs. 1 to 4 As shown, an image enhancement method for nighttime fatigue driving detection includes the following steps:

[0067] S1, real-time image acquisition, using the vehicle-mounted camera to shoot the driver's video in real time and extract the driver's image from the video;

[0068] S2, image quality judgment, judging the brightness of the image, and dividing the image into night image and normal light image;

[0069] S3, normal light image processing, using the image denoising module to process the collected driver image, improving the image quality and reducing interference; through the image enhancement module, the driver image is enhanced, the brightness, color and definition of the image are enhanced.

[0070] S4, low light image processing, first, low light image enhancement based on Retinex theory; construct and train Retinex model, decompose the image into reflection component and illumination component; use night image reflection component, night image illumination component, reference image reflection component and reference image illumination component to train enhancement network model, input the collected low light image, get the enhanced image;

[0071] S5, construct infrared and visible light fusion network model, use infrared image and normal light image data set to train fusion network, input the enhanced image and infrared light image into the fusion network to get the final image, which is used for driver fatigue detection;

[0072] S6, face region division, divide the fused face region and extract key features;

[0073] S7, fatigue discrimination, detect the fatigue state, if the detection result is fatigue, intelligent reminding is carried out, such as tightening the safety belt, voice prompt and vibrating seat.

[0074] Embodiment two, in addition to including all the technical features in embodiment one, it also includes:

[0075] In S2, when the average brightness of the image is higher than 0.45, the image is divided into normal light image, and the normal light image is preprocessed; when the average brightness of the image is lower than 0.45, the image is divided into night image, and the low light image enhancement based on Retinex theory and the infrared and visible light fusion image enhancement are carried out.

[0076] In S4, the Retinex model is constructed and trained, and the low light image enhancement based on Retinex theory for low light image includes:

[0077] S41, construct data set, collect night image under visible light source and corresponding normal light reference image, divide it into training set and test set according to proportion, label the image, and generate corresponding enhancement target;

[0078] S42, build Retinex night image enhancement model based on U-Net structure;

[0079] S43, using the data set created in S41 to perform multiple iterative training on the decomposition network until convergence is achieved or the set maximum number of iterations is reached, and the performance of the trained neural network model is evaluated using the test set. If the performance requirements are not met, the training is continued;

[0080] S44, using the data set created in S41 to perform multiple iterative training on the enhanced network until convergence is achieved or the set maximum number of iterations is reached, and the performance of the trained neural network model is evaluated using the test set. If the performance requirements are not met, the training is continued;

[0081] S45. Use the trained low-light image enhancement network to enhance the night image to be enhanced.

[0082] The Retinex nighttime image enhancement model specifically includes the following steps: the input layer of the decomposition network receives the image data to be processed, with a size of 256×256×3; downsampling is performed through successive convolutional layers and maximum pooling layers, enabling the feature map to extract both low-level and high-level features; the intermediate layers use multiple convolutional blocks and ReLU activation functions to enhance nonlinear feature representation; the output layer generates the decomposed image data, ensuring that the output is consistent in size with the input image, and using the mean squared error loss function to optimize model performance;

[0083] The input layer of the enhancement network receives the reflection and illumination components from the decomposition network. The CBAM attention mechanism is added during downsampling to enhance the network's focus on important features. The downsampling process gradually restores the image resolution through transposed convolution layers, and uses jump connections to fuse the feature maps in the encoder with those in the decoder to ensure that detailed information is preserved. The output layer generates an enhanced image of the same size as the input image through 1x1 convolution. The smoothness loss function is used to optimize model performance, making the generated result closer to the real image in terms of visual quality.

[0084] The principles of the Retinex network model are as follows:

[0085] I(x,y)=L(x,y)*R(x,y)

[0086] Where I(x,y) represents the image information of the image observed or captured by the camera at the (x,y) coordinate, which is the input of the network model. L(x,y) represents the illumination component of the light source. R(x,y) represents the reflection component of the target object in the image for the long, medium and short wavelengths of the light. * represents pixel multiplication.

[0087] The loss function used by the decomposition network is the mean square error loss function, and the formula is as follows:

[0088]

[0089] Where N is the number of samples, which represents the total number of pixels in the image, and y (i,j) is the true color value of the pixel at position (i, j) of the input original image, It is the color value predicted by the pixel at the (i, j) position predicted by the decomposition network;

[0090] The loss function used by the enhanced network is the smoothness loss function, and the formula is as follows:

[0091]

[0092] Among them, I(x i,j ) represents the pixel value of the image at position (x, y), |I(x i,j )-I(x i+1,j )| represents the absolute difference between adjacent pixels in the horizontal direction of the target image; |I(x i,j )-I(x i,j+1 )| represents the absolute difference between adjacent pixels in the vertical direction of the target image;

[0093] The decomposition network is used to transform the night image S1 under visible light source and the corresponding normal light reference image S h Decomposed into reflection component and illumination component respectively, the night image reflection component R1, night image illumination component I1, reference image reflection component R h and the reference image illumination component I h ; Using the night image reflection component R1 and the reference image reflection component R h , enhance the reflection component through the enhancement network; use the night image illumination component I1 and the reference image illumination component I h , the illumination component is enhanced by the enhancement network; the enhanced reflection component and illumination component are fused to obtain the enhanced image under visible light source.

[0094] In addition to all the technical features of the first embodiment, this embodiment also includes:

[0095] S5 specifically includes:

[0096] S51. Build a data set, collect infrared and visible light images of night scenes, perform preprocessing, take a part of them to build a sample training set, and the other part to build a test set;

[0097] S52. Build an infrared and visible light fusion network model;

[0098] S53. Design a loss function by combining pixel loss function and gradient loss function;

[0099] S54, using the data set created in S51 to perform multiple iterations of training on the infrared and visible light fusion network until convergence is achieved or the set maximum number of iterations is reached, and the performance of the trained neural network model is evaluated using the test set. If the performance requirements are not met, the training is continued;

[0100] S55. Fusing the infrared light source image and the enhanced visible light source image using the trained network model to obtain a final image for determining the driver's fatigue state.

[0101] Furthermore, the network model specifically includes: the input layer receives the infrared image and visible light image data to be processed, with a size of 256×256×3; feature extraction is performed through multiple convolutional layers, so that the feature map can effectively obtain the key information in the image; each convolutional layer is followed by a ReLU activation function to enhance the nonlinear feature representation; the fusion layer uses the SE module to weight the extracted features to improve the expressiveness of important features; finally, the output layer generates a fused image of the same size as the input image through 1x1 convolution, and uses perceptual loss to optimize the model performance, so that the generated result is closer to the real image in visual quality.

[0102] Furthermore, the pixel loss function formula is as follows:

[0103]

[0104] Among them, y (i,j) is the true color value of the pixel at position (i, j) in the input image, is the color value of the pixel at position (i, j) predicted by the fusion network; δ is a threshold parameter that determines the switching point where the loss function changes from quadratic loss to linear loss;

[0105] The gradient loss function formula is as follows:

[0106]

[0107] in, Represents the gradient operator, which is used to measure the gradient difference between the output image and the true image;

[0108] The formula of the combined loss function is as follows:

[0109]

[0110] Among them, λ1 and λ2 are weight coefficients used to balance the impact of the two loss terms.

[0111] Furthermore, the division of the face area specifically includes:

[0112] S61, in the process of processing the facial image, firstly divide the facial region and fuse different image information to better define and identify the various components of the face;

[0113] S62. Extract key features from the divided face regions; these features include but are not limited to important facial landmarks such as eyes, nose, and mouth, as well as the overall contour and texture information of the face.

[0114] Furthermore, fatigue driving detection includes eye width ratio detection, specifically including:

[0115] Eye width ratio detection is performed using the EAR algorithm, which identifies the eye contour, calculates the distance between the positioning points, and derives the corresponding aspect ratio based on this.

[0116] The calculation formula is as follows:

[0117]

[0118] Among them, ‖p2-p6‖ and ‖p3-p5‖ represent the vertical distances between the upper and lower eyelids.

[0119] ‖p1-p4‖ represents the calculated horizontal width of the eye;

[0120] When the driver is awake, the EAR value will vary within a specific range; when the driver is fatigued, causing the eyes to close, the EAR value will drop to 0;

[0121] Under normal circumstances, a blink lasts approximately 0.01 to 0.02 seconds, and a complete blink takes approximately one to three frames. However, when fatigued, blinking slows down and the frequency of blinking increases. A blink is considered to have occurred when the EAR value rapidly drops below a certain threshold and then quickly rises back above it. First, the aspect ratio (EAR value) of the driver's left and right eyes in their normal state is calculated separately, and the average is calculated to obtain the final EAR value. This EAR value represents the driver's normal eye width ratio, eliminating the influence of individual differences.

[0122] During the detection process, the presence of blinking behavior is determined by comparing the fluctuations of the EAR value in a short period of time; if the blinking frequency within 1 minute is significantly higher than normal, it is determined to be fatigue driving.

[0123] The driver's fatigue status is detected in night-time images; if the detection result is fatigue, intelligent reminders are given, such as tightening the seat belt, voice prompts, and seat vibration.

[0124] Working principle: Use the decomposition network to decompose the night image S1 under visible light source and the corresponding normal light reference image S hdecompose into a reflection component and an illumination component respectively, to obtain a night image reflection component R1, a night image illumination component I1, a reference image reflection component R h and a reference image illumination component I h ; the night image reflection component R1 and the reference image reflection component R h are used to enhance the reflection component through an enhancement network; the night image illumination component I1 and the reference image illumination component I h are used to enhance the illumination component through an enhancement network; the enhanced reflection component and the enhanced illumination component are fused to obtain an enhanced visible light source image; the infrared light source image S r is fused with the enhanced visible light source image to obtain a final image for judging the driver fatigue state.

[0125] It is apparent for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, but can be implemented in other concrete forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be considered in all respects as illustrative and not restrictive, the scope of the present application being defined by the appended claims rather than the above description, and it is intended to embrace all changes and modifications that fall within the meaning and scope of the equivalent elements of the claims. Any reference signs in the claims should not be construed as limiting the claims to the figures in which the reference signs are used.

[0126] Furthermore, it should be understood that although the present specification is described in terms of embodiments, not every embodiment contains only one independent technical solution, and the description of the specification is only for the sake of clarity, and those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can be appropriately combined to form other embodiments that those skilled in the art can understand.

Claims

1. An image enhancement method for nighttime fatigue driving detection, characterized in that: The image enhancement method comprises the following steps: S1, real-time image acquisition, using the vehicle-mounted camera to shoot the driver's video in real time and extract the driver's image from the video; S2, image quality judgment, judges the brightness of the image and divides the image into night image and normal light image; S3, normal light image processing, using the image denoising module to process the collected driver image to improve image quality and reduce interference, and using the image enhancement module to enhance the driver image to enhance the image brightness, color and clarity; S4. Low-light image processing: First, perform low-light image enhancement based on Retinex theory; build and train a Retinex model to decompose the image into reflectance and illumination components; use the night image reflectance component, night image illumination component, reference image reflectance component, and reference image illumination component to train the enhancement network model, input the collected low-light image, and obtain the enhanced image; S5. Build an infrared and visible light fusion network model, use infrared image and normal light image datasets to train the fusion network, input the enhanced image and infrared image into the fusion network to obtain the final image for driver fatigue detection; S6, face region segmentation, dividing the fused face region and extracting key features; S7, fatigue identification, detects fatigue status, and if the detection result is fatigue, provides intelligent reminders, such as tightening seat belts, voice prompts, and seat vibration; In S4, the Retinex model is constructed and trained, and low-light image enhancement based on the Retinex theory is performed on low-light images, specifically including: S41. Build a data set, collect nighttime images under visible light sources and corresponding normal light reference images, divide them into training sets and test sets in proportion, annotate the images, and generate corresponding enhancement targets; S42. Build a Retinex night image enhancement model based on the U-Net structure; S43, using the training set created in S41 to perform multiple iterative training on the decomposition network until convergence is achieved or the set maximum number of iterations is reached, and using the test set to evaluate the performance of the trained neural network model. If the performance requirements are not met, continue training; S44, using the training set created in S41 to perform multiple iterative training on the enhanced network until convergence is achieved or the set maximum number of iterations is reached, and the performance of the trained neural network model is evaluated using the test set. If the performance requirements are not met, the training is continued; S45. Use the trained low-light image enhancement network to enhance the night image to be enhanced; The Retinex nighttime image enhancement model specifically includes: the input layer of the decomposition network receives the image data to be processed, with a size of 256×256×3; downsampling is performed through consecutive convolutional layers and maximum pooling layers, so that the feature map can extract low-level and high-level features; the intermediate layer uses multiple convolutional blocks and ReLU activation functions to enhance nonlinear feature representation; the output layer generates the decomposed image data, ensuring the consistency of the output and input image in size, and uses the mean square error loss function to optimize model performance; The input layer of the enhancement network receives the reflection and illumination components from the decomposition network. The CBAM attention mechanism is added during downsampling to enhance the network's focus on important features. The downsampling process gradually restores the image resolution through transposed convolution layers, and uses jump connections to fuse the feature maps in the encoder with those in the decoder to ensure that detailed information is preserved. The output layer generates an enhanced image of the same size as the input image through 1x1 convolution. The smoothness loss function is used to optimize model performance, making the generated result closer to the real image in terms of visual quality. The principles of the Retinex network model are as follows: I(x,y)=L(x,y)*R(x,y) Where I(x,y) represents the image information of the image observed or captured by the camera at the (x,y) coordinate, which is the input of the network model. L(x,y) represents the illumination component of the light source. R(x,y) represents the reflection component of the target object in the image for the long, medium and short wavelengths of the light. * represents pixel multiplication. The loss function used by the decomposition network is the mean square error loss function, and the formula is as follows: Where N is the number of samples, which represents the total number of pixels in the image, and y (i,j) is the true color value of the pixel at position (i, j) of the input original image, It is the color value predicted by the pixel at the (i, j) position predicted by the decomposition network; The loss function used by the enhanced network is the smoothness loss function, and the formula is as follows: Among them, I(x i,j ) represents the pixel value of the image at position (x, y), |I(x i,j )-I(x i+1,j )| represents the absolute difference between adjacent pixels in the horizontal direction of the target image; |I(x i,j )-I(x i,j+1 )| represents the absolute difference between adjacent pixels in the vertical direction of the target image; The decomposition network is used to transform the night image S1 under visible light source and the corresponding normal light reference image S h Decomposed into reflection component and illumination component respectively, the night image reflection component R1, night image illumination component I1, reference image reflection component R h and the reference image illumination component I h ; Using the night image reflection component R1 and the reference image reflection component R h , enhance the reflection component through the enhancement network; use the night image illumination component I1 and the reference image illumination component I h , the illumination component is enhanced by the enhancement network; the enhanced reflection component and illumination component are fused to obtain the enhanced image under visible light source.

2. The image enhancement method according to claim 1, wherein: In S2, when the average brightness of the image is higher than 0.45, the image is classified as a normal light image and conventional preprocessing of the normal light image is performed; when the average brightness of the image is lower than 0.45, the image is classified as a night image and low light image enhancement based on Retinex theory and image enhancement by fusion of infrared and visible light are performed.

3. The image enhancement method according to claim 1, wherein: Said S5 specifically includes: S51. Build a data set, collect infrared and visible light images of night scenes, perform preprocessing, take a part of them to build a sample training set, and the other part to build a test set; S52. Build an infrared and visible light fusion network model; S53. Design a loss function by combining pixel loss function and gradient loss function; S54, using the training set created in S51 to perform multiple iterative training on the infrared and visible light fusion network until convergence is achieved or the set maximum number of iterations is reached, and using the test set to evaluate the performance of the trained neural network model. If the performance requirements are not met, continue training; S55. Fusing the infrared light source image and the enhanced visible light source image using the trained network model to obtain a final image for determining the driver's fatigue state.

4. The image enhancement method according to claim 3, wherein: The network model specifically includes: The input layer receives the infrared image and visible light image data to be processed, with a size of 256×256×3; Feature extraction is performed through multiple convolutional layers, so that the feature map can effectively obtain the key information in the image; Each convolutional layer is followed by a ReLU activation function to enhance nonlinear feature representation; The fusion layer uses the SE module to weight the extracted features to improve the expressiveness of important features; Finally, the output layer generates a fused image of the same size as the input image through 1x1 convolution, and uses perceptual loss to optimize the model performance, making the generated result closer to the real image in visual quality.

5. The image enhancement method according to claim 4, wherein: The pixel loss function formula is as follows: Among them, y (i,j) is the true color value of the pixel at position (i, j) in the input image, is the color value of the pixel at position (i, j) predicted by the fusion network; δ is a threshold parameter that determines the switching point where the loss function changes from quadratic loss to linear loss; The gradient loss function formula is as follows: in, represents the gradient operator, which is used to measure the gradient difference between the output image and the true image; The formula of the combined loss function is as follows: Among them, λ1 and λ2 are weight coefficients used to balance the impact of the two loss terms.

6. The image enhancement method according to claim 1, wherein: In S6, the division of the face area specifically includes: S61. During the facial image processing, the facial region is first divided and different image information is integrated to better define and identify the various components of the face. S62. Extract key features from the divided face regions; these features include but are not limited to important facial landmarks such as eyes, nose, and mouth, as well as the overall contour and texture information of the face.

7. The image enhancement method according to claim 1, wherein: In S7, fatigue driving detection includes eye width ratio detection, which specifically includes: Eye width ratio detection is performed using the EAR algorithm, which identifies the eye contour, calculates the distance between the positioning points, and derives the corresponding aspect ratio based on this. The calculation formula is as follows: Among them, ‖p2-p6‖ and ‖p3-p5‖ represent the vertical distances between the upper eyelid and the lower eyelid, ‖p1-p4‖ represents the calculated horizontal width of the eye; When the driver is awake, the EAR value will vary within a specific range; when the driver is fatigued, causing the eyes to close, the EAR value will drop to 0; During the test, the presence of blinking behavior is determined by comparing the fluctuations of the EAR value in a short period of time. If the blinking frequency within 1 minute is significantly higher than normal, it is determined to be fatigue driving; The driver's fatigue status is detected in night images; if the detection result is fatigue, intelligent reminders are given, such as tightening the seat belt, voice prompts, and seat vibration.

Images