Image enhancement method for night fatigue driving detection

Abstract

The invention discloses an image enhancement method for night fatigue driving detection, and the method comprises the steps: decomposing a night image under a visible light source and a corresponding normal light reference image into a reflection component and an illumination component through a decomposition network; obtaining a night image reflection component, a night image illumination component, a reference image reflection component and a reference image illumination component; enhancing the reflection components through an enhancement network by using the night image reflection component and the reference image reflection component; using the night image illumination component and the reference image illumination component to enhance the illumination component through an enhancement network; fusing the enhanced reflection component and illumination component to obtain an enhanced image under the visible light source; and fusing the infrared light source image and the enhanced image under the visible light source to obtain a final image which is used for judging the fatigue state of the driver. According to the method, the night image of the driver is effectively enhanced, the image is clearer and brighter, and the accuracy of a fatigue detection system is improved.

Description

Technical Field

[0001] The invention relates to an image enhancement method, in particular to an image enhancement method for nighttime fatigue driving detection, and belongs to the technical field of image processing. Background Art

[0002] With the rapid development of the global economy, motor vehicles are becoming more and more popular in our lives. Convenient transportation has led to an increase in traffic accidents. According to statistics, fatigue driving is one of the main causes of traffic accidents, accounting for about 10% to 20% of the total number of accidents. Among them, night driving is more prone to neural and visual fatigue, leading to slow reaction and lack of concentration, which in turn increases the risk of accidents. Traditional fatigue detection methods are prone to generate a lot of noise when the light intensity is insufficient, resulting in poor image quality, unclear feature extraction, reduced recognition accuracy and untimely reminders, which brings great safety hazards to night driving.

[0003] Therefore, it is necessary to process the low-light image to improve the image quality. In the prior art, a low-light image enhancement method based on the deep Retinex theory disclosed in the publication number CN117455822A includes: decomposing the low-light image and the reference image in the data set into corresponding reflection maps and illumination maps; establishing an optimization objective function, and introducing constraints on the structure and texture of the two sub-images, and iteratively solving the optimal solution of the optimization objective function by the alternating direction minimization method; in the iterative optimization process, a deep neural network prior estimation model is used to guide the optimization. The illumination map and the reflection map obtained by multiple iterations are multiplied to obtain the final enhanced image. The traditional low-light image enhancement method includes histogram equalization, gamma transform, dark channel prior, denoising, and multi-scale Retinex. Among them, histogram equalization and gamma correction can enhance the contrast and brightness of the image, but over-enhancement will occur. Dark channel verification and multi-scale Retinex are complex to calculate, which affects the real-time fatigue detection speed. Denoising inevitably results in detail loss and uneven processing. Summary of the invention

[0004] The purpose of the present invention is to provide an image enhancement method for nighttime fatigue driving detection in order to solve at least one of the above-mentioned technical problems, which can effectively enhance the driver's night image, make the image clearer and brighter, and facilitate the fatigue detection system to make real-time judgment on the driver's fatigue state.

[0005] The present invention achieves the above-mentioned object through the following technical scheme: an image enhancement method for nighttime fatigue driving detection, the image enhancement method comprising the following steps:

[0006] S1, real-time image acquisition, using the vehicle-mounted camera to shoot the driver's video in real time and extract the driver's image from the video;

[0007] S2, image quality judgment, judging the brightness of the image and dividing the image into nighttime image and normal light image;

[0008] S3, normal light image processing, using the image denoising module to process the collected driver image to improve image quality and reduce interference; the driver image is enhanced through the image enhancement module to enhance the image brightness, color and clarity.

[0009] S4, low-light image processing, firstly, low-light image enhancement based on Retinex theory is performed; a Retinex model is constructed and trained to decompose the image into a reflection component and an illumination component; the enhancement network model is trained using the night image reflection component, the night image illumination component, the reference image reflection component and the reference image illumination component, and the collected low-light image is input to obtain the enhanced image;

[0010] S5. Construct an infrared and visible light fusion network model, use infrared image and normal light image data sets to train the fusion network, input the enhanced image and infrared light image into the fusion network to obtain the final image for driver fatigue detection;

[0011] S6, face area segmentation, segmenting the fused face area and extracting key features;

[0012] S7, fatigue identification, detects the fatigue status. If the detection result is fatigue, intelligent reminders are given, such as tightening the seat belt, voice prompts and vibrating seats.

[0013] As a further solution of the present invention: in S2, when the average brightness of the image is higher than 0.45, the image is divided into a normal light image, and conventional preprocessing of the normal light image is performed; when the average brightness of the image is lower than 0.45, the image is divided into a night image, and low-light image enhancement based on Retinex theory and image enhancement by fusion of infrared and visible light are performed.

[0014] As a further solution of the present invention: in S4, constructing and training a Retinex model, and performing low-light image enhancement based on the Retinex theory on the low-light image specifically includes:

[0015] S41, constructing a data set, collecting night images under visible light sources and corresponding normal light reference images, dividing them into a training set and a test set in proportion, annotating the images, and generating corresponding enhanced targets;

[0016] S42. Build a Retinex night image enhancement model based on U-Net structure;

[0017] S43, using the data set created in S41 to perform multiple iterations of training on the decomposition network until convergence is achieved or the set maximum number of iterations is reached, and the performance of the trained neural network model is evaluated using the test set, and if the performance requirement is not met, the training is continued;

[0018] S44, using the data set created in S41 to perform multiple iterations of training on the enhanced network until convergence is achieved or the set maximum number of iterations is reached, and the performance of the trained neural network model is evaluated using the test set, and if the performance requirement is not met, the training is continued;

[0019] S45. Use the trained low-light image enhancement network to enhance the night image to be enhanced.

[0020] As a further solution of the present invention: the Retinex night image enhancement model specifically includes: the input layer of the decomposition network receives the image data to be processed, with a size of 256×256×3; downsampling is performed through continuous convolutional layers and maximum pooling layers, so that the feature map can extract low-level and high-level features; the middle layer uses multiple convolutional blocks and ReLU activation functions to enhance the nonlinear feature representation; the output layer generates the decomposed image data, ensures the consistency of the output and input images in size, and uses the mean square error loss function to optimize the model performance;

[0021] The input layer of the enhancement network receives the reflection component and illumination component from the decomposition network; the CBAM attention mechanism is added during downsampling to enhance the network's attention to important features, and the downsampling process gradually restores the image resolution through the transposed convolution layer, and uses jump connections to fuse the feature map in the encoder with the feature map in the decoder to ensure that the detail information is preserved; the output layer generates an enhanced image of the same size as the input image through 1x1 convolution, and uses a smoothness loss function to optimize the model performance, so that the generated result is closer to the real image in visual quality;

[0022] The principle of the Retinex network model is as follows:

[0023] I(x,y)=L(x,y)*R(x,y)

[0024] Among them, I(x,y) represents the image information of the image observed or captured by the camera at the (x,y) coordinate, that is, the input of the network model, L(x,y) represents the illumination component of the light source, R(x,y) represents the reflection component of the target object in the image for the long, medium and short waves of the light, and * represents pixel multiplication;

[0025] The loss function used by the decomposition network is the mean square error loss function, and the formula is as follows:

[0026]

[0027] Where N is the number of samples, which represents the total number of pixels in the image, and y (i,j) is the true color value of the pixel at position (i, j) of the input original image, It is the color value predicted by the pixel at the (i, j) position predicted by the decomposition network;

[0028] The loss function used by the enhanced network is the smoothness loss function, and the formula is as follows:

[0029]

[0030] Among them, I(x i,j ) represents the pixel value of the image at position (x, y), |I(x i,j )-I(x i+1,j )| represents the absolute difference between adjacent pixels in the horizontal direction of the target image; |I(x i,j )-I(x i,j+1 )| represents the absolute difference between adjacent pixels in the vertical direction of the target image;

[0031] The decomposition network is used to transform the night image S1 under visible light source and the corresponding normal light reference image S h Decomposed into reflection component and illumination component respectively, the reflection component R1 of night image, the illumination component I1 of night image, and the reflection component R h and the reference image illumination component I h ; Using the night image reflection component R1 and the reference image reflection component R h , enhance the reflection component through the enhancement network; use the night image illumination component I1 and the reference image illumination component I h , the illumination component is enhanced through the enhancement network; the enhanced reflection component and the illumination component are fused to obtain the enhanced image under the visible light source.

[0032] As a further solution of the present invention: S5 specifically includes:

[0033] S51, construct a data set, collect infrared and visible light images of night scenes, perform preprocessing, take a part of them to construct a sample training set, and the other part to construct a test set;

[0034] S52. Build an infrared and visible light fusion network model;

[0035] S53, designing a loss function by combining the pixel loss function and the gradient loss function;

[0036] S54, using the data set created in S51 to perform multiple iterations of training on the infrared and visible light fusion network until convergence is achieved or the set maximum number of iterations is reached, and the performance of the trained neural network model is evaluated using the test set, and if the performance requirement is not met, the training is continued;

[0037] S55. The infrared light source image and the enhanced visible light source image are fused using the trained network model to obtain a final image for judging the driver's fatigue state.

[0038] As a further solution of the present invention: the network model specifically includes: an input layer receives infrared images and visible light image data to be processed, with a size of 256×256×3; feature extraction is performed through multiple convolutional layers, so that the feature map can effectively obtain key information in the image; each convolutional layer is followed by a ReLU activation function to enhance the nonlinear feature representation; the fusion layer uses the SE module to weight the extracted features to improve the expression ability of important features; finally, the output layer generates a fused image of the same size as the input image through 1x1 convolution, and uses perceptual loss to optimize the model performance, so that the generated result is closer to the real image in visual quality.

[0039] As a further solution of the present invention: the pixel loss function formula is as follows:

[0040]

[0041] Among them, y (i,j) is the true color value of the pixel at position (i, j) of the input image, It is the color value predicted by the pixel at the (i, j) position predicted by the fusion network; δ is a threshold parameter that determines the switching point where the loss function changes from quadratic loss to linear loss;

[0042] The gradient loss function formula is as follows:

[0043]

[0044] in, represents the gradient operator, which is used to measure the gradient difference between the output image and the true image;

[0045] The formula of the combined loss function is as follows:

[0046]

[0047] Among them, λ1 and λ2 are weight coefficients used to balance the impact of the two loss terms.

[0048] As a further solution of the present invention: in S6, the division of the face area specifically includes:

[0049] S61, in the process of processing the face image, firstly divide the face area and fuse different image information to better define and identify the various components of the face;

[0050] S62. Extract key features from the divided face regions; these features include but are not limited to important facial landmarks such as eyes, nose, and mouth, as well as the overall contour and texture information of the face.

[0051] As a further solution of the present invention: in S7, fatigue driving detection includes eye width ratio detection, specifically including:

[0052] Eye aspect ratio detection is performed using the EAR algorithm, which identifies the eye contour, calculates the distance between the positioning points, and derives the corresponding aspect ratio based on this;

[0053] The calculation formula is as follows:

[0054]

[0055] Among them, ‖p2-p6‖ and ‖p3-p5‖ represent the vertical distances between the upper eyelid and the lower eyelid.

[0056] ‖p1-p4‖ means calculating the horizontal width of the eye;

[0057] When the driver is awake, the EAR value will vary within a specific range; when the driver is fatigued, causing the eyes to close, the EAR value will drop to 0;

[0058] During the test, the fluctuation of EAR value in a short period of time is compared to determine whether there is blinking behavior; if the blinking frequency within 1 minute is significantly higher than normal, it is judged as fatigue driving;

[0059] The driver's fatigue state is detected in night images; if the detection result is fatigue, intelligent reminders are given, such as tightening the seat belt, voice prompts and vibrating seats.

[0060] The beneficial effects of the present invention are as follows: the method of introducing deep learning into low-light image enhancement based on Retinex theory of the present invention performs well in visual tasks such as denoising and feature enhancement, and also shows higher accuracy, robustness and processing speed in the field of night image enhancement compared with traditional technologies. At the same time, it has more prominent effects in improving image details and texture quality compared with traditional methods, and can enhance the night image of the driver, making the image clearer and brighter, which is conducive to the fatigue detection system to make real-time detection of the driver's fatigue state. The present invention adopts infrared light and visible light fusion for nighttime fatigue driving detection. The unique acquisition principle of infrared images enables it to work in complete darkness or low light conditions without relying on external light sources, which makes it unaffected by changes in illumination; it can effectively highlight significant targets. The fusion of infrared and visible light images can make full use of the rich detail information contained in the visible light image, thereby improving the clarity and recognizability of the overall image; this cross-modal fusion not only enhances the accuracy of nighttime fatigue detection, but also significantly improves the efficiency of real-time monitoring, so that the system can quickly and accurately identify potential safety hazards, thereby effectively preventing accidents caused by fatigue driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is a flow chart of nighttime image enhancement and fatigue detection of the present invention;

[0062] Figure 2 It is a principle diagram of night image enhancement of the present invention;

[0063] Figure 3 This is a structural diagram of a night image enhancement network model based on Retinex theory of the present invention;

[0064] Figure 4 This is a structural diagram of the infrared and visible light fusion network model of the present invention. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] Embodiment 1, as Figures 1 to 4 As shown, an image enhancement method for nighttime fatigue driving detection comprises the following steps:

[0067] S1, real-time image acquisition, using the vehicle-mounted camera to shoot the driver's video in real time and extract the driver's image from the video;

[0068] S2, image quality judgment, judging the brightness of the image and dividing the image into nighttime image and normal light image;

[0069] S3, normal light image processing, using the image denoising module to process the collected driver image to improve image quality and reduce interference; the driver image is enhanced through the image enhancement module to enhance the image brightness, color and clarity.

[0070] S4, low-light image processing, firstly, low-light image enhancement based on Retinex theory is performed; a Retinex model is constructed and trained to decompose the image into a reflection component and an illumination component; the enhancement network model is trained using the night image reflection component, the night image illumination component, the reference image reflection component and the reference image illumination component, and the collected low-light image is input to obtain the enhanced image;

[0071] S5. Construct an infrared and visible light fusion network model, use infrared image and normal light image data sets to train the fusion network, input the enhanced image and infrared light image into the fusion network to obtain the final image for driver fatigue detection;

[0072] S6, face area segmentation, segmenting the fused face area and extracting key features;

[0073] S7, fatigue identification, detects the fatigue status. If the detection result is fatigue, intelligent reminders are given, such as tightening the seat belt, voice prompts and vibrating seats.

[0074] Embodiment 2: In addition to all the technical features of Embodiment 1, this embodiment also includes:

[0075] In S2, when the average brightness of the image is higher than 0.45, the image is classified as a normal light image and conventional preprocessing of the normal light image is performed; when the average brightness of the image is lower than 0.45, the image is classified as a night image and low light image enhancement based on Retinex theory and image enhancement by fusion of infrared and visible light are performed.

[0076] In S4, the Retinex model is constructed and trained to enhance low-light images based on the Retinex theory. Specifically, the following steps are performed:

[0077] S41, constructing a data set, collecting night images under visible light sources and corresponding normal light reference images, dividing them into a training set and a test set in proportion, annotating the images, and generating corresponding enhanced targets;

[0078] S42. Build a Retinex night image enhancement model based on U-Net structure;

[0079] S43, using the data set created in S41 to perform multiple iterations of training on the decomposition network until convergence is achieved or the set maximum number of iterations is reached, and the performance of the trained neural network model is evaluated using the test set, and if the performance requirement is not met, the training is continued;

[0080] S44, using the data set created in S41 to perform multiple iterations of training on the enhanced network until convergence is achieved or the set maximum number of iterations is reached, and the performance of the trained neural network model is evaluated using the test set, and if the performance requirement is not met, the training is continued;

[0081] S45. Use the trained low-light image enhancement network to enhance the night image to be enhanced.

[0082] The Retinex night image enhancement model specifically includes: the input layer of the decomposition network receives the image data to be processed, with a size of 256×256×3; downsampling is performed through continuous convolutional layers and maximum pooling layers, so that the feature map can extract low-level and high-level features; the middle layer uses multiple convolutional blocks and ReLU activation functions to enhance nonlinear feature representation; the output layer generates the decomposed image data to ensure the consistency of the output and input images in size, and the mean square error loss function is used to optimize the model performance;

[0083] The input layer of the enhancement network receives the reflection component and illumination component from the decomposition network; the CBAM attention mechanism is added during downsampling to enhance the network's attention to important features, and the downsampling process gradually restores the image resolution through the transposed convolution layer, and uses jump connections to fuse the feature map in the encoder with the feature map in the decoder to ensure that the detail information is preserved; the output layer generates an enhanced image of the same size as the input image through 1x1 convolution, and uses a smoothness loss function to optimize the model performance, so that the generated result is closer to the real image in visual quality;

[0084] The principle of the Retinex network model is as follows:

[0085] I(x,y)=L(x,y)*R(x,y)

[0086] Among them, I(x,y) represents the image information of the image observed or captured by the camera at the (x,y) coordinate, that is, the input of the network model, L(x,y) represents the illumination component of the light source, R(x,y) represents the reflection component of the target object in the image for the long, medium and short waves of the light, and * represents pixel multiplication;

[0087] The loss function used by the decomposition network is the mean square error loss function, and the formula is as follows:

[0088]

[0089] Where N is the number of samples, which represents the total number of pixels in the image, and y (i,j) is the true color value of the pixel at position (i, j) of the input original image, It is the color value predicted by the pixel at the (i, j) position predicted by the decomposition network;

[0090] The loss function used by the enhanced network is the smoothness loss function, and the formula is as follows:

[0091]

[0092] Among them, I(x i,j ) represents the pixel value of the image at position (x, y), |I(x i,j )-I(x i+1,j )| represents the absolute difference between adjacent pixels in the horizontal direction of the target image; |I(x i,j )-I(x i,j+1 )| represents the absolute difference between adjacent pixels in the vertical direction of the target image;

[0093] The decomposition network is used to transform the night image S1 under visible light source and the corresponding normal light reference image S h Decomposed into reflection component and illumination component respectively, the reflection component R1 of night image, the illumination component I1 of night image, and the reflection component R h and the reference image illumination component I h ; Using the night image reflection component R1 and the reference image reflection component R h , enhance the reflection component through the enhancement network; use the night image illumination component I1 and the reference image illumination component I h , the illumination component is enhanced through the enhancement network; the enhanced reflection component and the illumination component are fused to obtain the enhanced image under the visible light source.

[0094] Embodiment 3: In addition to all the technical features of Embodiment 1, this embodiment also includes:

[0095] S5 specifically includes:

[0096] S51, construct a data set, collect infrared and visible light images of night scenes, perform preprocessing, take a part of them to construct a sample training set, and the other part to construct a test set;

[0097] S52. Build an infrared and visible light fusion network model;

[0098] S53, designing a loss function by combining the pixel loss function and the gradient loss function;

[0099] S54, using the data set created in S51 to perform multiple iterations of training on the infrared and visible light fusion network until convergence is achieved or the set maximum number of iterations is reached, and the performance of the trained neural network model is evaluated using the test set, and if the performance requirement is not met, the training is continued;

[0100] S55. The infrared light source image and the enhanced visible light source image are fused using the trained network model to obtain a final image for judging the driver's fatigue state.

[0101] Furthermore, the network model specifically includes: the input layer receives the infrared image and visible light image data to be processed, with a size of 256×256×3; feature extraction is performed through multiple convolutional layers, so that the feature map can effectively obtain the key information in the image; each convolutional layer is followed by a ReLU activation function to enhance the nonlinear feature representation; the fusion layer uses the SE module to weight the extracted features to improve the expression ability of important features; finally, the output layer generates a fused image of the same size as the input image through 1x1 convolution, and uses perceptual loss to optimize the model performance, so that the generated result is closer to the real image in visual quality.

[0102] Furthermore, the pixel loss function formula is as follows:

[0103]

[0104] Among them, y (i,j) is the true color value of the pixel at position (i, j) in the input image, It is the color value predicted by the pixel at the (i, j) position predicted by the fusion network; δ is a threshold parameter that determines the switching point where the loss function changes from quadratic loss to linear loss;

[0105] The gradient loss function formula is as follows:

[0106]

[0107] in, represents the gradient operator, which is used to measure the gradient difference between the output image and the true image;

[0108] The formula of the combined loss function is as follows:

[0109]

[0110] Among them, λ1 and λ2 are weight coefficients used to balance the impact of the two loss terms.

[0111] Furthermore, the division of the face area specifically includes:

[0112] S61, in the process of processing the face image, firstly divide the face area and fuse different image information to better define and identify the various components of the face;

[0113] S62. Extract key features from the divided face regions; these features include but are not limited to important facial landmarks such as eyes, nose, and mouth, as well as the overall contour and texture information of the face.

[0114] Furthermore, fatigue driving detection includes eye width ratio detection, specifically including:

[0115] Eye aspect ratio detection is performed using the EAR algorithm, which identifies the eye contour, calculates the distance between the positioning points, and derives the corresponding aspect ratio based on this;

[0116] The calculation formula is as follows:

[0117]

[0118] Among them, ‖p2-p6‖ and ‖p3-p5‖ represent the vertical distances between the upper eyelid and the lower eyelid.

[0119] ‖p1-p4‖ means calculating the horizontal width of the eye;

[0120] When the driver is awake, the EAR value will vary within a specific range; when the driver is fatigued, causing the eyes to close, the EAR value will drop to 0;

[0121] Under normal circumstances, a person's blinking time is about 0.01 to 0.02 seconds, and it takes about 1 to 3 frames to complete a blink; when fatigued, the blinking speed will slow down and the blinking frequency will increase. When the EAR value drops below a certain threshold quickly and then quickly rises above the threshold, it can be considered that a blink has occurred. First, calculate the aspect ratio (EAR value) of the left and right eyes of the driver in a normal state, and find their average value to obtain the final EAR value, which represents the eye width ratio of the driver in a normal state, excluding the influence of individual differences.

[0122] During the detection process, the presence of blinking behavior is determined by comparing the fluctuations of the EAR value in a short period of time; if the blinking frequency within 1 minute is significantly higher than normal, it is judged as fatigue driving.

[0123] The driver's fatigue state is detected in night images; if the detection result is fatigue, intelligent reminders are given, such as tightening the seat belt, voice prompts and vibrating seats.

[0124] Working principle: Use the decomposition network to decompose the night image S1 under visible light source and the corresponding normal light reference image S hDecomposed into reflection component and illumination component respectively, the reflection component R1 of night image, the illumination component I1 of night image, and the reflection component R h and the reference image illumination component I h ; Using the night image reflection component R1 and the reference image reflection component R h , enhance the reflection component through the enhancement network; use the night image illumination component I1 and the reference image illumination component I h , enhance the illumination component through the enhancement network; fuse the enhanced reflection component and illumination component to obtain the enhanced image under the visible light source; transform the infrared light source image S r The final image is fused with the enhanced image under visible light source to determine the driver's fatigue status.

[0125] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0126] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. An image enhancement method for nighttime fatigue driving detection, characterized in that: The image enhancement method comprises the following steps: S1, real-time image acquisition, using the vehicle-mounted camera to shoot the driver's video in real time and extract the driver's image from the video; S2, image quality judgment, judging the brightness of the image and dividing the image into nighttime image and normal light image; S3, normal light image processing, using the image denoising module to process the collected driver image to improve image quality and reduce interference, and using the image enhancement module to enhance the driver image to enhance the brightness, color and clarity of the image; S4, low-light image processing, firstly, low-light image enhancement based on Retinex theory is performed; a Retinex model is constructed and trained to decompose the image into a reflection component and an illumination component; the enhancement network model is trained using the night image reflection component, the night image illumination component, the reference image reflection component and the reference image illumination component, and the collected low-light image is input to obtain the enhanced image; S5. Construct an infrared and visible light fusion network model, use infrared image and normal light image data sets to train the fusion network, input the enhanced image and infrared light image into the fusion network to obtain the final image for driver fatigue detection; S6, face area segmentation, segmenting the fused face area and extracting key features; S7, fatigue identification, detects the fatigue status. If the detection result is fatigue, intelligent reminders are given, such as tightening the seat belt, voice prompts and vibrating seats.

2. The image enhancement method according to claim 1, characterized in that: In S2, when the average brightness of the image is higher than 0.45, the image is classified as a normal light image and conventional preprocessing of the normal light image is performed; when the average brightness of the image is lower than 0.45, the image is classified as a night image and low light image enhancement based on Retinex theory and image enhancement by fusion of infrared and visible light are performed.

3. The image enhancement method according to claim 1, characterized in that: In S4, the Retinex model is constructed and trained, and low-light image enhancement based on the Retinex theory is performed on the low-light image, specifically including: S41, constructing a data set, collecting night images under visible light sources and corresponding normal light reference images, dividing them into a training set and a test set in proportion, annotating the images, and generating corresponding enhanced targets; S42. Build a Retinex night image enhancement model based on U-Net structure; S43, using the data set created in S41 to perform multiple iterations of training on the decomposition network until convergence is achieved or the set maximum number of iterations is reached, and the performance of the trained neural network model is evaluated using the test set, and if the performance requirement is not met, the training is continued; S44, using the data set created in S41 to perform multiple iterations of training on the enhanced network until convergence is achieved or the set maximum number of iterations is reached, and the performance of the trained neural network model is evaluated using the test set, and if the performance requirement is not met, the training is continued; S45. Use the trained low-light image enhancement network to enhance the night image to be enhanced.

4. The image enhancement method according to claim 3, characterized in that: The Retinex night image enhancement model specifically includes: the input layer of the decomposition network receives the image data to be processed, with a size of 256×256×3; downsampling is performed through continuous convolutional layers and maximum pooling layers, so that the feature map can extract low-level and high-level features; the middle layer uses multiple convolutional blocks and ReLU activation functions to enhance nonlinear feature representation; the output layer generates decomposed image data to ensure the consistency of the output and input images in size, and uses the mean square error loss function to optimize the model performance; The input layer of the enhancement network receives the reflection component and illumination component from the decomposition network; the CBAM attention mechanism is added during downsampling to enhance the network's attention to important features, and the downsampling process gradually restores the image resolution through the transposed convolution layer, and uses jump connections to fuse the feature map in the encoder with the feature map in the decoder to ensure that the detail information is preserved; the output layer generates an enhanced image of the same size as the input image through 1x1 convolution, and uses a smoothness loss function to optimize the model performance, so that the generated result is closer to the real image in visual quality; The principle of the Retinex network model is as follows: I(x,y)=L(x,y)*R(x,y) Among them, I(x,y) represents the image information of the image observed or captured by the camera at the (x,y) coordinate, that is, the input of the network model, L(x,y) represents the illumination component of the light source, R(x,y) represents the reflection component of the target object in the image for the long, medium and short waves of the light, and * represents pixel multiplication; The loss function used by the decomposition network is the mean square error loss function, and the formula is as follows: Where N is the number of samples, which represents the total number of pixels in the image, and y (i,j) is the true color value of the pixel at position (i, j) of the input original image, It is the color value predicted by the pixel at the (i, j) position predicted by the decomposition network; The loss function used by the enhanced network is the smoothness loss function, and the formula is as follows: Among them, I(x i,j ) represents the pixel value of the image at position (x, y), |I(x i,j )-I(x i+1,j )| represents the absolute difference between adjacent pixels in the horizontal direction of the target image; |I(x i,j )-I(x i,j+1 )| represents the absolute difference between adjacent pixels in the vertical direction of the target image; The decomposition network is used to transform the night image S1 under visible light source and the corresponding normal light reference image S h Decomposed into reflection component and illumination component respectively, the reflection component R1 of night image, the illumination component I1 of night image, and the reflection component R h and the reference image illumination component I h ; Using the night image reflection component R1 and the reference image reflection component R h , enhance the reflection component through the enhancement network; use the night image illumination component I1 and the reference image illumination component I h , the illumination component is enhanced through the enhancement network; the enhanced reflection component and the illumination component are fused to obtain the enhanced image under the visible light source.

5. The image enhancement method according to claim 1, characterized in that: The S5 specifically includes: S51, construct a data set, collect infrared and visible light images of night scenes, perform preprocessing, take a part of them to construct a sample training set, and the other part to construct a test set; S52. Build an infrared and visible light fusion network model; S53, designing a loss function by combining the pixel loss function and the gradient loss function; S54, using the data set created in S51 to perform multiple iterations of training on the infrared and visible light fusion network until convergence is achieved or the set maximum number of iterations is reached, and the performance of the trained neural network model is evaluated using the test set, and if the performance requirement is not met, the training is continued; S55. The infrared light source image and the enhanced visible light source image are fused using the trained network model to obtain a final image for judging the driver's fatigue state.

6. The image enhancement method according to claim 5, characterized in that: The network model specifically includes: The input layer receives the infrared image and visible light image data to be processed, with a size of 256×256×3; Feature extraction is performed through multiple convolutional layers, so that the feature map can effectively obtain the key information in the image; Each convolutional layer is followed by a ReLU activation function to enhance nonlinear feature representation; The fusion layer uses the SE module to weight the extracted features to improve the expressiveness of important features; Finally, the output layer generates a fused image of the same size as the input image through 1x1 convolution, and uses perceptual loss to optimize the model performance so that the generated result is closer to the real image in visual quality.

7. The image enhancement method according to claim 5, characterized in that: The pixel loss function formula is as follows: Among them, y (i,j) is the true color value of the pixel at position (i, j) in the input image, It is the color value predicted by the pixel at the (i, j) position predicted by the fusion network; δ is a threshold parameter that determines the switching point where the loss function changes from quadratic loss to linear loss; The gradient loss function formula is as follows: in, represents the gradient operator, which is used to measure the gradient difference between the output image and the true image; The formula of the combined loss function is as follows: Among them, λ1 and λ2 are weight coefficients used to balance the impact of the two loss terms.

8. The image enhancement method according to claim 1, characterized in that: In S6, the division of the face area specifically includes: S61, in the process of processing the face image, firstly divide the face area and fuse different image information to better define and identify the various components of the face; S62. Extract key features from the divided face regions; these features include but are not limited to important facial landmarks such as eyes, nose, and mouth, as well as the overall contour and texture information of the face.

9. The image enhancement method according to claim 1, characterized in that: In S7, the fatigue driving detection includes eye width ratio detection, which specifically includes: Eye aspect ratio detection is performed using the EAR algorithm, which identifies the eye contour, calculates the distance between the positioning points, and derives the corresponding aspect ratio based on this; The calculation formula is as follows: Among them, ‖p2-p6‖ and ‖p3-p5‖ represent the vertical distance between the upper eyelid and the lower eyelid. ‖p1-p4‖ represents the horizontal width of the eye; When the driver is awake, the EAR value will vary within a specific range; when the driver is fatigued, causing the eyes to close, the EAR value will drop to 0; During the test, the fluctuation of EAR value in a short period of time is compared to determine whether there is blinking behavior; if the blinking frequency within 1 minute is significantly higher than normal, it is judged as fatigue driving; The driver's fatigue state is detected in night images; if the detection result is fatigue, intelligent reminders are given, such as tightening the seat belt, voice prompts and vibrating seats.

Images