A fatigue driving detection system based on face recognition

Abstract

The present invention discloses a fatigue driving detection system based on face recognition, which relates to the technical field of fatigue driving face recognition, and includes: an image acquisition device, which acquires driver's facial image data and determines the angle change between the driver and the image acquisition device during acquisition; an image processing device, which is used to process the facial image data, identify the shadow area and the illumination area in the facial image, and determine the light source position according to the change between the shadow area and the illumination area; a facial model, which is based on the change between the shadow area and the illumination area in the multi-angle facial image data and the light source position, and establishes the facial model based on the fluctuation degree data. The present application acquires facial images, determines the direction of illumination, establishes the light source position according to the different illumination directions, establishes the facial model according to the influence of illumination on the face, and identifies the facial muscle changes and the relaxation degree of the face according to the state of the facial model.

Description

Technical Field

[0001] The present invention relates to the technical field of fatigue driving face recognition, and in particular to a fatigue driving detection system based on face recognition. Background Art

[0002] Fatigue driving detection technology mainly determines whether the driver is in a fatigued state by analyzing the driver's facial features and behavior. The following are some commonly used methods and technologies:

[0003] Based on visual sensors: Face detection: Monitor the driver's facial image in real time through the camera, detect and locate the face. Eye state recognition: Use algorithms (such as YOLO, CNN, etc.) to detect the open and closed state of the eyes, and calculate the eye aspect ratio (EAR) to determine whether the eyes are closed. Facial expression analysis: Identify fatigue expressions such as yawning and heavy eyelids by analyzing the position and changes of facial feature points.

[0004] Based on biosensors: Heart rate monitoring: Monitor the driver's heart rate changes through wearable devices to determine their fatigue level. Brain wave detection: Use brain wave sensors to monitor the driver's brain wave activity and identify fatigue status.

[0005] Multi-feature fusion: Comprehensive analysis: Combining multiple features such as eye state, facial expression, head posture, etc., using machine learning algorithms to comprehensively judge the driver's fatigue state.

[0006] Based on vehicle information: Driving behavior analysis: By monitoring the vehicle's speed, steering wheel operation and other information, the driver's fatigue state can be indirectly judged.

[0007] Announcement No. CN117392647A discloses a fatigue driving detection method, detection device and storage medium, including the following steps: obtaining real-time face data, and processing the image of the real-time face data to obtain an eye image; inputting the eye image into an eye opening and closing detector to detect the closed eye state and record the time when the closed eye state occurs; judging whether the number of closed eye states exceeds a preset value within a preset time period, and if so, judging it as fatigue driving. The present invention uses the dlib face detection algorithm to obtain the driver's face data in real time, compares the driver's facial features with the 68 point features of face recognition, and uses the training samples of the neural network for comparison to implement the judgment of whether the driver is fatigue driving, realize efficient face detection, and quickly and accurately detect the driver's face and the phenomenon of fatigue driving.

[0008] However, since drivers usually wear obstructions such as sunglasses while driving, the test results may be biased during the fatigue driving test. Also, since drivers are affected by the lighting and other environmental factors while driving, their eyes may change accordingly, which may lead to deviations in the accuracy of fatigue driving detection and affect the final test results. Summary of the invention

[0009] One of the purposes of the present invention is to provide a fatigue driving detection system based on face recognition. This application establishes a face model and performs recognition according to the face environment, thereby reducing the interference of wearing sunglasses and lighting environment on the detection results, and further improving the accuracy of the detection. The face model can further identify the facial muscles of the face, and even when wearing sunglasses or the like to block the eye position, good fatigue driving detection can still be performed.

[0010] To achieve the above objectives, the present invention is implemented through the following technical solutions: A fatigue driving detection system based on face recognition, comprising:

[0011] An image acquisition device, which acquires facial image data of the driver and determines the change in angle between the driver and the image acquisition device during acquisition;

[0012] An image processing device is used to process facial image data, identify shadow areas and illuminated areas in the facial image, and determine the position of the light source according to the change between the shadow areas and the illuminated areas;

[0013] A face model, based on the changes between shadow areas and illuminated areas in the multi-angle facial image data and the position of the light source, determines the fluctuation degree data of the driver's face and establishes a face model based on the fluctuation degree data, the face model reflects the facial fluctuation characteristics of the face;

[0014] A relaxation analysis system, based on the changes between shadow areas and illuminated areas in the multi-angle facial image data, analyzes the facial fluctuation state in the facial image data, determines the relaxation degree of the driver's face according to the fluctuation state, and determines the driver's driving state according to the duration of the driver's facial relaxation degree and the change in relaxation degree;

[0015] The eye analysis system segments eye features based on multi-angle facial image data, determines the impact of light sources on eye feature data, and determines whether the driver's eye features meet the characteristics of fatigue driving during the facial data acquisition process of the image acquisition device.

[0016] In one or more embodiments of the present invention, the image acquisition device obtains the distance between the image acquisition device and the driver, determines the acquisition angle according to the distance between the image acquisition device and the driver, and performs angle marking on the multi-angle facial image data to determine the undulation data of the face. The facial image data acquired by the image acquisition device is divided into recognizable data and defect data. The recognizable data has a complete facial image, and the defect data is the noise of the facial image data.

[0017] In one or more embodiments of the present invention, the steps for facial image processing are as follows:

[0018] Segmenting facial features in facial image data to determine illuminated areas and shadowed areas in the facial features;

[0019] Determine the irradiation direction of the light source according to the changes of the shadow area and the illuminated area in the multi-angle facial image data;

[0020] Arrange the multi-angle facial image data based on the shooting time, construct the change range of the shadow area and the light area at different times, and calculate the change time of the change range of the shadow area and the light area;

[0021] Obtain the shooting angle of the facial image data, and mark the shooting angles corresponding to different shadow areas and light areas;

[0022] Determine the position of the light source based on the coverage of the shadow area and the lit area.

[0023] In one or more embodiments of the present invention, the shadow area and the illuminated area are determined and marked by analyzing the brightness and contrast of the facial image data, and the change values ​​of the shadow area and the illuminated area are determined by an edge detection algorithm. The illumination position of the light source is calculated based on the change values ​​of the shadow area and the illuminated area and the change in the facial image data acquisition angle.

[0024] In one or more embodiments of the present invention, the first facial image data is determined as the initial facial image data, the image acquisition device acquisition point is X, the driver position is Y, the light source position is B, and a coordinate system is constructed with the image acquisition device acquisition point as the origin, and the distance d between the image acquisition device acquisition point X and the driver position Y is calculated. XY :

[0025]

[0026] Wherein, (x1, y1) is the X coordinate point, (x2, y2) is the Y coordinate point, and the distance between the edge of the shadow area in the initial facial image data and the facial feature point a1 is obtained;

[0027] The facial image data are marked as the second facial image data, the third facial image data...the nth facial image data in sequence according to the shooting time, and the distance between the acquisition point of the image acquisition device and the driver's position and the distance between the edge of the shadow area and the facial feature point a1 are calculated respectively. The light source position is calculated according to the different change values ​​in the multiple facial image data.

[0028] In one or more embodiments of the present invention, the steps of calculating the light source position are as follows:

[0029] The light source direction angle Bv is calculated by the shadow area change value and the distance dXY between the image acquisition device acquisition point X and the driver's position Y:

[0030]

[0031] Wherein, Δθ is the change value of the two facial image data;

[0032] Calculate the light source position Bc based on the light source direction angle and known distance:

[0033] B c =(d XY ·cos(θ), d XY sin(θ), h);

[0034] Where h is the height of the light source.

[0035] In one or more embodiments of the present invention, the light source position is brought into the facial image data at different angles, the facial relief data corresponding to the facial shadow area under different light source illumination angles is measured, and a face model is constructed based on the facial relief data. The facial relief data H is:

[0036] H = l·tan(α);

[0037] Among them, l is the shadow length and α is the illumination angle of the light source.

[0038] In one or more embodiments of the present invention, according to the changes in the ups and downs of the face in the multi-angle facial image data, the state of the face model is adjusted, so that the face model changes according to the ups and downs of the face, thereby analyzing the muscle state of the face model, and determining the driving state of the driver according to the changes in the muscle state of the face model;

[0039] Based on the facial angle of the defective data, the angle and duration of the facial model are adjusted, and the driving status corresponding to the facial model angle is analyzed.

[0040] In one or more embodiments of the present invention, the brightness and contrast of the shadow area and the illuminated area in the facial image data are analyzed to determine the contrast between the shadow area and the illuminated area, the light intensity of the light source is analyzed, and different light intensity range values ​​are set corresponding to different environmental conditions, and whether the light source has an impact on the eye features is determined according to the light intensity range value;

[0041] Obtain eye features in facial image data, determine the opening and closing state of the eye features, calculate the aspect ratio of the eyes, and analyze the state corresponding to the eye features.

[0042] In one or more embodiments of the present invention, the steps of analyzing the degree of facial relaxation of the driver are as follows:

[0043] Analyze facial undulations: Calculate facial undulations by analyzing feature points in the facial model (such as eye corners, mouth corners, nose tip, etc.); Curvature analysis: Calculate the curvature of different facial regions and identify high curvature regions (such as wrinkles, depressions, etc.); Height map: Generate a facial height map to show height changes in different facial regions;

[0044] Assess muscle relaxation: Assess muscle relaxation based on facial fluctuations

[0045] In one or more embodiments of the present invention, the above.

[0046] Through the above technical solution, the present invention has the following beneficial effects:

[0047] 1. This application collects facial images, determines the direction of lighting, establishes the position of the light source according to the different lighting directions, establishes a facial model through the impact of lighting on the face, and identifies facial muscle changes and the degree of relaxation according to the state of the facial model to determine whether the driver is in a state of fatigue driving, thereby further improving recognition accuracy.

[0048] 2. Due to the different positions of lighting directions, the degree of light and shadow reflected on the human face is different. Therefore, the position and direction of the light source can be determined by the changes in the shadow and lighting positions, thereby determining the specific data of the light source. The intensity of the light source has different effects on the driver's eyes. Therefore, by weakening the light source, the impact on the driver can further improve the accuracy of fatigue driving detection.

[0049] 3. By depicting the shadows of facial expressions through lighting, the ups and downs of the face can be determined. Based on the facial ups and downs data, a face model is established. By analyzing the muscle changes of the face model, the degree of relaxation of the driver's muscles can be determined. Based on the changes in the degree of relaxation, it can be determined whether the driver is in a state of fatigue driving.

[0050] 4. By comparing multiple groups of images, the driver's facial features and eye features are further analyzed, and whether the driver is in a state of fatigue driving is determined based on the changes in the features. Due to the limitations of the analysis of the driver's eye and facial features, the accuracy of fatigue driving detection is further guaranteed by combining facial features with eye features and analyzing fatigue driving from multiple angles. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a schematic diagram of the detection system of the present invention;

[0052] Figure 2 It is a schematic diagram of the change of the collection angle between the collection point X and the driver position Y at different positions of the present invention. DETAILED DESCRIPTION

[0053] The following will disclose multiple embodiments of the present invention with the accompanying drawings. For the purpose of clear description, many practical details will be described together in the following description. However, it should be understood that these practical details should not be used to limit the present invention. In other words, in some embodiments of the present invention, these practical details are not necessary. And if it is possible to implement, the features of different embodiments can be applied interchangeably.

[0054] Unless otherwise defined, all words (including technical and scientific terms) used herein have their usual meanings, which can be understood by those familiar with this field. Furthermore, the definitions of the above words in commonly used dictionaries should be interpreted as the meanings consistent with the relevant fields of the present invention in the content of this specification. Unless otherwise clearly defined, these words will not be interpreted as idealized or overly formal meanings.

[0055] See also Figure 1-Figure 2 The present invention provides a fatigue driving detection system based on face recognition, which recognizes the facial and eye features of the driver, establishes a face model based on the light irradiation angle, determines the driver's muscle relaxation degree through the fluctuation state of the face model, and determines whether the driver is in a fatigue driving state according to the muscle relaxation degree and eye features.

[0056] In one embodiment, the fatigue driving detection system includes:

[0057] An image acquisition device, which acquires facial image data of the driver and determines the change in angle between the driver and the image acquisition device during acquisition;

[0058] An image processing device is used to process facial image data, identify shadow areas and illuminated areas in the facial image, and determine the position of the light source according to the change between the shadow areas and the illuminated areas;

[0059] A face model, based on the changes between shadow areas and illuminated areas in the multi-angle facial image data and the position of the light source, determines the fluctuation degree data of the driver's face and establishes a face model based on the fluctuation degree data, the face model reflects the facial fluctuation characteristics of the face;

[0060] A relaxation analysis system, based on the changes between shadow areas and illuminated areas in the multi-angle facial image data, analyzes the facial fluctuation state in the facial image data, determines the relaxation degree of the driver's face according to the fluctuation state, and determines the driver's driving state according to the duration of the driver's facial relaxation degree and the change in relaxation degree;

[0061] The eye analysis system segments eye features based on multi-angle facial image data, determines the impact of light sources on eye feature data, and determines whether the driver's eye features meet the characteristics of fatigue driving during the facial data acquisition process of the image acquisition device.

[0062] In this embodiment, the driver's driving state is determined by analyzing the driver's facial features and eye features, so that it is possible to analyze whether the driver is in a fatigued driving state. In order to ensure the accuracy of the analysis, the impact of light on the driver is considered, and a face model of the driver is established based on the light state. Multi-angle facial image data is matched with the face model to determine the driver's muscle relaxation state, and the driver's specific state is analyzed in combination with the driver's eye features, thereby further improving the accuracy of facial recognition.

[0063] However, under strong light conditions, the driver's eye features will change, such as squinting or using sunglasses to cover the eyes. The impact of light at different angles on the shadow area and light area of ​​the driver's facial image data can determine the driver's facial undulation data and the corresponding facial muscle state. Therefore, under the influence of light, the accuracy of fatigue driving detection can also be guaranteed.

[0064] In one embodiment, the image acquisition device obtains the distance between the image acquisition device and the driver, determines the acquisition angle based on the distance between the image acquisition device and the driver, and performs angle marking on multi-angle facial image data to determine the ups and downs data of the face. The facial image data acquired by the image acquisition device is divided into recognizable data and defect data. The recognizable data has a complete facial image, and the defect data is the noise of the facial image data.

[0065] In this embodiment, since the facial image data collected by the image acquisition device are at different times and the driver's posture and viewing angle in the cab are also different, part of the facial image data can be used for recognition processing, while other part of the facial image data cannot be used for recognition processing. By classifying the data, useful data can be quickly retrieved, further improving the data processing speed.

[0066] Since the recognition of face models and eye features requires facial data, there are no reference conditions for defective data.

[0067] In one embodiment, the steps for facial image processing are as follows:

[0068] Segmenting facial features in facial image data to determine illuminated areas and shadowed areas in the facial features;

[0069] Determine the irradiation direction of the light source according to the changes of the shadow area and the illuminated area in the multi-angle facial image data;

[0070] Arrange the multi-angle facial image data based on the shooting time, construct the change range of the shadow area and the light area at different times, and calculate the change time of the change range of the shadow area and the light area;

[0071] Obtain the shooting angle of the facial image data, and mark the shooting angles corresponding to different shadow areas and light areas;

[0072] Determine the position of the light source based on the coverage of the shadow area and the lit area.

[0073] In this embodiment, the speed of the driver will change while he is driving the vehicle. Therefore, by marking the time and shooting angle of the multi-angle facial image data, the driver's route and travel process can be analyzed. The angle of lighting will cause changes in facial shadows and lighting areas. Therefore, the shadow areas and lighting areas in the driver's facial image data will differ due to the different positions of the light source.

[0074] Multi-angle facial image data is used to determine the position of the light source, and the driver's facial undulations are determined based on the position of the light source and the different shooting angles. The driver's facial undulations correspond to the driver's facial features. Therefore, a face model can be constructed through the driver's facial undulations data.

[0075] In one embodiment, the shadow area and the illuminated area are determined and marked by analyzing the brightness and contrast of the facial image data, and the change values ​​of the shadow area and the illuminated area are determined by an edge detection algorithm. According to the change values ​​of the shadow area and the illuminated area and the change of the facial image data acquisition angle, the illumination position of the light source is calculated.

[0076] In this embodiment, an edge detection algorithm is used to determine the change value of the shadow area and the illuminated area. The calculation is performed by marking the facial feature points and the distances between the shadow area and the illuminated area and the facial feature points. The facial feature point a1 is marked, and the distance between the edge of the shadow area or the edge of the illuminated area and a1 is obtained. The change value is determined based on the change in the distance between the edge of the shadow area or the edge of the illuminated area and a1 in the two-angle facial image data.

[0077] Exemplarily, the shooting angle of the first facial image data is 30 degrees, the distance between the edge of the shadow area and the facial feature point a1 is 1 cm, and the shooting angle of the second facial image data is 45 degrees, the distance between the edge of the shadow area and the facial feature point a1 is 1.5 centimeters, and the change value is equal to (1.5-1) / (45-30)≈0.033.

[0078] In one embodiment, the first facial image data is determined as the initial facial image data, the image acquisition device acquisition point is X, the driver position is Y, the light source position is B, and a coordinate system is constructed with the image acquisition device acquisition point as the origin, and the distance d between the image acquisition device acquisition point X and the driver position Y is calculated. XY :

[0079]

[0080] Wherein, (x1, y1) is the X coordinate point, (x2, y2) is the Y coordinate point, and the distance between the edge of the shadow area in the initial facial image data and the facial feature point a1 is obtained;

[0081] The facial image data are marked as the second facial image data, the third facial image data...the nth facial image data in sequence according to the shooting time, and the distance between the acquisition point of the image acquisition device and the driver's position and the distance between the edge of the shadow area and the facial feature point a1 are calculated respectively. The light source position is calculated according to the different change values ​​in the multiple facial image data.

[0082] In this embodiment, since the acquisition angle of the image acquisition device is fixed, the distance between the acquisition point and the driver's position can be calculated to determine the distance between the driver and the image acquisition device, as well as the change in position on the road. This can avoid the driver's left and right deviation during road travel, which affects image recognition and further improves the accuracy of fatigue driving identification.

[0083] Among them, the change values ​​between every two adjacent facial image data are different, representing the changes in the light source position and facial undulation features. By combining multiple facial image data, facial undulation data can be further accurately acquired.

[0084] In one embodiment, the steps of calculating the light source position are as follows:

[0085] The light source direction angle Bv is calculated by the shadow area change value and the distance dXY between the image acquisition device acquisition point X and the driver's position Y:

[0086]

[0087] Wherein, Δθ is the change value of the two facial image data;

[0088] Calculate the light source position Bc based on the light source direction angle and known distance:

[0089] B c =(d XY ·cos(θ), d XY sin(θ), h);

[0090] Where h is the height of the light source.

[0091] In this embodiment, the angle of the light source direction is calculated first, and then the position of the light source is calculated again based on the angle of the light source direction to determine the position of the light source. By determining the position of the light source, it is possible to further analyze whether the light source will have an impact on the driver, and to be more accurate in eye feature recognition.

[0092] Determine the height h of the light source by the shadow length method:

[0093] Measure the length L of the driver's facial shadow,

[0094] Using the principle of similar triangles, calculate the height h of the light source:

[0095] h = L·tan(θ).

[0096] In one embodiment, the light source position is brought into the facial image data at different angles, the facial relief data corresponding to the facial shadow area under different light source illumination angles is measured, and a face model is constructed based on the facial relief data. The facial relief data H is:

[0097] H = l·tan(α);

[0098] Among them, l is the shadow length and α is the illumination angle of the light source.

[0099] By calculating the facial undulation data according to the position of the light source, it is possible to determine the undulation of the driver's face, such as the height of the driver's nose bridge, the muscle condition of the driver's cheeks, and the undulation of the driver's mouth and chin, thereby obtaining the driver's three-dimensional facial data for constructing a face model.

[0100] In one embodiment, according to the changes in the ups and downs of the face in the multi-angle facial image data, the state of the face model is adjusted so that the face model changes according to the ups and downs of the face, thereby analyzing the muscle state of the face model, and determining the driving state of the driver according to the changes in the muscle state of the face model;

[0101] Based on the facial angle of the defective data, the angle and duration of the facial model are adjusted, and the driving status corresponding to the facial model angle is analyzed.

[0102] In this embodiment, since the defect data is facial image data that does not contain complete facial data, it usually means the adjustment of the driver's head position. During driving, the change in the angle position of the driver's head can also correspond to the detection of the driver's fatigue driving state. Therefore, after establishing the face model, the face model is adjusted according to the facial image data at different angles and the defect data to simulate the driver's head state and facial state during the image acquisition process, identify the driver's driving state during driving, and analyze the facial fluctuations through the relaxation analysis system to determine the degree of relaxation of the driver's face.

[0103] In one embodiment, the brightness and contrast of the shadow area and the illuminated area in the facial image data are analyzed to determine the contrast between the shadow area and the illuminated area, the light intensity of the light source is analyzed, and different light intensity range values ​​are set corresponding to different environmental conditions, and whether the light source has an impact on the eye features is determined according to the light intensity range value;

[0104] Obtain eye features in facial image data, determine the opening and closing state of the eye features, calculate the aspect ratio of the eyes, and analyze the state corresponding to the eye features.

[0105] In this embodiment, the eye features corresponding to normal driving and fatigue driving are different:

[0106] Normal driving: Eyes are usually open, blinking frequency is normal (about 15-20 times per minute), and each blink lasts for a short time (about 100-400 milliseconds);

[0107] Fatigue driving: The eyes may close frequently, the blinking frequency increases, and the eye closure time is prolonged. When driving while fatigued, the aspect ratio of the eyes (EAR) will be significantly reduced.

[0108] In the prior art, fatigue driving or normal driving is judged by eye feature recognition. Therefore, by identifying the aspect ratio of the eye features, the driving state of the driver can be detected. By combining eye features with facial features, the accuracy of fatigue driving recognition can be further improved.

[0109] In one embodiment, the steps of analyzing the degree of facial relaxation of the driver are as follows:

[0110] Analyze facial undulations: Calculate facial undulations by analyzing feature points in the facial model (such as eye corners, mouth corners, nose tip, etc.); Curvature analysis: Calculate the curvature of different facial regions and identify high curvature regions (such as wrinkles, depressions, etc.); Height map: Generate a facial height map to show height changes in different facial regions;

[0111] Assess the degree of muscle relaxation: Assess the degree of muscle relaxation based on the changes in facial movements.

[0112] In this embodiment, relaxed muscles generally exhibit the following characteristics:

[0113] Increased wrinkles: Increased number and depth of facial wrinkles;

[0114] Skin sagging: Facial skin droops, especially in areas such as eye bags and nasolabial folds;

[0115] Decreased muscle tone: The tension in facial muscles decreases, resulting in a relaxed facial expression.

[0116] Exemplarily, the facial curvature is calculated: Curvature is a quantity that describes the degree of curvature of a curve or surface. For the facial curvature, the following method can be used:

[0117] Gaussian curvature and mean curvature

[0118] Gaussian curvature: Gaussian curvature K is the intrinsic curvature of the surface at a point, defined as the product of the two principal curvatures k1 and k2 at that point:

[0119] K=k1·k2.

[0120] Among them, the driver's driving status is identified according to the changes in the degree of relaxation of facial muscles, which can determine whether the driver is in a fatigue driving state. Combined with the recognition of eye features, the analysis of driver fatigue driving is more accurate, avoiding the misjudgment of fatigue driving caused by analyzing eye features under light source conditions.

[0121] In summary, the technical solutions disclosed in the above embodiments of the present invention have at least the following advantages:

[0122] 1. This application collects facial images, determines the direction of lighting, establishes the position of the light source according to the different lighting directions, establishes a facial model through the impact of lighting on the face, and identifies facial muscle changes and the degree of relaxation according to the state of the facial model to determine whether the driver is in a state of fatigue driving, thereby further improving recognition accuracy.

[0123] 2. Due to the different positions of lighting directions, the degree of light and shadow reflected on the human face is different. Therefore, the position and direction of the light source can be determined by the changes in the shadow and lighting positions, thereby determining the specific data of the light source. The intensity of the light source has different effects on the driver's eyes. Therefore, by weakening the light source, the impact on the driver can further improve the accuracy of fatigue driving detection.

[0124] 3. By depicting the shadows of facial expressions through lighting, the ups and downs of the face can be determined. Based on the facial ups and downs data, a face model is established. By analyzing the muscle changes of the face model, the degree of relaxation of the driver's muscles can be determined. Based on the changes in the degree of relaxation, it can be determined whether the driver is in a state of fatigue driving.

[0125] 4. By comparing multiple groups of images, the driver's facial features and eye features are further analyzed, and whether the driver is in a state of fatigue driving is determined based on the changes in the features. Due to the limitations of the analysis of the driver's eye and facial features, the accuracy of fatigue driving detection is further guaranteed by combining facial features with eye features and analyzing fatigue driving from multiple angles.

[0126] Although the present invention is disclosed in conjunction with the above embodiments, it is not intended to limit the present invention. Anyone familiar with this technology can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the definition of the attached claims.

Claims

1. A fatigue driving detection system based on face recognition, characterized in that: include: An image acquisition device, which acquires facial image data of the driver and determines the change in angle between the driver and the image acquisition device during acquisition; An image processing device is used to process facial image data, identify shadow areas and illuminated areas in the facial image, and determine the position of the light source according to the change between the shadow areas and the illuminated areas; A face model, based on the changes between shadow areas and illuminated areas in the multi-angle facial image data and the position of the light source, determines the fluctuation degree data of the driver's face and establishes a face model based on the fluctuation degree data, the face model reflects the facial fluctuation characteristics of the face; A relaxation analysis system, based on the changes between shadow areas and illuminated areas in the multi-angle facial image data, analyzes the facial fluctuation state in the facial image data, determines the relaxation degree of the driver's face according to the fluctuation state, and determines the driver's driving state according to the duration of the driver's facial relaxation degree and the change in relaxation degree; The eye analysis system segments eye features based on multi-angle facial image data, determines the impact of light sources on eye feature data, and determines whether the driver's eye features meet the characteristics of fatigue driving during the facial data acquisition process of the image acquisition device.

2. The fatigue driving detection system based on face recognition according to claim 1 is characterized in that: The image acquisition device obtains the distance between the image acquisition device and the driver, and determines the acquisition angle based on the distance between the image acquisition device and the driver. The multi-angle facial image data should be angle-marked to determine the ups and downs data of the face. The facial image data collected by the image acquisition device is divided into recognizable data and defect data. The recognizable data has a complete facial image, and the defect data is the noise of the facial image data.

3. The fatigue driving detection system based on face recognition according to claim 2 is characterized in that: The steps for facial image processing are as follows: Segmenting facial features in facial image data to determine illuminated areas and shadowed areas in the facial features; Determine the irradiation direction of the light source according to the changes of the shadow area and the illuminated area in the multi-angle facial image data; Arrange the multi-angle facial image data based on the shooting time, construct the change range of the shadow area and the light area at different times, and calculate the change time of the change range of the shadow area and the light area; Obtain the shooting angle of the facial image data, and mark the shooting angles corresponding to different shadow areas and light areas; Determine the position of the light source based on the coverage of the shadow area and the lit area.

4. The fatigue driving detection system based on face recognition according to claim 3 is characterized in that: By analyzing the brightness and contrast of facial image data, the shadow area and the light area are determined and marked. The change values ​​of the shadow area and the light area are determined by the edge detection algorithm. According to the change values ​​of the shadow area and the light area and the change of the facial image data acquisition angle, the irradiation position of the light source is calculated.

5. The fatigue driving detection system based on face recognition according to claim 4 is characterized in that: Determine the first facial image data as the initial facial image data, the image acquisition device acquisition point is X, the driver position is Y, the light source position is B, and the image acquisition device acquisition point is used as the origin to construct a coordinate system, and calculate the distance d between the image acquisition device acquisition point X and the driver position Y XY : Wherein, (x1, y1) is the X coordinate point, (x2, y2) is the Y coordinate point, and the distance between the edge of the shadow area in the initial facial image data and the facial feature point a1 is obtained; The facial image data are marked as the second facial image data, the third facial image data...the nth facial image data in sequence according to the shooting time, and the distance between the acquisition point of the image acquisition device and the driver's position and the distance between the edge of the shadow area and the facial feature point a1 are calculated respectively. The light source position is calculated according to the different change values ​​in the multiple facial image data.

6. The fatigue driving detection system based on face recognition according to claim 5 is characterized in that: The steps to calculate the light source position are as follows: The change value of the shadow area and the distance d between the image acquisition device acquisition point X and the driver's position Y XY Calculate the light source direction angle Bv: Wherein, Δθ is the change value of the two facial image data; Calculate the light source position Bc based on the light source direction angle and known distance: B c =(d XY ·cos(θ),d XY ·sin(θ),h); Where h is the height of the light source.

7. The fatigue driving detection system based on face recognition according to claim 6 is characterized in that: The light source position is introduced into the facial image data at different angles, and the facial relief data corresponding to the facial shadow area under different light source illumination angles is measured. The face model is constructed based on the facial relief data. The facial relief data H is: H = l·tan(α); Among them, l is the shadow length and α is the illumination angle of the light source.

8. The fatigue driving detection system based on face recognition according to claim 7 is characterized in that: According to the changes in the facial ups and downs in the multi-angle facial image data, the state of the facial model is adjusted so that the facial model changes according to the facial ups and downs, thereby analyzing the muscle state of the facial model and determining the driving state of the driver according to the changes in the muscle state of the facial model; Based on the facial angle of the defective data, the angle and duration of the facial model are adjusted, and the driving status corresponding to the facial model angle is analyzed.

9. The fatigue driving detection system based on face recognition according to claim 8 is characterized in that: Analyze the brightness and contrast of shadow areas and illuminated areas in facial image data, determine the contrast between shadow areas and illuminated areas, analyze the light intensity of the light source, and set different light intensity range values ​​corresponding to different environmental conditions, and determine whether the light source has an impact on eye features based on the light intensity range values; Obtain eye features in facial image data, determine the opening and closing state of the eye features, calculate the aspect ratio of the eyes, and analyze the state corresponding to the eye features.

10. The fatigue driving detection system based on face recognition according to claim 9, characterized in that: The steps for analyzing the degree of facial relaxation of drivers are as follows: By analyzing the feature points in the facial model, the facial fluctuations are calculated: curvature analysis, calculating the curvature of different facial areas and identifying high curvature areas; height map, generating a facial height map to show the height changes of different facial areas; Assess the degree of muscle relaxation: Assess the degree of muscle relaxation based on the changes in facial movements.

Images