A fatigue driving terminal recognition processing method and system based on artificial intelligence

Abstract

The application relates to a kind of fusion artificial intelligence fatigue driving terminal identification processing method and system, it is related to vehicle-mounted artificial intelligence vision detection technical field, it includes obtaining the face image of driver;Get fatigue related area;The image corresponding to fatigue related area is defined as fatigue related image;Fatigue related image is input into the fatigue identification model of pre-setting and fatigue characteristics are obtained;Standard fatigue characteristics are obtained by normalizing fatigue characteristics;Weighted fusion is calculated to comprehensive fatigue index by standard fatigue characteristics;Comprehensive fatigue index and pre-set threshold are segmented and matched to obtain the threshold segment where comprehensive fatigue index falls into;According to the threshold segment where comprehensive fatigue index falls into, corresponding fatigue level is found in pre-set threshold level library;According to fatigue level, corresponding prompting scheme is found in pre-set level scheme library, and prompting scheme is executed to driver.The application has the effect of improving the accuracy of fatigue driving identification.

Description

Technical Field

[0001] This invention relates to the field of vehicle-mounted artificial intelligence visual inspection technology, and in particular to a fatigue driving terminal recognition and processing method and system that integrates artificial intelligence. Background Technology

[0002] With the increasing frequency of road traffic, fatigued driving has become a major cause of traffic accidents. It can easily lead to vehicle control errors and decreased reaction speed, seriously threatening driving safety. Therefore, real-time and accurate monitoring of driver fatigue is a key link in ensuring driving safety.

[0003] At present, the fatigue driving monitoring solutions on the market and in existing technologies are mainly based on single visual monitoring. The mainstream solution is to collect the driver's facial images through an in-vehicle camera and extract only eye-related features (such as eyelid opening and blinking frequency) as the basis for fatigue judgment, relying solely on a single eye feature to identify fatigue state.

[0004] Regarding the aforementioned technologies, in complex scenarios such as when the driver's eyes are obstructed or there are sudden changes in lighting, eye features are prone to extraction failure. Furthermore, relying solely on a single feature cannot achieve cross-validation of multi-dimensional features to improve judgment accuracy, leaving room for improvement. Summary of the Invention

[0005] To improve the accuracy of fatigue driving identification, this invention provides a fatigue driving terminal identification and processing method and system that integrates artificial intelligence.

[0006] In a first aspect, the present invention provides a fatigue driving terminal recognition and processing method integrating artificial intelligence, which adopts the following technical solution: A fatigue driving terminal recognition and processing method integrating artificial intelligence includes: Step 1: Obtain the driver's facial image; Step 2: Denoise the facial image and extract the ROI to obtain fatigue-related regions, which include the facial region, eye region, and mouth region; Step 3: Define the images corresponding to fatigue-related regions as fatigue-related images; Step 4: Input fatigue-related images into a preset fatigue recognition model to obtain fatigue features, including eyelid opening and closing degree, blinking frequency, eye closing duration, yawning frequency, head posture, and gaze direction; Step 5: Normalize the fatigue characteristics to obtain standard fatigue characteristics; Step 6: Calculate the comprehensive fatigue index by weighted fusion of standard fatigue characteristics; Step 7: Match the comprehensive fatigue index with the preset threshold segments to obtain the threshold segments into which the comprehensive fatigue index falls; Step 8: Based on the threshold segment into which the comprehensive fatigue index falls, find the corresponding fatigue level in the preset threshold level library; Step 9: Based on the fatigue level, find the corresponding reminder plan in the preset level plan library and execute the reminder plan for the driver.

[0007] Optionally, based on the fatigue level, a corresponding reminder plan can be retrieved from a preset level plan library, and the method for executing the reminder plan on the driver includes: Step 90: Based on the reminder scheme, find the corresponding reminder flow rate and reminder duration; Step 91: Determine the driver's current facial orientation based on head posture; Step 92: Determine the cheek region based on the current facial orientation, facial area, and preset cheek geometry; Step 93: Calculate the vertical height difference based on the cheek area and the preset reminder device position; Step 94: If the vertical height difference falls within the preset normal height range, plan the reminder orientation of the reminder device pointing towards the cheek area based on the cheek area and the position of the reminder device; Step 95: If the reminder direction falls within the preset rotatable range, control the reminder device to rotate to the reminder direction, and execute the reminder plan for the driver according to the reminder flow rate and reminder duration.

[0008] Optionally, it also includes a handling method if the vertical height difference falls within a preset normal height range and the orientation does not fall within a preset rotatable range, the method including: Step 950: Extract the upper and lower orientations from the rotatable range; Step 951: If the reminder direction is greater than the upper limit direction, the upper limit direction shall be used as the limit direction of the reminder direction; Step 952: If the reminder direction is less than the lower limit direction, the lower limit direction will be used as the limit direction of the reminder direction; Step 953: Calculate the orientation deviation based on the suggested orientation and the corresponding extreme orientation; Step 954: Calculate the blowing direction and blowing speed of the blower based on the orientation deviation; Step 955: Control the blowing device to blow air according to the blowing direction and speed, and at the same time control the reminder device to rotate to the extreme orientation, and execute the reminder plan for the driver according to the reminder flow rate and reminder duration.

[0009] Optionally, it also includes a handling method if the vertical height difference does not fall within a preset normal height range and the warning direction falls within a preset rotatable range, the method including: Step 940: If the vertical height difference is greater than 0, then the preset vertical upward direction will be used as the vertical blowing direction. Step 941: If the vertical height difference is less than 0, then the preset vertical downward direction will be used as the vertical blowing direction. Step 942: Find the vertical wind speed in the preset height wind speed library based on the absolute value of the vertical height difference; Step 943: Control the blower to blow air in a vertical direction and at a vertical air speed, while simultaneously controlling the reminder device to rotate to the reminder orientation; Step 944: Implement the reminder plan for the driver according to the reminder flow rate and reminder duration.

[0010] Optionally, it also includes a processing method if the vertical height difference does not fall within a preset normal height range and the orientation warning does not fall within a preset rotatable range, the method including: Step 945: Based on the indicated orientation and rotatable range, perform steps 950 to 954 to calculate the horizontal blowing direction and horizontal blowing speed of the blower; Step 946: Calculate the vertical blowing direction and vertical blowing speed of the blowing device based on the vertical height difference by performing steps 940 to 942; Step 947: Combine the horizontal and vertical airflow directions to calculate the composite airflow direction; Step 948: Combine the horizontal and vertical airflow velocities to calculate the composite airflow velocity; Step 949: Control the blowing device to blow air according to the combined blowing direction and combined blowing speed, and at the same time control the reminder device to rotate to the limit orientation, and execute the reminder plan for the driver according to the reminder flow rate and reminder duration.

[0011] Optionally, it also includes a treatment method for when the eye area is missing, the method comprising: Step 20: Obtain vehicle driving data, including steering wheel angle change rate, lane departure frequency, and continuous driving duration; Step 21: Input the vehicle driving data into the preset behavioral fatigue detection model to obtain the behavioral fatigue index; Step 22: If the behavioral fatigue index is greater than the preset normal behavioral threshold, then output the preset abnormal behavior signal and execute the preset tentative reminder scheme for the driver; Step 23: If the behavioral fatigue index is less than or equal to the preset normal behavioral threshold, output the preset non-abnormal signal and continue to execute steps 20 to 21 until the eye area can be extracted.

[0012] Optionally, methods for implementing a preset, tentative reminder scheme to the driver include: Step 220: Extract the eye-occluded area from the facial image; Step 221: Calculate the probe direction based on the eye-obstruction area and the preset location of the reminder device; Step 222: Control the reminder device to provide tentative reminders to the driver according to the probe direction, preset probe flow rate and probe duration; Step 223: When the trial period ends, acquire the driver's real-time facial image within the preset reaction time. Step 224: Analyze the real-time facial images according to preset reaction action features to obtain analysis results; Step 225: If the analysis results show characteristics of reactive actions, then output a preset awakening signal; Step 226: If the analysis results do not show any reaction action characteristics, the control reminder device will execute the preset enhanced reminder scheme for the driver.

[0013] Optionally, it also includes a method for handling missing mouth area, the method comprising: Step 2200: Normalize the eyelid opening and closing degree, blinking frequency, eye closure duration and gaze direction to obtain eye fatigue characteristics; Step 2201: Calculate the eye fatigue index by weighted fusion of eye fatigue characteristics; Step 2202: If the eye fatigue index is higher than the preset eye fatigue threshold, calculate the trial airflow direction and trial airflow speed based on the facial area and the preset location of the blower. Step 2203: Control the blowing device to blow air onto the driver's face area according to the trial blowing direction and trial blowing speed, and at the same time acquire the driver's current facial image; Step 2204: Extract the current facial image according to the preset deformation features. If the current facial image does not have deformation features, output the preset awake signal. Step 2205: If the current facial image has deformed features, the control reminder device will execute a preset enhanced reminder scheme for the driver.

[0014] Optional, also includes: Step 9500: When the reminder program ends, calculate the total duration at the end; Step 9501: If the end time reaches the preset sober duration, obtain the driver's verification facial image and execute steps 2 to 8 to obtain the verification fatigue level; Step 9502: Based on the verified fatigue level, find the corresponding reminder plan in the level plan library, and execute steps 90 to 95.

[0015] Secondly, the present invention provides a fatigue driving terminal recognition and processing system that integrates artificial intelligence, employing the following technical solution: A fatigue driving terminal recognition and processing system integrating artificial intelligence includes: The acquisition module is used to acquire facial images, vehicle driving data, real-time facial images, current facial images, and verification facial images; The memory is used to store the program of the fatigue driving terminal recognition and processing method that integrates artificial intelligence as described above; The processor loads and executes programs from memory.

[0016] In summary, the present invention has at least one of the following beneficial technical effects: It solves the problems that eye feature extraction is prone to failure in complex scenarios such as when the driver's eyes are obstructed or when there are sudden changes in lighting, and that relying on a single feature cannot achieve cross-validation of multi-dimensional features and has insufficient accuracy. It achieves high-precision real-time assessment of fatigue status and effective wake-up. This solves the problem that the water spray device cannot dynamically adjust the spray direction according to the driver's face orientation, and achieves the effect of accurately spraying water mist onto the sensitive areas of the cheeks. This invention solves the problem that the mechanical rotation range of the reminder device is limited, and that it cannot be aligned with the target area when the driver's face is facing beyond the rotation limit, thus causing the water spray reminder to fail. It achieves the effect of using airflow to apply lateral force to the water mist for angle compensation. Attached Figure Description

[0017] Figure 1 This is a flowchart of a fatigue driving terminal recognition and processing method that integrates artificial intelligence, as described in an embodiment of this application. Detailed Implementation

[0018] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0019] This invention discloses a fatigue driving terminal recognition and processing method integrating artificial intelligence. (Refer to...) Figure 1 A fatigue driving terminal recognition and processing method integrating artificial intelligence includes: Step 1: Obtain the driver's facial image.

[0020] A facial image refers to a digital image of the driver's facial area captured by a near-infrared camera. Facial images are obtained in real time by installing a near-infrared camera (wavelength 850nm or 940nm) in the vehicle's rearview mirror or dashboard, in conjunction with active near-infrared LED illumination.

[0021] Step 2: Denoise the facial image and extract the ROI to obtain fatigue-related regions, which include the facial region, eye region, and mouth region.

[0022] Noise reduction refers to removing random noise, lighting noise, and motion blur from an image, using bilateral filtering or Gaussian filtering.

[0023] The fatigue-related regions refer to the three image sub-regions most associated with fatigued driving characteristics, including the facial region (containing the complete face), the eye region (left and right eyes individually or together), and the mouth region (containing the upper and lower lips). The fatigue-related regions are obtained by cropping the Region of Interest (ROI) from the facial image. Specifically, the facial region is located using MTCNN or RetinaFace face detection algorithms; the eye region (keypoints 36 to 41 for the left eye and 42 to 47 for the right eye) and the mouth region (keypoints 48 to 68) are located based on 68 facial keypoints; after cropping, the image is normalized to 128 x 128 pixels.

[0024] Step 3: Define the image corresponding to the fatigue-related region as a fatigue-related image.

[0025] Fatigue-related images refer to sub-images containing only fatigue-related regions obtained from facial images after ROI extraction. The images of the three sub-regions extracted in step 2—the facial region, the eye region, and the mouth region—are collectively referred to as fatigue-related images.

[0026] Step 4: Input fatigue-related images into a preset fatigue recognition model to obtain fatigue features, including eyelid opening and closing degree, blinking frequency, eye closure duration, yawning frequency, head posture, and gaze direction.

[0027] A fatigue recognition model is a deep learning model that extracts multiple fatigue features of drivers from fatigue-related images. The preferred fatigue recognition model uses the YOLOv5 object detection network; alternatively, a hybrid architecture combining a convolutional neural network and a long short-term memory network can be used. If the YOLOv5 architecture is used, a lightweight version of YOLOv5s is employed, taking a 640x640 pixel image as input and outputting bounding boxes and feature parameters for the eyes, mouth, and head. Taking the hybrid architecture as an example, the convolutional neural network part uses a lightweight MobileNetV3 or EfficientNet-B0 network, taking a 128x128x3 pixel image as input and outputting a 512-dimensional feature vector; the long short-term memory network part is a single-layer LSTM with 128 hidden layers, taking a 30-frame input sequence and outputting time-series features. The model was trained using publicly available datasets (such as the Driving Fatigue Dataset, YAWDD Dataset, and UTA-RLDD Dataset) and self-collected data. Annotations included eye keypoint coordinates, mouth keypoint coordinates, head posture angle, gaze direction angle, and fatigue level (0 to 3). Training parameters were 200 epochs, a learning rate of 0.001, and a batch size of 32. The model outputs eyelid opening and closing degree, blinking frequency, eye-closing duration, yawning frequency, head posture, and gaze direction. After training, the model was deployed to a neural network processing unit and ran in real-time inference on the dashcam terminal.

[0028] Fatigue characteristics refer to physiological and behavioral parameters extracted from fatigue-related images to quantify the degree of driver fatigue. Eyelid opening / closing refers to the ratio of the distance between the upper and lower eyelids. Blinking frequency refers to the number of blinks per unit of time. Eye closure duration refers to the duration of a single eye closure. Yawning frequency refers to the number of yawns per unit of time. Head posture refers to the pitch, yaw, and roll angles of the head in three-dimensional space. Gaze direction refers to the horizontal and vertical angles of gaze.

[0029] Fatigue features were calculated based on the output of the fatigue recognition model. Head posture and gaze direction were directly output by the model; eyelid opening / closing, blinking frequency, eye-closing duration, and yawning frequency were obtained through geometric calculations and statistical methods based on the coordinates of key eye and mouth points output by the model. The specific calculation process is as follows: the positions of the six key points for eyelid opening / closing are defined as follows: p1 is the inner corner of the eye (closest to the bridge of the nose), p2 is the uppermost point of the upper eyelid, p3 is the rightmost (or leftmost) point of the upper eyelid, p4 is the inner corner of the eye (closest to the ear), p5 is the rightmost (or leftmost) point of the lower eyelid, and p6 is the lowermost point of the lower eyelid. The formula for calculating eyelid opening / closing is: ; In actual calculations, the eyelid opening and closing angles of the left and right eyes are calculated separately, and then the average of the two is taken as the final eyelid opening and closing angle. When the eyelid opening and closing angle is less than 0.2, the driver is determined to be in a closed-eye state; when the eyelid opening and closing angle is greater than or equal to 0.2, the driver is determined to be in an open-eye state. Blinking frequency is calculated frame by frame in continuous images to determine the closed-eye state. A complete blink is defined as the process of changing from an open-eye state to a closed-eye state and then returning to an open-eye state. The total number of blinks in 1 minute is counted to obtain the blinking frequency. Eye-closing duration is determined in continuous images by detecting the moment when the eyelid opening and closing angle decreases from greater than or equal to 0.2 to less than 0.2, which is recorded as the start time of eye-closing; and detecting the moment when the eyelid opening and closing angle recovers from less than 0.2 to greater than or equal to 0.2, which is recorded as the end time of eye-closing; the duration of a single eye-closing is the difference between the two. Yawning frequency is calculated based on the coordinates of eight key points of the mouth output by the fatigue recognition model to calculate the aspect ratio of the mouth. The locations of the eight key points are defined as follows: m1 is the left corner of the mouth, m5 is the right corner of the mouth, and m2, m3, and m4 are multiple key points on the upper and lower lips. The formula for calculating the aspect ratio of the mouth is: ; The larger the aspect ratio of the mouth, the wider the mouth is open. A mouth aspect ratio greater than 0.6 is considered to be in an open state. If the mouth remains open for more than 1 second, it is considered a yawn. The total number of yawns within 1 minute is the yawn frequency.

[0030] Step 5: Normalize the fatigue characteristics to obtain standard fatigue characteristics.

[0031] Standard fatigue characteristics refer to fatigue characteristic values ​​obtained after normalization, with values ​​uniformly ranging from [0, 1], ensuring comparability between different characteristics. Standard fatigue characteristics are obtained by performing min-max normalization on the fatigue characteristics, using the following formula: ; The output range is 0 to 1; the statistical ranges of each fatigue feature are as follows: eyelid opening and closing degree is 0.1 to 0.4, blinking frequency is 5 to 30 times / minute, eye closure duration is 100 to 500 milliseconds, yawning frequency is 0 to 5 times / minute, head posture is -30 degrees to +30 degrees, and gaze direction is -30 degrees to +30 degrees. After substituting the six features into the above formulas for calculation, six standard fatigue feature values ​​with values ​​ranging from 0 to 1 are obtained. Arranging them in order constitutes the standard fatigue feature vector.

[0032] Step 6: Calculate the comprehensive fatigue index by weighted fusion of standard fatigue characteristics.

[0033] The comprehensive fatigue index is a continuous numerical value ranging from 0 to 1, quantifying the driver's fatigue level at any given moment; a higher value indicates a more severe level of fatigue. The comprehensive fatigue index is calculated using a weighted fusion algorithm, with example weights for the following: eyelid opening / closing degree 0.25, blinking frequency 0.15, eye closure duration 0.15, yawning frequency 0.20, head posture 0.15, and gaze direction 0.10. These weights can be obtained by training on labeled datasets using logistic regression or linear support vector machines, or they can be set based on the experience of those skilled in the art.

[0034] Step 7: Match the comprehensive fatigue index with the preset threshold segments to obtain the threshold segments into which the comprehensive fatigue index falls.

[0035] Threshold segmentation refers to dividing the range of the comprehensive fatigue index [0, 1] into several continuous intervals, each interval corresponding to a fatigue level. The threshold segmentation is determined by those skilled in the art through statistical analysis of experimental data. For example, the comprehensive fatigue index can be divided into four continuous intervals: the first interval is [0, 0.3], corresponding to a normal state; the second interval is [0.3, 0.6], corresponding to mild fatigue; the third interval is [0.6, 0.8], corresponding to moderate fatigue; and the fourth interval is [0.8, 1.0], corresponding to severe fatigue.

[0036] Step 8: Based on the threshold segment into which the comprehensive fatigue index falls, find the corresponding fatigue level in the preset threshold level library.

[0037] The threshold level library contains a mapping relationship between threshold segments and fatigue levels. This library is pre-defined and stored by those skilled in the art; for example, [0, 0.3) is mapped to level 0 (normal), [0.3, 0.6) to level 1 (mild fatigue), [0.6, 0.8) to level 2 (moderate fatigue), and [0.8, 1.0] to level 3 (severe fatigue). After the comprehensive fatigue index is calculated, the system automatically compares it with the threshold segments, determines the corresponding interval, and retrieves the corresponding fatigue level from the threshold level library as the current driver's fatigue status assessment result.

[0038] Fatigue level refers to a grading index that characterizes the degree of driver fatigue, usually divided into four levels: Level 0 (normal), Level 1 (mild fatigue), Level 2 (moderate fatigue), and Level 3 (severe fatigue). Different levels correspond to different driver states: Level 0 indicates the driver is alert, focused, and reacts quickly; Level 1 indicates the driver shows slight signs of fatigue, possibly accompanied by occasional increased blinking frequency or brief periods of distraction, but can still maintain basic driving operations; Level 2 indicates increased driver fatigue, possibly characterized by frequent yawning, slight head drooping, or prolonged deviation of vision from the road ahead, significantly increasing driving risk; Level 3 indicates severe driver fatigue, possibly characterized by persistent eye closure, significant forward head tilting, or sluggish response to external stimuli, at which point the probability of a traffic accident is extremely high.

[0039] Step 9: Based on the fatigue level, find the corresponding reminder plan in the preset level plan library and execute the reminder plan for the driver.

[0040] The fatigue level alert library contains a mapping relationship between fatigue levels and alert schemes. Established through preset mapping relationships, the library assigns different water spray alert parameters to each fatigue level (normal, mild fatigue, moderate fatigue, and severe fatigue). For example: normal fatigue level corresponds to no alert scheme; mild fatigue level corresponds to an alert flow rate of 2 m / s and an alert duration of 0.3 seconds; moderate fatigue level corresponds to an alert flow rate of 4 m / s and an alert duration of 0.6 seconds; and severe fatigue level corresponds to an alert flow rate of 6 m / s and an alert duration of 1.0 second. The specific values ​​for the flow rate and duration can be obtained through experimental calibration. For example, by testing the driver's reaction to different water spray intensities at different fatigue levels, parameters that effectively wake the driver without causing excessive fright can be selected as preset values. It should be noted that the core alert method of this scheme is water spraying, but depending on actual needs, other alert methods can be added to the alert schemes corresponding to each fatigue level, such as on-screen text prompts, screen flashing, and vibration alarms. During system operation, the corresponding reminder flow rate and reminder duration can be quickly obtained by looking up the table in the level scheme library according to the fatigue level, and the water spraying reminder operation can be executed.

[0041] A reminder scheme refers to a set of driver reminder actions triggered based on fatigue level. Its core component is a water spray reminder, which includes two basic parameters: reminder flow rate and reminder duration. Reminder schemes are obtained by looking up a table in a fatigue level scheme database. Detailed steps for executing a reminder scheme for a driver will be provided later.

[0042] The method for retrieving the corresponding reminder plan from a preset level plan library based on the fatigue level and executing the reminder plan for the driver includes: Step 90: Based on the reminder scheme, find the corresponding reminder flow rate and reminder duration.

[0043] The alert flow rate refers to the initial velocity of the water mist as it leaves the nozzle when the alert program is executed. The alert flow rate is obtained by looking up a table in the fatigue level database, and is specifically determined based on the currently assessed fatigue level. For example, mild fatigue corresponds to 2 m / s, moderate fatigue to 4 m / s, and severe fatigue to 6 m / s. The alert flow rate is controlled by adjusting the pump voltage or duty cycle of the device; the higher the voltage or the larger the duty cycle, the faster the water flow rate.

[0044] The reminder duration refers to the duration of a single water spraying action during the execution of the reminder plan. The reminder duration is obtained by looking up a table in the fatigue level database, and is specifically determined based on the currently assessed fatigue level. For example, mild fatigue corresponds to 0.3 seconds, moderate fatigue to 0.6 seconds, and severe fatigue to 1.0 second. The reminder duration is controlled by setting the opening time of the solenoid valve in the device.

[0045] Step 91: Determine the driver's current facial orientation based on head posture.

[0046] Current facial orientation refers to the direction the driver's face points in three-dimensional space, represented by three Euler angles: pitch (up / down direction, head up or down), yaw (left / right direction, turn left or right), and roll (left / right tilt, ears close to or far from shoulders). The current facial orientation is obtained based on the head posture output by the fatigue recognition model. Specifically, a perspective n-point algorithm is used to solve the head rotation matrix based on 68 key facial points, obtaining the pitch, yaw, and roll angles. The pitch angle ranges from -30 degrees to +30 degrees, with positive values ​​indicating head up and negative values ​​indicating head down; the yaw angle ranges from -45 degrees to +45 degrees, with positive values ​​indicating head turn right and negative values ​​indicating head turn left; the roll angle ranges from -20 degrees to +20 degrees, with positive values ​​indicating head tilt to the right and negative values ​​indicating head tilt to the left.

[0047] Step 92: Determine the cheek region based on the current facial orientation, facial area, and preset cheek geometry.

[0048] Facial geometric features refer to the fixed spatial positional relationship of the cheek region relative to other facial key points (such as the tip of the nose, the corners of the eyes, and the corners of the mouth). These features are obtained through facial statistics. Based on a standard face model, the cheek region is located in the lower middle part of the face. Specifically, the cheek center point is located by shifting the nose tip as a reference point, horizontally outward by approximately one-quarter of the face width, and vertically downward by approximately one-tenth of the face height. In one specific implementation, the cheek key point coordinates can also be directly obtained through facial key point detection. For example, in a 68-point facial key point model, the left cheek corresponds to point 3 or 4, and the right cheek corresponds to point 15 or 14.

[0049] The cheek region refers to the skin area on the driver's face located below the eyes and on both sides of the mouth. This area is relatively sensitive, and water spray can effectively wake the driver without obstructing their vision. The cheek region is obtained as follows: First, the driver's current facial orientation is determined based on head posture, and the preset cheek geometry is spatially rotated and corrected. Then, within the facial region bounding box, the pixel coordinates of the cheek's center point are located based on the corrected geometry. Finally, a preset radius (e.g., 30 pixels) is expanded outward from this center point to form a circular or rectangular area, which serves as the final cheek region.

[0050] Step 93: Calculate the vertical height difference based on the cheek area and the preset reminder device position.

[0051] The location of the warning device refers to the fixed installation coordinates of the washer fluid spray system inside the vehicle. The location of the warning device is determined in advance by those skilled in the art and is usually installed below the interior rearview mirror or above the dashboard.

[0052] The vertical height difference refers to the difference in height between the cheek area and the alert device in the vertical direction. The vertical height difference is calculated through the following steps: First, obtain the pixel coordinates of the cheek area in the image, and convert these pixel coordinates into three-dimensional spatial coordinates using camera calibration parameters; then, extract the y-value (vertical coordinate) from this spatial coordinate system; finally, calculate the difference between the y-value of the cheek area and the y-value of the alert device's position, thus obtaining the vertical height difference.

[0053] Step 94: If the vertical height difference falls within the preset normal height range, plan the reminder orientation of the reminder device pointing towards the cheek area based on the cheek area and the position of the reminder device.

[0054] The normal height range refers to the vertical height difference interval within which the water spray can normally reach the cheek area without any other assistance. The normal height range is obtained through experimental calibration by those skilled in the art. After the reminder device is fixedly installed, the effective spray coverage area of ​​the water spray device (reminder device) in the vertical direction is measured without any other assistance. For example, when the water spray device (reminder device) sprays horizontally, the water mist falls naturally, and its effective coverage vertical height range is approximately 2 cm above and below the nozzle height. Therefore, the normal height range is set to [-2 cm, +2 cm] (i.e., the absolute value of the vertical height difference is less than or equal to 2 cm).

[0055] The alert orientation refers to the direction vector that the alert device's nozzle needs to point towards, that is, the spatial direction from the alert device's position to the center point of the driver's cheek area. The alert orientation is calculated based on the cheek area position and the alert device's position. The alert orientation vector equals the three-dimensional coordinates of the cheek area's center point minus the three-dimensional coordinates of the alert device's position. This vector is then converted into horizontal (yaw) and vertical (pitch) angles in a spherical coordinate system, yielding the target angles at which the alert device needs to rotate.

[0056] If the vertical height difference falls within the preset normal height range, it means that the driver's cheek height is basically level with the warning device. At this time, the warning direction of the warning device pointing towards the cheek area is planned according to the cheek area and the position of the warning device.

[0057] Step 95: If the reminder direction falls within the preset rotatable range, control the reminder device to rotate to the reminder direction, and execute the reminder plan for the driver according to the reminder flow rate and reminder duration.

[0058] The rotatable range refers to the angular range within which the reminder device can mechanically rotate in the horizontal direction. For example, a rotatable range of -30 degrees to +30 degrees means that the reminder device can rotate a maximum of 30 degrees to the left and 30 degrees to the right. The rotatable range is obtained through the mechanical structural parameters of the reminder device.

[0059] The reminder device is a hardware unit that performs the water spray reminder operation. Installed inside the vehicle, it can rotate to a designated orientation and spray water mist onto the driver's cheek area according to a preset reminder flow rate and duration. The core function of the reminder device is to alert a fatigued driver to regain attention through the tactile stimulation generated by the water spray. The reminder device consists of a miniature water pump or electromagnetic pump, a water tank, a rotatable nozzle, a miniature servo motor or stepper motor, and water pipes. It is fixedly installed below the rearview mirror, above the dashboard, on the steering column, or inside the A-pillar. After installation, its three-dimensional coordinates and zero-position orientation are calibrated and recorded. Rotation control is driven by the servo motor, with a horizontal rotation range of 30 to 45 degrees to the left and right, and it can also perform horizontal rotation. Water spray control adjusts the water pump drive voltage or PWM duty cycle to achieve the water flow rate, and controls the activation time to achieve the reminder duration. The nozzle uses an atomizing nozzle. Safety features include spraying away from the eyes, nose, and mouth, aiming only at the cheeks, a single spray volume of 0.5 to 1.5 ml, and maintaining a gentle reminder even at high speeds.

[0060] If the warning direction falls within the preset rotatable range, it means that the angle is within the limit angle of the mechanical rotation of the warning device. The servo motor of the warning device is controlled to drive the nozzle to rotate to the warning direction, and then the water pump is started to spray water mist according to the warning flow rate and warning duration obtained from the grade scheme library to complete the water spray warning operation for the driver.

[0061] This includes a method for handling situations where the vertical height difference falls within a preset normal height range, but the orientation does not fall within a preset rotatable range. This method includes: Step 950: Extract the upper limit orientation and lower limit orientation from the rotatable range.

[0062] The upper limit orientation refers to the maximum angle value within the preset rotation range of the reminder device, that is, the limit position for rightward rotation. The upper limit orientation is obtained by reading the mechanical structure parameters of the reminder device.

[0063] The lower limit orientation refers to the minimum angle value within the preset rotation range of the reminder device, that is, the extreme position for leftward rotation. The lower limit orientation is obtained by reading the mechanical structure parameters of the reminder device.

[0064] Step 951: If the reminder direction is greater than the upper limit direction, the upper limit direction will be used as the limit direction for the reminder direction.

[0065] The limit orientation refers to the closest boundary angle that the reminder device can actually rotate to when the reminder orientation exceeds the rotatable range. The limit orientation is obtained by comparing the reminder orientation with the upper and lower limit orientations. Specifically, the logic is as follows: if the reminder orientation is greater than the upper limit orientation, the upper limit orientation is assigned to the limit orientation; if the reminder orientation is less than the lower limit orientation, the lower limit orientation is assigned to the limit orientation.

[0066] If the indicated orientation is greater than the upper limit orientation, it means the required rotation angle exceeds the mechanical right-turn limit. In this case, the upper limit orientation will be used as the limit orientation.

[0067] Step 952: If the reminder orientation is less than the lower limit orientation, the lower limit orientation shall be used as the limit orientation of the reminder orientation.

[0068] If the indicated orientation is less than the lower limit orientation, it means the required rotation angle exceeds the mechanical left-turn limit. In this case, the lower limit orientation will be used as the limit orientation.

[0069] Step 953: Calculate the orientation deviation based on the suggested orientation and the corresponding limit orientation.

[0070] Orientation deviation refers to the angular difference between the alert orientation and the limit orientation, representing the angle that the alert device's mechanical rotation cannot reach. Orientation deviation is a signed numerical value; its absolute value indicates the magnitude of the angle that needs compensation, and its sign indicates the direction of compensation. Orientation deviation is obtained by subtracting the limit orientation from the alert orientation.

[0071] Step 954: Calculate the blowing direction and blowing speed of the blower based on the orientation deviation.

[0072] A blower is an auxiliary device that generates directional airflow. It applies lateral force to the sprayed water mist through the jet of air, causing the mist to deflect during flight and thus altering its landing point. The blower is used to compensate for horizontal angular deviations or altitude differences that the mechanical rotation of the reminder device cannot reach. The blower consists of the following components: a miniature fan or air pump to generate airflow; an adjustable-direction air outlet or a set of fixed-direction air outlets to guide the airflow; a miniature servo (if an adjustable-direction air outlet is used) to drive the air outlet to rotate to the target angle; and a set of air ducts to connect the air source and the air outlet. The blower is fixedly installed near the reminder device, with the air outlet facing the side of the water mist spray path so that the airflow can effectively act on the water mist. The blower can use a DC brushless fan, and the wind speed is controlled by adjusting the fan drive voltage or PWM duty cycle; for example, the wind speed adjustment range is 0 to 10 m / s.

[0073] The airflow direction refers to the direction of the airflow generated by the air blower, used to deflect the water mist towards the target direction. The airflow direction is determined by the sign of the orientation deviation. If the orientation deviation is positive (indicating the orientation is greater than the limit orientation), it means the water mist needs to be deflected to the right, so the airflow direction is set to the right (i.e., the airflow blows from left to right); if the orientation deviation is negative (indicating the orientation is less than the limit orientation), it means the water mist needs to be deflected to the left, so the airflow direction is set to the left (i.e., the airflow blows from right to left).

[0074] Airflow velocity refers to the speed of the airflow generated by the blowing device. Airflow velocity determines the angle at which water mist is deflected by the airflow: the higher the airflow velocity, the larger the deflection angle; the lower the airflow velocity, the smaller the deflection angle. Airflow velocity is determined based on the absolute value of the orientation deviation, and the two are positively correlated. For example, a linear mapping formula can be used: airflow velocity equals the absolute value of the orientation deviation multiplied by a proportionality coefficient, where the proportionality coefficient is obtained through experimental calibration. For example, if the orientation deviation is 15 degrees and the proportionality coefficient is 0.2, then the airflow velocity is 3 meters per second.

[0075] Step 955: Control the blowing device to blow air according to the blowing direction and speed, and at the same time control the reminder device to rotate to the extreme orientation, and execute the reminder plan for the driver according to the reminder flow rate and reminder duration.

[0076] When the vertical height difference is normal and the warning orientation exceeds the rotatable range, the blower and the warning device work together simultaneously: the warning device rotates to the limit orientation, the blower determines the blowing direction based on the sign of the orientation deviation, and determines the blowing speed based on the absolute value of the orientation deviation. After both start synchronously, the warning device sprays water according to the warning flow rate and warning duration; the water mist is further deflected by the airflow based on the mechanical spray direction, so that the actual landing point is consistent with the warning orientation, thus achieving the effect of reminding the driver.

[0077] This includes a method for handling situations where the vertical height difference does not fall within a preset normal height range, but the orientation falls within a preset rotatable range. This method includes: Step 940: If the vertical height difference is greater than 0, then the preset vertical upward direction will be used as the vertical blowing direction.

[0078] The vertical upward direction refers to the direction in which the airflow from the blower is perpendicular to the ground and points towards the vehicle's roof. This vertical upward direction is predetermined by the structural design of the blower. During installation, the vertical upward orientation of the air outlet is marked as the reference zero point.

[0079] The vertical airflow direction refers to the direction of the airflow generated by the airflow device in the vertical plane, including two possible directions: vertically upward and vertically downward. It is used to lift water mist upward or press it downward to compensate for vertical height deviation. The vertical airflow direction is determined by the sign of the vertical height difference.

[0080] If the vertical height difference is greater than 0, it means that the cheek is higher than the reminder device; at this time, the preset vertical upward direction will be used as the vertical blowing direction.

[0081] Step 941: If the vertical height difference is less than 0, then the preset vertical downward direction is taken as the vertical blowing direction.

[0082] The vertically downward direction refers to the direction in which the airflow from the blower is perpendicular to the ground and points towards the vehicle floor. This vertically downward direction is predetermined by the structural design of the blower. When installing the blower, the vertically downward orientation of the air outlet is designated as the reference direction.

[0083] If the vertical height difference is less than 0, it means that the cheek is lower than the reminder device; at this time, the preset vertical downward direction will be used as the vertical blowing direction.

[0084] Step 942: Find the vertical wind speed in the preset height wind speed library based on the absolute value of the vertical height difference.

[0085] The height-wind speed database contains a mapping relationship between the absolute value of the vertical height difference and the vertical wind speed. The database was established through experimental calibration. The specific steps are as follows: After the reminder device is fixedly installed, measure the minimum wind speed required to deflect the water mist precisely to the cheek area at different vertical height differences (e.g., 0 cm, 2 cm, 4 cm, 6 cm, 8 cm, and 10 cm); record the measured data pairs (absolute height difference, wind speed) as a mapping table. For intermediate values ​​not listed in the table, they can be calculated using linear interpolation.

[0086] Vertical airflow velocity refers to the speed of the airflow generated by the air-blowing device. The vertical airflow velocity is obtained by looking up the absolute value of the vertical height difference in a preset height airflow velocity database.

[0087] Step 943: Control the blower to blow air in a vertical direction and at a vertical air speed, while simultaneously controlling the reminder device to rotate to the reminder orientation.

[0088] Simultaneously control the reminder device to rotate to the reminder orientation, and control the blower to blow air in a vertical direction and at a vertical air speed.

[0089] Step 944: Implement the reminder plan for the driver according to the reminder flow rate and reminder duration.

[0090] Based on the reminder flow rate and reminder duration obtained from the grade scheme library, the reminder device is controlled to spray water mist onto the driver's cheek area to complete the water spray reminder operation.

[0091] This includes a method for handling situations where the vertical height difference does not fall within a preset normal height range and the orientation does not fall within a preset rotatable range. This method includes: Step 945: Based on the indicated orientation and rotatable range, perform steps 950 to 954 to calculate the horizontal blowing direction and horizontal blowing speed of the blower.

[0092] The horizontal airflow direction refers to the direction of the airflow generated by the blowing device on the horizontal plane, including two possible directions: left and right. The horizontal airflow direction is obtained by performing steps 950 to 954.

[0093] The horizontal airflow velocity refers to the velocity of the airflow generated by the blowing device. The horizontal airflow velocity is obtained by performing steps 950 to 954.

[0094] Step 946: Calculate the vertical blowing direction and vertical blowing speed of the blowing device based on the vertical height difference by performing steps 940 to 942.

[0095] Step 947: Combine the horizontal and vertical airflow directions to calculate the composite airflow direction.

[0096] The composite airflow direction refers to the overall airflow direction obtained by vector combining the horizontal and vertical airflow directions. The composite airflow direction is calculated by vector combining the horizontal and vertical airflow directions. Specifically, using a three-dimensional coordinate system as a reference, the horizontal airflow direction provides the X-axis (left-right) component, and the vertical airflow direction provides the Y-axis (up-down) component. The direction angle of the composite airflow direction can be calculated using the following formulas: Horizontal deflection angle = angle corresponding to the horizontal airflow direction (left is negative, right is positive), Vertical pitch angle = angle corresponding to the vertical airflow direction (up is positive, down is negative). For example, if the horizontal airflow direction is to the right and the vertical airflow direction is upward, then the composite airflow direction is upward to the right.

[0097] Step 948: Combine the horizontal and vertical airflow velocities to calculate the composite airflow velocity.

[0098] The composite wind speed refers to the total wind speed obtained by vector combining the horizontal and vertical wind speeds. Since the horizontal and vertical directions are perpendicular, the formula for calculating the composite wind speed is: Composite wind speed equals the square of the horizontal wind speed plus the square of the vertical wind speed, then taking the square root of the sum. For example, if the horizontal wind speed is 3 meters per second and the vertical wind speed is 4 meters per second, then the composite wind speed is 5 meters per second.

[0099] Step 949: Control the blowing device to blow air according to the combined blowing direction and combined blowing speed, and at the same time control the reminder device to rotate to the limit orientation, and execute the reminder plan for the driver according to the reminder flow rate and reminder duration.

[0100] When both the horizontal and vertical directions exceed the normal range, the system simultaneously controls the blowing device to blow air according to the combined blowing direction and combined blowing speed, and controls the reminder device to rotate to the limit orientation. Then, water is sprayed according to the reminder flow rate and reminder duration. The mechanical rotation provides the basic direction, and the two-dimensional blowing compensates for the horizontal angle deviation and vertical height deviation, so that the water mist accurately reaches the target cheek area.

[0101] This also includes a treatment method for when the eye area is missing, which includes: Step 20: Obtain vehicle driving data, including steering wheel angle change rate, lane departure frequency, and continuous driving duration.

[0102] Vehicle driving data refers to dynamic parameters reflecting the vehicle's operating status and driver behavior. Vehicle driving data is acquired in real time via the vehicle's CAN bus or OBD interface. Specifically, it includes at least one of the following parameters: steering wheel angle change rate (collected by a steering wheel angle sensor, calculating the angle change per unit time); lane departure frequency (collected by a forward-facing camera or lane keeping assist system, counting the number of times the vehicle deviates from the lane line per unit time); and continuous driving time (accumulated by a timer since the last rest stop). Steering wheel angle change rate refers to the change in steering wheel angle per unit time. Lane departure frequency refers to the number of times the vehicle deviates from the lane line per unit time. Continuous driving time refers to the cumulative time the vehicle has been continuously driving since the last rest stop.

[0103] Step 21: Input the vehicle driving data into the preset behavioral fatigue detection model to obtain the behavioral fatigue index.

[0104] The behavioral fatigue detection model is a mathematical model that maps vehicle driving data to a behavioral fatigue index. It employs a comprehensive scoring rule model: scores are assigned based on whether various vehicle driving data exceed preset thresholds, and these scores are accumulated to obtain the behavioral fatigue index. For example, the following scoring rules are set: 1 point for a steering wheel angle change rate exceeding 30 degrees / second, 1 point for a lane departure frequency exceeding 3 times / minute, and 1 point for continuous driving time exceeding 120 minutes. The behavioral fatigue index is the sum of these scores, ranging from 0 to 3 points. When the behavioral fatigue index is greater than or equal to 2 points, it is considered abnormal behavior. The above thresholds are determined through statistical calibration using real-vehicle driving data, selecting the 95% driver fatigue trigger threshold.

[0105] The behavioral fatigue index is a value calculated using a behavioral fatigue detection model. It is obtained by inputting vehicle driving data into the model. When using a comprehensive scoring rule model, the behavioral fatigue index is the sum of scores from each scoring rule, ranging from 0 to 3 points. For example, if the steering wheel angle change rate exceeds a threshold (1 point), the lane departure frequency exceeds a threshold (1 point), and the continuous driving time does not exceed a threshold (0 points), then the behavioral fatigue index is 2 points.

[0106] Step 22: If the behavioral fatigue index is greater than the preset normal behavioral threshold, a preset abnormal behavior signal is output, and a preset trial reminder scheme is executed on the driver.

[0107] The normal behavior threshold refers to the critical value used to judge whether a driver's behavior is normal. The normal behavior threshold is obtained through experimental calibration by those skilled in the art. Specifically, a large amount of vehicle driving data is collected under normal driving conditions, the statistical distribution of the behavioral fatigue index is calculated, and the 90th or 95th percentile is used as the normal behavior threshold. For example, when using a comprehensive scoring rule model, the normal behavior threshold is set to 1 point: a score of 0 or 1 is considered normal behavior, and a score of 2 or 3 is considered abnormal behavior.

[0108] An abnormal behavior signal is a warning signal that occurs when the behavioral fatigue index exceeds the normal behavioral threshold. The abnormal behavior signal is preset by someone skilled in the art and is triggered when the behavioral fatigue index exceeds the normal behavioral threshold.

[0109] The trial alert program involves spraying water on the driver when there is a loss of vision in the eye area and the vehicle is behaving abnormally. The driver's reaction to the water spray is observed to determine if they are truly fatigued. The specific details of the trial alert program are explained in subsequent steps, see steps 220 to 226.

[0110] If the behavioral fatigue index is greater than the preset normal behavior threshold, the vehicle behavior data indicates that the driver is abnormal. At this time, an abnormal behavior signal is output, and a preset trial reminder scheme is executed on the driver. The driver's true state is further confirmed through water spraying and reaction detection.

[0111] Step 23: If the behavioral fatigue index is less than or equal to the preset normal behavioral threshold, output the preset non-abnormal signal and continue to execute steps 20 to 21 until the eye area can be extracted.

[0112] The "No Abnormality Signal" is a notification signal that appears when the behavioral fatigue index is less than or equal to the normal behavioral threshold. This signal is pre-set by those skilled in the art and is triggered when the behavioral fatigue index is less than or equal to the normal behavioral threshold.

[0113] If the behavior fatigue index is less than or equal to the preset normal behavior threshold, it indicates that the vehicle behavior is normal. At this time, a no-abnormal signal is output, and steps 20 to 21 are continuously executed to continuously update the behavior fatigue index until the eye area can be extracted.

[0114] The methods for implementing a pre-set, tentative reminder plan for the driver include: Step 220: Extract the eye-occluded area from the facial image.

[0115] The eye occlusion region refers to the spatial area in a driver's facial image where an object obscures the eyes, such as sunglasses lenses, hands, or hair. The eye occlusion region is determined by first using a face detection algorithm to locate the eye area within the facial region, then analyzing the image of the eye area to identify the outline and position of the occluding object.

[0116] Step 221: Calculate the probe direction based on the eye-obstruction area and the preset location of the reminder device.

[0117] The trial orientation refers to the direction the alert device nozzle should point, specifically the spatial direction from the alert device's location towards the center point of the lower edge of the eye-obstructing area. The trial orientation is calculated using the center point of the lower edge of the eye-obstructing area and the preset alert device location.

[0118] Step 222: Control the reminder device to provide tentative reminders to the driver according to the tentative direction, preset tentative flow rate and tentative duration.

[0119] The test flow rate refers to the initial velocity of the water mist leaving the nozzle during a test spray. The test flow rate is preset by someone skilled in the art. For example, the test flow rate is preset to 2 meters per second. Under normal driving conditions, water is sprayed with the lower edge of the area obscured by the eyes as the target, and the flow rate that can be clearly perceived by the driver without causing fright or discomfort is selected as the test flow rate.

[0120] The trial duration refers to the duration of a single water spray during a trial spray. The trial duration is preset by those skilled in the art. For example, the trial duration is preset to 0.3 seconds, selecting the shortest effective duration while ensuring that sufficient water mist can reach the target area at the lower edge.

[0121] Step 223: When the trial period ends, acquire the driver's real-time facial image within the preset reaction time.

[0122] Reaction time refers to the length of the time window during which the driver can react after the initial water spray. The reaction time is preset by someone skilled in the art. For example, the preset reaction time is 3 seconds. This value is based on the average human reaction time to sudden stimuli. Normal instinctive reactions (such as blinking, turning the head, or raising a hand to wipe the face) typically occur within 0.5 to 2 seconds, and a 3-second reaction time covers the reaction time of the vast majority of drivers.

[0123] Real-time facial images refer to continuously captured images of the driver's face during the reaction time following the initial water spray. These images are obtained by continuously activating a near-infrared camera within the reaction time.

[0124] When the trial period ends, it indicates that the trial water spraying operation is complete. At this point, the reaction observation phase begins, during which real-time facial images of the driver are acquired within the preset reaction time.

[0125] Step 224: Analyze the real-time facial image according to the preset reaction action features to obtain the analysis results.

[0126] Reaction action features refer to the identifiable visual patterns in an image of a driver's instinctive response to water spray stimulation, including changes in head posture, hand movements, and blink frequency. Since the water spray target is the lower edge of the area obscured by the eyes (near the cheek), typical reaction actions include: rubbing the face or removing the obstruction with the hand, rapid head rotation or tilting back, and a significant increase in blink frequency. The specific rules for determining reaction action features are as follows: within the reaction time after water spray, if the change in head pitch angle, yaw angle, or roll angle exceeds a preset threshold (e.g., 15 degrees), it is considered a reaction action; by detecting preset hand key points, it is determined whether the hand enters the face detection box, or if the minimum distance between the hand key point and the face key point is less than a preset threshold (e.g., 5 cm), it is considered a reaction action; the blink frequency within 2 seconds after water spray is calculated to the baseline blink frequency within 2 seconds before water spray, and if the increase exceeds a preset threshold (e.g., 50%), it is considered a reaction action. Meeting any one of these features indicates the presence of a reaction action feature. The above thresholds were determined through statistical calibration using real vehicle samples, selecting the critical value that is stably triggered by 95% of drivers.

[0127] The analysis result refers to the judgment conclusion obtained after analyzing real-time facial images according to reaction action features. The analysis result is obtained by comparing real-time facial images with preset reaction action features. In continuous images acquired within the reaction time, the presence of reaction action features is analyzed frame by frame or segment by segment. If a reaction action feature is detected at any moment within the reaction time, the analysis result is "reaction action feature exists"; if no reaction action feature is detected throughout the entire reaction time, the analysis result is "reaction action feature does not exist".

[0128] Step 225: If the analysis results show reactive behavior characteristics, then output a preset awakening signal.

[0129] A wake-up signal is a prompt signal that is set in advance by a person skilled in the art when the analysis results show reactive behavior.

[0130] If the analysis results show reactive behavior, it indicates that the driver is responding to the water spray stimulus and is judged to be in a conscious state. At this time, a conscious signal is output.

[0131] Step 226: If the analysis results do not show any reaction action characteristics, the control reminder device will execute the preset enhanced reminder scheme for the driver.

[0132] An enhanced alert scheme refers to a stronger and more obvious alert action performed when the analysis results do not indicate a reaction action. Enhanced alert schemes are preset by those skilled in the art and are stronger than conventional alert schemes. For example, an enhanced alert scheme may include one or more of the following actions: increasing the water spray velocity to 6 m / s, increasing the water spray duration to 1.0 second, activating a buzzer to emit a high-frequency alarm sound, and triggering seat vibration. The enhanced alert scheme continues to execute until a driver reaction is detected or the vehicle comes to a safe stop.

[0133] If the analysis results show no behavioral characteristics, it indicates that the driver is not responding to the water spray stimulus and is judged to be in a state of fatigue. At this time, the control reminder device will execute a preset enhanced reminder program to wake up the driver through stronger stimulation.

[0134] This also includes a treatment method for when the mouth area is missing, which includes: Step 2200: Normalize the eyelid opening and closing degree, blinking frequency, eye closure duration and gaze direction to obtain eye fatigue characteristics.

[0135] Eye fatigue characteristics refer to the standardized feature values ​​obtained by normalizing four eye-related fatigue features: eyelid opening and closing, blinking frequency, eye closure duration, and gaze direction. The values ​​range from 0 to 1. Eye fatigue characteristics are obtained by performing min-max normalization on each of the four features, as described in step 5.

[0136] Step 2201: Calculate the eye fatigue index by weighted fusion of eye fatigue characteristics.

[0137] The eye fatigue index is a numerical value obtained by weighted fusion of eye fatigue features. The index is calculated using a weighted fusion algorithm, with weights set as follows: eyelid opening / closing degree 0.30, blinking frequency 0.25, eye closure duration 0.25, and gaze direction 0.20. These weights can be obtained by training on labeled datasets using logistic regression or linear support vector machines, or they can be set based on expert experience. The output range is 0 to 1, where 0 indicates alertness and 1 indicates extreme fatigue.

[0138] Step 2202: If the eye fatigue index is higher than the preset eye fatigue threshold, calculate the trial airflow direction and trial airflow speed based on the facial area and the preset location of the air blower.

[0139] The eye fatigue threshold is a critical value used to determine whether a driver is suspected of being fatigued. The eye fatigue threshold is obtained through experimental calibration by those skilled in the art. Specifically, eye feature data are collected from the driver in a normal, alert state while wearing a mask, and the statistical distribution of the eye fatigue index is calculated. The 90th or 95th percentile is used as the eye fatigue threshold.

[0140] The test airflow direction refers to the direction of the airflow generated by the blower device, used to blow air onto the driver's face area to test whether the mask deforms due to yawning. The test airflow direction is calculated based on the face area and the preset position of the blower device. The specific steps are as follows: First, obtain the coordinates of the center point of the face area (image pixel coordinates); second, convert these pixel coordinates into three-dimensional spatial coordinates using camera calibration parameters; then, calculate the direction vector from the blower device position to these spatial coordinates; finally, convert this direction vector into horizontal and vertical angles to obtain the test airflow direction.

[0141] The test airflow speed refers to the velocity of the airflow generated by the blowing device. The test airflow speed is obtained by being preset by someone skilled in the art. For example, the test airflow speed is preset to 3 meters per second. Under the normal condition of the driver wearing a mask, air is blown onto the face at different speeds. The speed selected is one that allows the deformation of the mask when yawning to be clearly detected, without causing discomfort or fright to the driver.

[0142] If the eye fatigue index is higher than the preset eye fatigue threshold, it indicates that the driver is suspected of being fatigued. At this time, the test airflow direction and wind speed are calculated based on the facial area and the position of the blower. The blower is then activated to blow air onto the face, and the deformation of the mask is detected to confirm whether the driver is yawning.

[0143] Step 2203: Control the blowing device to blow air onto the driver's face area according to the trial blowing direction and trial blowing speed, and at the same time acquire the driver's current facial image.

[0144] The current facial image refers to the driver's facial image captured in real time after the blower is activated. The current facial image is obtained by continuously capturing images using a near-infrared camera during the blower operation.

[0145] Step 2204: Extract the current facial image according to the preset deformation features. If the current facial image does not have deformation features, output the preset awake signal.

[0146] Deformation features refer to the visual pattern of inward concavity of the mask material caused by inhalation when a driver wearing a mask yawns. This is specifically manifested as the mask area shrinking inward, and abrupt changes in local grayscale or texture. Deformation features are obtained by analyzing image changes in the mask area within a current facial image. The specific detection method is as follows: The mask area from the tip of the nose to the chin is determined using geometric localization based on facial key points; the mask contour is extracted using Canny edge detection, or the optical flow field is calculated using the Farneback optical flow method; based on the contour difference area calculation method, the contour difference of the mask area before and after yawning is compared; a temporal multi-frame differential verification method is used to eliminate overall displacement interference caused by yawning; if the area of ​​inward shrinkage exceeds a preset threshold (e.g., 10% of the mask area area), a deformation feature is determined to exist.

[0147] If the current facial image does not show any deformation features, it means that no yawning behavior has been detected, and the driver is determined to be awake. At this time, the preset awakening signal is output.

[0148] Step 2205: If the current facial image has deformed features, the control reminder device will execute a preset enhanced reminder scheme for the driver.

[0149] The enhanced reminder scheme is defined and obtained in step 226.

[0150] If the current facial image shows distorted features, it indicates that yawning has been detected, confirming that the driver is fatigued. At this time, the control reminder device will execute a preset enhanced reminder scheme for the driver.

[0151] This also includes: Step 9500: When the reminder program ends, accumulate the duration at the end.

[0152] The end time refers to the accumulated time starting from the moment the reminder program is completed. The end time is obtained by accumulating the time through a timer. When the reminder program is completed (i.e., the moment the reminder device finishes spraying water and the blower is turned off), the built-in timer starts counting down.

[0153] When the reminder process ends, it indicates that the reminder operation is complete. At this point, the duration of the reminder ends is accumulated. Once the duration of the reminder ends reaches the preset awake time, a verification facial image is obtained.

[0154] Step 9501: If the end time reaches the preset sober duration, obtain the driver's verification facial image and execute steps 2 to 8 to obtain the verification fatigue level.

[0155] Awareness duration refers to the length of time the system waits for re-verification after the reminder ends. Awareness duration is preset by those skilled in the art; for example, a preset alert duration of 5 seconds is used. After a driver is reminded, the average time required for them to recover from fatigue to normal driving is observed, and a value that covers the recovery time of most drivers without causing excessive verification delays is selected as the alert duration.

[0156] Verification facial images refer to the driver's facial images that are re-captured after the required awake time has elapsed at the end of the period. Verification facial images are acquired in real-time using a near-infrared camera.

[0157] The verified fatigue level refers to the fatigue level recalculated based on the verified facial image. The verified fatigue level is obtained by performing steps 2 through 8.

[0158] If the time taken to regain consciousness reaches the preset wakefulness duration, it indicates that the verification waiting time has elapsed. At this point, a facial image is acquired for verification, and the fatigue level is reassessed.

[0159] Step 9502: Based on the verified fatigue level, find the corresponding reminder plan in the level plan library, and execute steps 90 to 95.

[0160] Based on the verified fatigue level, the corresponding reminder scheme is searched in the level scheme library, and then steps 90 to 95 are executed. If the verified fatigue level is normal, no reminder is executed; if it is mild, moderate or severe fatigue, a water spray reminder is executed with the corresponding reminder flow rate and reminder duration, respectively.

[0161] Based on the same inventive concept, embodiments of the present invention provide a fatigue driving terminal recognition and processing system that integrates artificial intelligence.

[0162] A fatigue driving terminal recognition and processing system integrating artificial intelligence includes: The acquisition module is used to acquire facial images, vehicle driving data, real-time facial images, current facial images, and verification facial images; The memory stores a computer program that can be loaded by a processor and executed as a fatigue driving terminal recognition and processing method that integrates artificial intelligence; The processor loads and executes programs from memory.

[0163] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0164] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A fatigue driving terminal recognition and processing method integrating artificial intelligence, characterized in that, include: Step 1: Obtain the driver's facial image; Step 2: Denoise the facial image and extract the ROI to obtain fatigue-related regions, which include the facial region, eye region, and mouth region; Step 3: Define the images corresponding to fatigue-related regions as fatigue-related images; Step 4: Input fatigue-related images into a preset fatigue recognition model to obtain fatigue features, including eyelid opening and closing degree, blinking frequency, eye closing duration, yawning frequency, head posture, and gaze direction; Step 5: Normalize the fatigue characteristics to obtain standard fatigue characteristics; Step 6: Calculate the comprehensive fatigue index by weighted fusion of standard fatigue characteristics; Step 7: Match the comprehensive fatigue index with the preset threshold segments to obtain the threshold segments into which the comprehensive fatigue index falls; Step 8: Based on the threshold segment into which the comprehensive fatigue index falls, find the corresponding fatigue level in the preset threshold level library; Step 9: Based on the fatigue level, find the corresponding reminder plan in the preset level plan library and execute the reminder plan for the driver.

2. The fatigue driving terminal recognition and processing method integrating artificial intelligence according to claim 1, characterized in that, The method for executing the reminder plan on the driver by searching for the corresponding reminder plan in the preset level plan library based on the fatigue level includes: Step 90: Based on the reminder scheme, find the corresponding reminder flow rate and reminder duration; Step 91: Determine the driver's current facial orientation based on head posture; Step 92: Determine the cheek region based on the current facial orientation, facial area, and preset cheek geometry; Step 93: Calculate the vertical height difference based on the cheek area and the preset reminder device position; Step 94: If the vertical height difference falls within the preset normal height range, plan the reminder orientation of the reminder device pointing towards the cheek area based on the cheek area and the position of the reminder device; Step 95: If the reminder direction falls within the preset rotatable range, control the reminder device to rotate to the reminder direction, and execute the reminder plan for the driver according to the reminder flow rate and reminder duration.

3. The fatigue driving terminal recognition and processing method integrating artificial intelligence according to claim 2, characterized in that, It also includes a handling method for situations where the vertical height difference falls within a preset normal height range, but the orientation does not fall within a preset rotatable range. This method includes: Step 950: Extract the upper and lower orientations from the rotatable range; Step 951: If the reminder direction is greater than the upper limit direction, the upper limit direction shall be used as the limit direction of the reminder direction; Step 952: If the reminder direction is less than the lower limit direction, the lower limit direction will be used as the limit direction of the reminder direction; Step 953: Calculate the orientation deviation based on the suggested orientation and the corresponding extreme orientation; Step 954: Calculate the blowing direction and blowing speed of the blower based on the orientation deviation; Step 955: Control the blowing device to blow air according to the blowing direction and speed, and at the same time control the reminder device to rotate to the extreme orientation, and execute the reminder plan for the driver according to the reminder flow rate and reminder duration.

4. The fatigue driving terminal recognition and processing method integrating artificial intelligence according to claim 2, characterized in that, It also includes a handling method for situations where the vertical height difference does not fall within a preset normal height range, but the orientation falls within a preset rotatable range. This method includes: Step 940: If the vertical height difference is greater than 0, then the preset vertical upward direction will be used as the vertical blowing direction. Step 941: If the vertical height difference is less than 0, then the preset vertical downward direction will be used as the vertical blowing direction. Step 942: Find the vertical wind speed in the preset height wind speed library based on the absolute value of the vertical height difference; Step 943: Control the blower to blow air in a vertical direction and at a vertical air speed, while simultaneously controlling the reminder device to rotate to the reminder orientation; Step 944: Implement the reminder plan for the driver according to the reminder flow rate and reminder duration.

5. The fatigue driving terminal recognition and processing method integrating artificial intelligence according to claim 3, characterized in that, It also includes a handling method for situations where the vertical height difference does not fall within a preset normal height range and the orientation does not fall within a preset rotatable range. This method includes: Step 945: Based on the indicated orientation and rotatable range, perform steps 950 to 954 to calculate the horizontal blowing direction and horizontal blowing speed of the blower; Step 946: Calculate the vertical blowing direction and vertical blowing speed of the blowing device based on the vertical height difference by performing steps 940 to 942; Step 947: Combine the horizontal and vertical airflow directions to calculate the composite airflow direction; Step 948: Combine the horizontal and vertical airflow velocities to calculate the composite airflow velocity; Step 949: Control the blowing device to blow air according to the combined blowing direction and combined blowing speed, and at the same time control the reminder device to rotate to the limit orientation, and execute the reminder plan for the driver according to the reminder flow rate and reminder duration.

6. The fatigue driving terminal recognition and processing method integrating artificial intelligence according to claim 1, characterized in that, It also includes methods for handling cases where the eye area is missing, including: Step 20: Obtain vehicle driving data, including steering wheel angle change rate, lane departure frequency, and continuous driving duration; Step 21: Input the vehicle driving data into the preset behavioral fatigue detection model to obtain the behavioral fatigue index; Step 22: If the behavioral fatigue index is greater than the preset normal behavioral threshold, then output the preset abnormal behavior signal and execute the preset tentative reminder scheme for the driver; Step 23: If the behavioral fatigue index is less than or equal to the preset normal behavioral threshold, output the preset non-abnormal signal and continue to execute steps 20 to 21 until the eye area can be extracted.

7. The fatigue driving terminal recognition and processing method integrating artificial intelligence according to claim 6, characterized in that, Methods for implementing pre-set, tentative reminder schemes to drivers include: Step 220: Extract the eye-occluded area from the facial image; Step 221: Calculate the probe direction based on the eye-obstruction area and the preset location of the reminder device; Step 222: Control the reminder device to provide tentative reminders to the driver according to the probe direction, preset probe flow rate and probe duration; Step 223: When the trial period ends, acquire the driver's real-time facial image within the preset reaction time. Step 224: Analyze the real-time facial images according to preset reaction action features to obtain analysis results; Step 225: If the analysis results show characteristics of reactive actions, then output a preset awakening signal; Step 226: If the analysis results do not show any reaction action characteristics, the control reminder device will execute the preset enhanced reminder scheme for the driver.

8. The fatigue driving terminal recognition and processing method integrating artificial intelligence according to claim 1, characterized in that, It also includes a treatment method for when the mouth area is missing, which includes: Step 2200: Normalize the eyelid opening and closing degree, blinking frequency, eye closure duration and gaze direction to obtain eye fatigue characteristics; Step 2201: Calculate the eye fatigue index by weighted fusion of eye fatigue characteristics; Step 2202: If the eye fatigue index is higher than the preset eye fatigue threshold, calculate the trial airflow direction and trial airflow speed based on the facial area and the preset location of the blower. Step 2203: Control the blowing device to blow air onto the driver's face area according to the trial blowing direction and trial blowing speed, and at the same time acquire the driver's current facial image; Step 2204: Extract the current facial image according to the preset deformation features. If the current facial image does not have deformation features, output the preset awake signal. Step 2205: If the current facial image has deformed features, the control reminder device will execute a preset enhanced reminder scheme for the driver.

9. The fatigue driving terminal recognition and processing method integrating artificial intelligence according to claim 2, characterized in that, Also includes: Step 9500: When the reminder program ends, calculate the total duration at the end; Step 9501: If the end time reaches the preset sober duration, obtain the driver's verification facial image and execute steps 2 to 8 to obtain the verification fatigue level; Step 9502: Based on the verified fatigue level, find the corresponding reminder plan in the level plan library, and execute steps 90 to 95.

10. A fatigue driving terminal recognition and processing system integrating artificial intelligence, characterized in that, include: The acquisition module is used to acquire facial images, vehicle driving data, real-time facial images, current facial images, and verification facial images; A memory for storing a program for a fatigue driving terminal recognition and processing method integrating artificial intelligence as described in any one of claims 1 to 9; The processor loads and executes programs from memory.

Images