A method and system for alleviating driver fatigue

Abstract

The present invention relates to a method and system for alleviating fatigue driving of automobile drivers, which collects driver's facial images through a video acquisition module, collects vehicle driving index data through a vehicle controller, and determines whether the driver is in a state of facial fatigue based on a fatigue detection algorithm of a neural network. The driving stability is determined based on the vehicle driving index data, and then the driver's current fatigue driving level is determined according to the driver's facial fatigue state and driving stability. Fatigue treatment measures are implemented based on the fatigue driving level, and the fatigue treatment measures include sound stimulation, light stimulation, tactile stimulation, etc. Finally, if the fatigue treatment measures are ineffective, the automatic driving is activated to drive the vehicle to a safe area or a bumpy area that can better stimulate the driver. The present invention forms a complete, stable and reliable guarantee link from the judgment, classification, stimulation treatment to automatic emergency response of fatigue driving, which can well prevent and deal with the problem of fatigue driving and ensure driving safety.

Description

Technical Field

[0001] The present invention relates to the field of vehicle driving technology, and in particular to a method and system for alleviating fatigue driving of a car driver. Background Art

[0002] With rapid economic development, the number of vehicles on the road continues to grow rapidly. By the end of 2022, the number of motor vehicles in China had exceeded 400 million. Consequently, the number of traffic accidents and casualties caused by this has also increased daily. Among the many factors that cause traffic accidents, driver fatigue is a key and fatal one, and it is also extremely hidden. Fatigue is particularly prone to occur after lack of sleep or long periods of driving. Failure to address these situations can lead to traffic accidents. Summary of the Invention

[0003] In view of the above-mentioned defects of the prior art, the present invention proposes a method and system for alleviating fatigue driving of automobile drivers to solve the problems described in the background art.

[0004] To achieve the above-mentioned purpose, the present invention adopts the following technical solution: a method for alleviating fatigue driving of automobile drivers, comprising:

[0005] Obtain the driver's current driving data and determine whether the driver is showing signs of facial fatigue or substandard driving stability;

[0006] Determine the driver's current driving fatigue level based on their facial fatigue status and driving stability, and implement fatigue treatment measures based on the driving fatigue level;

[0007] Determine whether the fatigue treatment measures are effective. If the fatigue treatment measures are ineffective, activate automatic driving and drive the vehicle to a bumpy road section or stop in a safe area.

[0008] Furthermore, the current driving data includes driver's facial data and driving stability data, and the driving stability data includes the accuracy, timeliness and continuity of vehicle speed and direction adjustment.

[0009] Furthermore, the determination of whether the driver is in a fatigue state specifically adopts a fatigue detection algorithm based on a neural network, specifically:

[0010] Step 1: Initialize the network learning rate, randomly initialize the network parameter weights and bias values, and set the network structure;

[0011] Step 2: Randomly select facial expression images from the database and input them into the network one by one. The network is trained according to the images fed into each batch.

[0012] Step 3: The network compares the trained output value with the expected value of the image and calculates the output error;

[0013] Step 4: According to the back propagation principle, the error is back propagated and the network parameter weights and bias values ​​are adjusted;

[0014] Step 5: Determine the number of iterations. If the expected number of iterations is reached, go to step 6. Otherwise, go to step 3.

[0015] Step 6: CNN extracts features from the ROIs in the continuous video frames. The extracted sequence features are fed into the LSTM unit to learn the temporal features. The features extracted by the LSTM are fed into the Softmax classifier to perform classification training on the current video frame.

[0016] Step 7: Determine whether the face is in a fatigue state based on the image data trained by classification.

[0017] Furthermore, the facial fatigue state includes three states: blinking, yawning, and nodding.

[0018] Furthermore, determining the driver's current driving fatigue level includes:

[0019] If the driver has facial fatigue and one of the accuracy, timeliness and continuity of the driving stability data does not meet the standards, the current fatigue driving level is determined to be the first level;

[0020] If the driver has facial fatigue and the accuracy, timeliness and continuity of the driving stability data do not meet the standards, the current fatigue driving level is determined to be the second level;

[0021] If the driver has facial fatigue and the accuracy, timeliness and continuity of the driving stability data do not meet the standards, the current fatigue driving level is determined to be the third level;

[0022] Furthermore, the fatigue treatment measures implemented based on the fatigue driving level are specifically:

[0023] If the current fatigue driving level is level 1, the corresponding fatigue treatment method is to turn on the strong light in the car, play the default music, and adjust the volume to the maximum;

[0024] If the current fatigue driving level is the second level, the corresponding fatigue treatment method is to turn on the bright lights in the car, play the default music with the volume turned up to the maximum, turn on the air conditioner in the car to blow cold air to the driver, and turn on stimulating fragrance;

[0025] If the current fatigue driving level is the third level, the corresponding fatigue treatment method is to turn on the strong lights in the car, play the default music, adjust the volume to the maximum, turn on the air conditioner in the car to blow cold air to the driver, turn on stimulating fragrance, and control the steering wheel to heat up to stimulate the driver's hands.

[0026] Furthermore, the automatic driving is started to drive the vehicle to a bumpy road section or stop in a safe area, specifically:

[0027] If the fatigue driving section is a highway, the automatic driving system will control the vehicle to stop in the emergency lane or service area and turn on the hazard warning lights;

[0028] If the fatigue driving section is a national or provincial highway, the autonomous vehicle will be driven to a bumpy section of a rural road or a speed bump;

[0029] If the fatigue driving section is a rural road, the automatic driving system will control the vehicle to stop on the roadside.

[0030] Furthermore, the accuracy, timeliness and continuity of the vehicle speed and direction adjustment are specifically:

[0031] Accuracy includes speed accuracy and direction accuracy. Speed ​​accuracy means that the vehicle speed is within the prescribed range of the corresponding driving lane. If it exceeds the prescribed vehicle driving speed or is 20% lower than the prescribed vehicle driving speed for half an hour continuously, the vehicle speed accuracy is unqualified. Direction accuracy means that the vehicle driving direction is within the lane boundary. If the vehicle deviates from the lane boundary by half a meter for 10 consecutive minutes, the driving accuracy is unqualified.

[0032] Timeliness includes speed timeliness and direction timeliness. Speed ​​timeliness refers to the vehicle's reaction time to slow down or brake when the vehicle-to-vehicle distance is less than the safe distance. If the reaction time for slowing down or braking is delayed by 1 second at high speed or 3 seconds at low speed, it will be disqualified. Direction timeliness refers to the vehicle's reaction time to adjust its direction in avoiding oncoming vehicles, obstacles, or turning. If the reaction time for adjusting the direction is delayed by 1 second at high speed or 3 seconds at low speed, it will be disqualified.

[0033] Continuity includes speed continuity and direction continuity. Speed ​​continuity is the difference between the current speed of the vehicle and the speed 5 seconds before and after on the same type of road. If the difference ratio of the current speed to the speed 5 seconds before and after is greater than 10%, it is unqualified; direction continuity is the number of times the vehicle deviates from the lane within a certain period of time. In the case of high speed, it will be unqualified if it deviates three times in a row within 200 meters. In the case of low speed, it will be unqualified if it deviates three times in a row within 100 meters.

[0034] The present invention also proposes a system for alleviating fatigue driving of automobile drivers, comprising:

[0035] The video acquisition module collects the driver's facial information in real time and transmits it to the central control host for analysis of whether the driver is in a fatigue driving state;

[0036] The vehicle controller sends vehicle speed and gear information to the central control host, providing vehicle driving index data to determine whether the driver's driving stability is not up to standard;

[0037] The central control host uses the driver's facial information and vehicle driving index data to determine whether the driver is in a fatigue driving state and sends relevant instructions to each actuator;

[0038] The speaker receives the drive from the central control host and plays the relevant audio;

[0039] The display screen receives the drive from the central control host and plays the relevant video;

[0040] The air conditioning controller and body controller forward the central control host's instructions, controlling the fan, air outlet motor, fragrance module, steering wheel heater, and ambient light to execute the corresponding instructions;

[0041] The autonomous driving module executes the instructions of the central control host to switch the autonomous driving state.

[0042] The present invention has the following technical effects:

[0043] The present invention collects the driver's facial image through a video acquisition module, collects vehicle driving index data through a vehicle controller, determines whether the driver is in a state of facial fatigue based on a fatigue detection algorithm based on a neural network, determines driving stability based on the vehicle driving index data, and then determines the driver's current fatigue driving level based on the driver's facial fatigue state and driving stability, and implements fatigue treatment measures based on the fatigue driving level. Fatigue treatment measures include sound stimulation, light stimulation, tactile stimulation, etc. Finally, it determines whether the fatigue treatment measures are effective. If the fatigue treatment measures are ineffective, the automatic driving is activated to drive the vehicle to a safe area or a bumpy area that can better stimulate the driver. The present invention forms a complete, stable and reliable guarantee link from the judgment, classification, stimulation treatment to automatic emergency response of fatigue driving, which can well prevent and deal with fatigue driving problems and ensure driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings, which constitute part of the present invention, are intended to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are intended to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0045] Figure 1 Flow chart of the method of the present invention.

[0046] Figure 2 This is a system principle block diagram of the present invention.

[0047] Figure 3 This is a neural network model diagram for fatigue driving judgment of the present invention. DETAILED DESCRIPTION

[0048] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other.

[0049] The following detailed description is an exemplary description and is intended to provide further detailed description of the present invention. Unless otherwise indicated, all technical terms used in the present invention have the same meaning as those generally understood by those skilled in the art to which the present invention belongs. The terms used in the present invention are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention.

[0050] like Figure 1 As shown, a method for alleviating driver fatigue in a car includes:

[0051] Obtain the driver's current driving data and determine whether the driver is showing signs of facial fatigue or substandard driving stability;

[0052] Determine the driver's current driving fatigue level based on their facial fatigue status and driving stability, and implement fatigue treatment measures based on the driving fatigue level;

[0053] Determine whether the fatigue treatment measures are effective. If the fatigue treatment measures are ineffective, activate automatic driving and drive the vehicle to a bumpy road section or stop in a safe area.

[0054] In this embodiment, the current driving data includes the driver's facial data and driving stability data, and the driving stability data includes the accuracy, timeliness and continuity of the vehicle's speed and direction adjustment.

[0055] Accuracy includes speed accuracy and direction accuracy. Speed ​​accuracy means that the vehicle speed is within the prescribed range of the corresponding driving lane. If it exceeds the prescribed vehicle driving speed or is 20% lower than the prescribed vehicle driving speed for half an hour continuously, the vehicle speed accuracy is unqualified. Direction accuracy means that the vehicle driving direction is within the lane boundary. If the vehicle deviates from the lane boundary by half a meter for 10 consecutive minutes, the driving accuracy is unqualified.

[0056] Timeliness includes speed timeliness and direction timeliness. Speed ​​timeliness refers to the vehicle's reaction time to slow down or brake when the vehicle-to-vehicle distance is less than the safe distance. If the reaction time for slowing down or braking is delayed by 1 second at high speed or 3 seconds at low speed, it will be disqualified. Direction timeliness refers to the vehicle's reaction time to adjust its direction in avoiding oncoming vehicles, obstacles, or turning. If the reaction time for adjusting the direction is delayed by 1 second at high speed or 3 seconds at low speed, it will be disqualified.

[0057] Continuity includes speed continuity and direction continuity. Speed ​​continuity is the difference between the current speed of the vehicle and the speed 5 seconds before and after on the same type of road. If the difference ratio of the current speed to the speed 5 seconds before and after is greater than 10%, it is unqualified; direction continuity is the number of times the vehicle deviates from the lane within a certain period of time. In the case of high speed, it will be unqualified if it deviates three times in a row within 200 meters. In the case of low speed, it will be unqualified if it deviates three times in a row within 100 meters.

[0058] like Figure 3 As shown, to determine whether the driver is in a fatigue state, a fatigue detection algorithm based on a neural network is specifically used, specifically:

[0059] Step 1: Initialize the network learning rate, randomly initialize the network parameter weights and bias values; set the network structure: for example, the convolution kernel size is 5×5, and the number of samples per batch is 20;

[0060] Step 2: Randomly select facial expression images from the database and input them into the network one by one. The network is trained according to the images fed into each batch.

[0061] Step 3: The network compares the trained output value with the expected value of the image and calculates the output error;

[0062] Step 4: According to the back propagation principle, the error is back propagated and the network parameter weights and bias values ​​are adjusted;

[0063] Step 5: Determine the number of iterations. If the expected number of iterations is reached, go to step 6. Otherwise, go to step 3.

[0064] Step 6: CNN extracts features from the ROIs in the continuous video frames. The extracted sequence features are fed into the LSTM unit to learn the temporal features. The features extracted by the LSTM are fed into the Softmax classifier to perform classification training on the current video frame.

[0065] Step 7: Determine whether the facial state is fatigue based on the image data trained by classification, where facial fatigue states include blinking, yawning, and nodding.

[0066] In this embodiment, determining the driver's current fatigue driving level includes:

[0067] If the driver is in a state of facial fatigue and the accuracy, timeliness and continuity of the driving stability data do not meet the standards, the current fatigue driving level is determined to be level 1;

[0068] If the driver is in a state of facial fatigue and the accuracy, timeliness and continuity of the driving stability data do not meet the standards, the current fatigue driving level is determined to be the second level;

[0069] If the driver is experiencing facial fatigue and the driving stability data does not meet the accuracy, timeliness, and continuity standards, the current fatigue driving level is determined to be Level 3.

[0070] In this embodiment, fatigue treatment measures are implemented based on the fatigue driving level, specifically:

[0071] If the current fatigue driving level is level 1, the corresponding fatigue treatment method is to turn on the strong lights in the car, play the default music, and adjust the volume to the maximum;

[0072] If the current fatigue driving level is the second level, the corresponding fatigue treatment method is to turn on the bright lights in the car, play the default music with the volume turned up to the maximum, turn on the air conditioner to blow cold air to the driver, and turn on stimulating fragrance;

[0073] If the current fatigue driving level is the third level, the corresponding fatigue treatment method is to turn on the strong lights in the car, play the default music, adjust the volume to the maximum, turn on the air conditioner in the car to blow cold air to the driver, turn on stimulating fragrance, and control the steering wheel to heat up to stimulate the driver's hands.

[0074] In this embodiment, the automatic driving is started to drive the vehicle to a bumpy road section or stop in a safe area, specifically:

[0075] If the fatigue driving section is a highway, the automatic driving system will control the vehicle to stop in the emergency lane or service area and turn on the hazard warning lights;

[0076] If the fatigue driving section is a national or provincial highway, the autonomous vehicle will be driven to a bumpy section of a rural road or a speed bump;

[0077] If the fatigue driving section is a rural road, the automatic driving system will control the vehicle to stop on the roadside.

[0078] like Figure 2 As shown, the present invention also proposes a system for alleviating driver fatigue, comprising:

[0079] The video acquisition module collects the driver's facial information in real time and transmits it to the central control host for analysis of whether the driver is in a fatigue driving state;

[0080] The vehicle controller sends vehicle speed and gear information to the central control host, providing vehicle driving index data to determine whether the driver's driving stability is not up to standard;

[0081] The central control host uses the driver's facial information and vehicle driving index data to determine whether the driver is in a fatigue driving state and sends relevant instructions to each actuator;

[0082] The speaker receives the drive from the central control host and plays the relevant audio;

[0083] The display screen receives the drive from the central control host and plays the relevant video;

[0084] The air conditioning controller and body controller forward the central control host's instructions, controlling the fan, air outlet motor, fragrance module, steering wheel heater, and ambient light to execute the corresponding instructions;

[0085] The autonomous driving module executes the instructions of the central control host to switch the autonomous driving state.

[0086] The implementation process of the present invention is described below:

[0087] 1. When the key is in the ON position, the video acquisition module collects the driver's facial information in real time and sends it to the central control host for data analysis to determine the vehicle speed and gear information. If the speed is greater than 0 km / h and the vehicle is in gear D, the module starts to determine whether there is fatigue driving;

[0088] 2. The central control host determines the driver's current fatigue level based on facial information and driving stability data:

[0089] If the driver is in a state of facial fatigue and the accuracy, timeliness and continuity of the driving stability data do not meet the standards, the current fatigue driving level is determined to be level 1;

[0090] If the driver is in a state of facial fatigue and the accuracy, timeliness and continuity of the driving stability data do not meet the standards, the current fatigue driving level is determined to be the second level;

[0091] If the driver is in a state of facial fatigue and the accuracy, timeliness and continuity of the driving stability data do not meet the standards, the current fatigue driving level is determined to be the third level.

[0092] 3. When the central control host determines that the driver is in a fatigue driving state, it will initiate treatment measures based on the fatigue registration, specifically:

[0093] If the current fatigue driving level is the first level, the corresponding fatigue treatment method is to turn on the strong light in the car, play the default music, and adjust the volume to the maximum; after turning it on, the driver's fatigue driving will continue to be judged. If it is judged that the driver is in an excited state after being stimulated by these measures, the central control host will control the relevant states of the interior ambient light, multimedia, etc. to the memory state, which will be used as the stimulation control data for the next fatigue driving.

[0094] If the current fatigue driving level is Level 2, the corresponding fatigue treatment method is to turn on the car's strong lights, play the default music at maximum volume, turn on the car's air conditioning to blow cold air to the driver, and turn on stimulating fragrance. When turning on the cold air and fragrance, the central control unit sends a command to the air conditioning controller, which adjusts the main driver's air conditioning outlet to directly face the driver's face, sets the blowing mode to face, and opens the fan gear to maximum air volume. The cooling is turned on and set to the lowest temperature to stimulate the driver with strong cold air, and simultaneously switches to a stimulating fragrance.

[0095] If the current fatigue driving level is Level 3, the corresponding fatigue treatment method is to turn on the strong lights in the car, play the default music at maximum volume, turn on the air conditioner to blow cold air to the driver, and turn on stimulating fragrance. At the same time, the steering wheel is controlled to heat up to stimulate the driver's hands. The central control unit sends a command to the body controller. After receiving the command, the body controller adjusts the steering wheel heater to maximum power and temperature (this temperature must reach the temperature that stimulates the human body and can be calibrated). By increasing the temperature of the steering wheel, the driver's palms are stimulated to relieve the driver's fatigue, and a text reminder or voice broadcast is accompanied by the driver's fatigue reminder.

[0096] 4. Determine whether the fatigue management measures are effective. If not, activate autonomous driving mode. Specifically, the intelligent driving module intervenes and determines the driving conditions. If conditions permit, the vehicle automatically navigates to an uneven and bumpy section of road (such as an area with many speed bumps or potholes). Vibrating the vehicle up and down stimulates the driver with a strong driving experience, while providing text or voice notifications to remind the driver of fatigue. If conditions prohibit (such as on a highway), the vehicle automatically navigates to a safe area (such as an emergency lane or service area), stops, and activates the hazard warning lights.

[0097] 5. Finally, if the central control host determines that the driver has recovered from fatigue driving, the control-related modules other than the intelligent driving module will be restored to their pre-fatigue driving states, and the driver will be prompted to manually exit the automatic driving mode. After the driver confirms, the automatic driving mode will be exited.

[0098] It is understood from common technical knowledge that the present invention may be implemented by other embodiments that do not depart from its spirit or essential features. Therefore, the embodiments disclosed above are, in all respects, merely illustrative and not exclusive. All modifications within the scope of the present invention or equivalent to the scope of the present invention are intended to be encompassed by the present invention.

[0099] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0100] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0101] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0102] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the scope of protection of the claims of the present invention.

Claims

1. A method for alleviating fatigue driving of automobile drivers, characterized in that: include: Obtain the driver's current driving data and determine whether the driver is showing signs of facial fatigue or substandard driving stability; Determine the driver's current driving fatigue level based on their facial fatigue status and driving stability, and implement fatigue treatment measures based on the driving fatigue level; Determine whether fatigue treatment measures are effective. If not, activate automatic driving and drive the vehicle to a bumpy road section or stop in a safe area. The driver's current driving data includes the driver's facial data and driving stability data, wherein the driving stability data includes the accuracy, timeliness and continuity of the vehicle speed and direction adjustment; The accuracy, timeliness and continuity of the vehicle speed and direction adjustment are specifically: Accuracy includes speed accuracy and direction accuracy. Speed ​​accuracy means that the vehicle speed is within the prescribed range of the corresponding driving lane. If it exceeds the prescribed vehicle driving speed or is 20% lower than the prescribed vehicle driving speed for half an hour continuously, the vehicle speed accuracy is unqualified. Direction accuracy means that the vehicle driving direction is within the lane boundary. If the vehicle deviates from the lane boundary by half a meter for 10 consecutive minutes, the driving accuracy is unqualified. Timeliness includes speed timeliness and direction timeliness. Speed ​​timeliness refers to the vehicle's reaction time to slow down or brake when the vehicle-to-vehicle distance is less than the safe distance. If the reaction time for slowing down or braking is delayed by 1 second at high speed or 3 seconds at low speed, it will be disqualified. Direction timeliness refers to the vehicle's reaction time to adjust its direction in avoiding oncoming vehicles, obstacles, or turning. If the reaction time for adjusting the direction is delayed by 1 second at high speed or 3 seconds at low speed, it will be disqualified. Continuity includes speed continuity and direction continuity. Speed ​​continuity is the difference between the current speed of the vehicle and the speed 5 seconds before and after on the same type of road. If the difference ratio of the current speed to the speed 5 seconds before and after is greater than 10%, it is unqualified; direction continuity is the number of times the vehicle deviates from the lane within a certain period of time. In the case of high speed, it will be unqualified if it deviates three times in a row within 200 meters. In the case of low speed, it will be unqualified if it deviates three times in a row within 100 meters.

2. A method for alleviating driver fatigue according to claim 1, characterized in that: The determination of whether the driver is in a state of facial fatigue is performed by using a fatigue detection algorithm based on a neural network, specifically: Step 1: Initialize the network learning rate, randomly initialize the network parameter weights and bias values, and set the network structure; Step 2: Randomly select facial expression images from the database and input them into the network one by one. The network is trained according to the images fed into each batch. Step 3: The network compares the trained output value with the expected value of the image and calculates the output error; Step 4: According to the back propagation principle, the error is back propagated and the network parameter weights and bias values ​​are adjusted; Step 5: Determine the number of iterations. If the expected number of iterations is reached, go to step 6. Otherwise, go to step 3. Step 6: CNN extracts features from the ROIs in the continuous video frames. The extracted sequence features are fed into the LSTM unit to learn the temporal features. The features extracted by the LSTM are fed into the Softmax classifier to perform classification training on the current video frame. Step 7: Determine whether the face is in a fatigue state based on the image data trained by classification.

3. A method for alleviating driver fatigue according to claim 1 or 2, characterized in that: The facial fatigue states include blinking, yawning, and nodding.

4. The method for alleviating driver fatigue according to claim 1, characterized in that: Determining the driver's current driving fatigue level includes: If the driver has facial fatigue and one of the accuracy, timeliness and continuity of the driving stability data does not meet the standards, the current fatigue driving level is determined to be the first level; If the driver has facial fatigue and the accuracy, timeliness and continuity of the driving stability data do not meet the standards, the current fatigue driving level is determined to be the second level; If the driver is in a state of facial fatigue and the accuracy, timeliness and continuity of the driving stability data are not up to standard, the current fatigue driving level is determined to be the third level.

5. The method for alleviating driver fatigue according to claim 1, characterized in that: The fatigue treatment measures implemented based on the fatigue driving level are specifically: If the current fatigue driving level is level 1, the corresponding fatigue treatment method is to turn on the strong light in the car, play the default music, and adjust the volume to the maximum; If the current fatigue driving level is the second level, the corresponding fatigue treatment method is to turn on the bright lights in the car, play the default music with the volume turned up to the maximum, turn on the air conditioner in the car to blow cold air to the driver, and turn on stimulating fragrance; If the current fatigue driving level is the third level, the corresponding fatigue treatment method is to turn on the strong lights in the car, play the default music, adjust the volume to the maximum, turn on the air conditioner in the car to blow cold air to the driver, turn on stimulating fragrance, and control the steering wheel to heat up to stimulate the driver's hands.

6. The method for alleviating driver fatigue according to claim 1, characterized in that: The method of starting the automatic driving and driving the vehicle to a bumpy road section or stopping in a safe area is as follows: If the fatigue driving section is a highway, the automatic driving system will control the vehicle to stop in the emergency lane or service area and turn on the hazard warning lights; If the fatigue driving section is a national or provincial highway, the autonomous vehicle will be driven to a bumpy section of a rural road or a speed bump; If the fatigue driving section is a rural road, the automatic driving system will control the vehicle to stop on the roadside.

7. A system for alleviating driver fatigue driving, based on the method for alleviating driver fatigue driving according to claim 1, characterized in that: include: The video acquisition module collects the driver's facial information in real time and transmits it to the central control host for analysis of whether the driver is in a fatigue driving state; The vehicle controller sends vehicle speed and gear information to the central control host, providing vehicle driving index data to determine whether the driver's driving stability is not up to standard; The central control host uses the driver's facial information and vehicle driving index data to determine whether the driver is in a fatigue driving state and sends relevant instructions to each actuator; The speaker receives the drive from the central control host and plays the relevant audio; The display screen receives the drive from the central control host and plays the relevant video; The air conditioning controller and body controller forward the central control host's instructions, controlling the fan, air outlet motor, fragrance module, steering wheel heater, and ambient light to execute the corresponding instructions; The autonomous driving module executes the instructions of the central control host to switch the autonomous driving state.

Images