Driver attention detection method and device and storage medium

Abstract

The invention provides a driver attention detection method. The driver attention detection method comprises the following steps: S1, acquiring automatic driving level information of a vehicle and a driving speed of the vehicle; s2, when the automatic driving level of the vehicle is lower than the level L2 and the running speed of the vehicle reaches a first threshold speed, executing a first fatigue judgment step, and when the automatic driving level of the vehicle is from the level L2 to the level L4 and the running speed of the vehicle reaches a second threshold speed, executing a distraction judgment step and / or a second fatigue judgment step; and S3, correspondingly executing different alarm reminding operations and system operations according to the execution result of the first fatigue judgment step, the execution result of the distraction judgment step and the execution result of the second fatigue judgment step. The invention further provides a driver attention detection device. Compared with the prior art, the system has the advantages that the driver fatigue detection function and the distraction detection function are combined with the vehicle warning system and the vehicle operation system, and accidents caused by driver fatigue or distraction can be reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of automobile active safety, and in particular to a driver attention detection method and a detection device. Background Art

[0002] According to incomplete statistics, nearly 40% of traffic accidents are caused by driver fatigue or distracted driving. In 2024, we have included the performance requirements and test methods of driver attention detection and reminder systems into national standards, and OEMs are obliged to complete the design of driver attention detection and reminders in accordance with the standards. It can be seen that the development of driver attention detection technology, including fatigue driving detection and distracted driving detection, is very important.

[0003] At present, there are indeed many topics studying how to effectively detect and remind drivers of fatigue. However, there are many shortcomings in the prior art. First, the technical solutions in the prior art only focus on the detection of fatigue and cannot judge whether the driver is distracted. Second, the technical solutions in the prior art have few indicators for evaluating the degree of fatigue, and the actual driving situation is not considered comprehensively and the logic is not rigorous enough. Third, the technical solutions in the prior art have too few dimensions for evaluating the degree of fatigue, and the level of fatigue evaluation is only fatigue or non-fatigue, but in fact, the driver's fatigue can be divided into many levels, and it cannot be roughly evaluated by fatigue and non-fatigue. Fourth, the technical solutions in the prior art do not make adaptive changes to the changes in the driving state of the vehicle when performing fatigue detection. The automatic driving state of the vehicle includes multiple different levels from L0 to L5. When performing fatigue detection, it should be properly distinguished, but the technical solutions in the prior art do not make such distinctions. Fifth, there is no complete set of evaluation criteria for driver fatigue detection or distraction detection. The reason for this defect is that the technicians are more based on theoretical research, lack product development and production experience, and do not give enough consideration to the actual driving conditions of the vehicle.

[0004] Taking the patent publication number CN108545080A in the prior art as an example, the document discloses a driver fatigue detection method and system, which determines whether the driver is fatigued by comprehensive analysis of three indicators: yawning, closing eyes and driving time. However, the driver fatigue detection method and system have the above-mentioned five disadvantages.

[0005] Therefore, there is an urgent need to develop a more reasonable and practical driver attention detection method and a driver attention detection device. Summary of the invention

[0006] The purpose of the present invention is to provide a driver attention detection method to address the deficiencies of the prior art, which comprises the following steps:

[0007] Step S1, obtaining vehicle automatic driving level information and vehicle driving speed;

[0008] Step S2: when the vehicle's autonomous driving level is lower than level L2 and the vehicle's driving speed reaches a first threshold speed, a first fatigue determination step is performed; when the vehicle's autonomous driving level is between level L2 and level L4 and the vehicle's driving speed reaches a second threshold speed, a distraction determination step and / or a second fatigue determination step is performed;

[0009] Step S3, performing different alarm reminder operations and system operations according to the execution results of the first fatigue determination step, the distraction determination step and the second fatigue determination step.

[0010] Preferably, the first fatigue determination step includes:

[0011] Step S11, obtaining eye feature information, mouth feature information and behavior feature information of the driver, and analyzing the obtained feature information;

[0012] Step S12: when the Perclos value exceeds the reference value B31, or the time the driver keeps closing his eyes exceeds the reference value B41, or the driver is identified as yawning, it is determined that the driver is in a fatigue driving state based on camera recognition; when the Perclos value is not higher than the reference value B31, and the time the driver keeps closing his eyes is not higher than the reference value B41, and the driver is not identified as yawning, it is determined that the driver is in a non-fatigue driving state based on camera recognition;

[0013] Step S13, obtaining continuous driving time data and a message signal triggered by the driver;

[0014] Step S14: when the continuous driving time does not exceed 4 hours and there is a message signal triggered by the driver, it is determined that the driver is in a non-fatigue driving state based on behavior judgment; when the continuous driving time is more than 4 hours, or no message signal triggered by the driver is detected within the continuous time C1, it is determined that the driver is in a fatigue driving state based on behavior judgment;

[0015] Step S15: When it is determined that the driver is in a non-fatigue driving state based on camera recognition and the driver is in a non-fatigue driving state based on behavior judgment, the driver is finally determined to be in a non-fatigue driving state; when it is determined that the driver is in a fatigue driving state based on camera recognition or the driver is in a fatigue driving state based on behavior judgment, the driver is finally determined to be in a fatigue driving state.

[0016] Preferably, the second fatigue determination step includes:

[0017] Step S21, obtaining eye feature information, mouth feature information and behavior feature information of the driver, and analyzing the obtained feature information;

[0018] Step S22: when the Perclos value is not higher than the reference value B32, and the driver's continuous eye-closing time is not higher than the reference value B42, it is determined that the driver is in a non-fatigue driving state based on camera recognition;

[0019] When the Perclos value exceeds the reference value B32 and is not higher than the reference value B33, and the driver's continuous closing time is not higher than the reference value B42, or when the driver is identified to be yawning, it is determined that the driver is in a state of mild fatigue driving based on camera recognition;

[0020] When the Perclos value exceeds the reference value B33 and is not higher than the reference value B34, and the driver's continuous eye-closing time is not higher than the reference value B42, the driver is determined to be in a moderate fatigue driving state based on camera recognition;

[0021] When the Perclos value exceeds the reference value B34, or the driver's continuous closing time is higher than the reference value B42, the driver is judged to be in a state of severe fatigue driving based on camera recognition;

[0022] Step S23, obtaining continuous driving duration data and a message signal triggered by the driver;

[0023] Step S24: when the continuous driving time does not exceed 4 hours and there is a message signal triggered by the driver, it is determined that the driver is in a non-fatigue driving state based on behavior judgment; when the continuous driving time is more than 4 hours, or no message signal triggered by the driver is detected for the continuous time C2, it is determined that the driver is in a fatigue driving state based on behavior judgment;

[0024] Step S25: when it is determined that the driver is in a non-fatigue driving state based on camera recognition and the driver is in a non-fatigue driving state based on behavior judgment, it is finally determined that the driver is in a non-fatigue driving state;

[0025] When the driver is determined to be in a non-fatigue driving state based on camera recognition and is determined to be in a fatigue driving state based on behavior judgment, the driver is finally determined to be in a mildly fatigued driving state;

[0026] When it is determined that the driver is in a state of mild driving fatigue based on camera recognition, the driver is finally determined to be in a state of mild driving fatigue;

[0027] When it is determined that the driver is in a moderate fatigue driving state based on camera recognition, the driver is finally determined to be in a moderate fatigue driving state;

[0028] When the driver is determined to be in a state of severe driving fatigue based on camera recognition and is determined to be in a state of non-fatigue driving based on behavior judgment, the driver is finally determined to be in a state of moderate driving fatigue;

[0029] When it is determined that the driver is in a severe fatigue driving state based on camera recognition and the driver is in a fatigue driving state based on behavior judgment, it is finally determined that the driver is in a severe fatigue driving state.

[0030] Preferably, by adjusting the reference value B32, the reference value B33, or the reference value B34, the evaluation standard of the fatigue degree can be changed.

[0031] Preferably, the message signal triggered by the driver includes one or more of a steering controller message signal, a turn signal message signal, an accelerator pedal message signal, a comfort system message signal, and an infotainment system message signal.

[0032] Preferably, the distraction determination step includes:

[0033] Step S31, obtaining eye feature information, mouth feature information and behavior feature information of the driver, and analyzing the obtained feature information;

[0034] Step S32: when the PRC value is less than the PRC reference value A2, or when the driver does not look at the road for a period of time exceeding the reference value A3, or when a smoking action is detected within a fixed time period, or when a drinking action is detected within a fixed time period, or when a hand-held phone call action is detected within a fixed time period, it is determined that the driver is in a distracted driving state;

[0035] When the PRC value is not lower than the PRC reference value A2, and when the driver's continuous time without looking at the road is not higher than the reference value A3, and no smoking, drinking, or hand-held phone calls are detected within a fixed time period, the driver is determined to be in a non-distracted driving state.

[0036] Preferably, the alarm reminder operation includes at least one of a continuous text warning and a sound warning, and the system operation includes one or more of turning off the automatic driving assistance system, braking the vehicle, tightening the seat belt, closing the windows and sunroof, turning on the hazard lights, turning on the interior lights, continuously honking the horn, and slowing the vehicle down to a stop.

[0037] The present invention also provides a driver attention detection device, which is used to execute the above-mentioned driver attention detection method.

[0038] Preferably, the detection device includes a judgment control module, a camera module, a feature analysis module, a warning module, and a vehicle data acquisition module, wherein the feature analysis module, the warning module, and the vehicle driving data collection module are respectively connected to the control module; the judgment control module is used to implement the logical judgment in the driver attention detection method; the camera module is used to obtain the driver's driving status; the feature analysis module is used to extract and analyze the driver's feature information, and is used to perform data training on the model and define basic parameters; the vehicle data acquisition module is used to collect information from various vehicle controllers; and the warning module is used to send an alarm reminder to the driver.

[0039] The present invention also provides a computer-readable storage medium storing a computer program, which is used to implement the above-mentioned driver attention detection method when executed by a processor.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] 1. The fatigue situation faced by car drivers is very complicated. The solution of the present invention takes into account sufficient indicators and reasonable judgment methods to determine whether the driver is fatigued and distracted.

[0042] 2. Driver distraction is also a cause of traffic accidents. The solution of the present invention not only takes into account the driver's fatigue, but the technical solution of the present invention can also detect whether the driver is in a distracted driving state.

[0043] 3. In the prior art, fatigue levels are divided into fatigue and non-fatigue, but there is no detailed division of the specific fatigue levels. The solution of the present invention divides the driver's fatigue state into four levels in detail, and each level has a clear definition.

[0044] 4. In the current prior art, the driver's fatigue detection function and distraction detection function are not applied to the vehicle warning system and vehicle operating system. However, the solution of the present invention achieves this. The solution of the present invention can warn the driver according to the detection results, and can also control the vehicle operating system to take emergency measures after the warning, so as to avoid accidents caused by driver fatigue or distraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flow chart of a driver attention detection method in Embodiment 1 of the present invention;

[0046] Figure 2 is a flow chart of the first fatigue determination step in the first embodiment of the present invention;

[0047] Figure 3is a flow chart of the second fatigue determination step in the first embodiment of the present invention;

[0048] Figure 4 is a flow chart of the distraction determination step in the first embodiment of the present invention;

[0049] Figure 5 is a schematic diagram of the structure of a driver attention detection device in Embodiment 2 of the present invention;

[0050] Figure 6 is the determination table 1 of step S15 of the present invention;

[0051] Figure 7 This is the determination table 2 of step S25 of the present invention.

[0052] Judgment control module 1, camera module 2, feature analysis module 3, warning module 4, vehicle data collection module 5 and vehicle control modules 6. DETAILED DESCRIPTION

[0053] The above and other technical features and advantages of the present invention are described in more detail below in conjunction with the accompanying drawings.

[0054] In the description of the present invention, it should be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, unless otherwise clearly and specifically defined.

[0055] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral one; it can be a mechanical connection, an electrical connection, or communication with each other; it can be a direct connection, or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0056] In the description of this specification, reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention.

[0057] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0058] In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.

[0059] In order to illustrate the technical solution of the present invention, a specific embodiment is provided below for illustration.

[0060] Before elaborating on the technical solution of this embodiment, the various levels of autonomous driving are defined in advance. The autonomous driving levels are divided into 6 levels from L5 to L6. The definitions of each level are as follows:

[0061] Level 0 (no automation): The car is fully operated by the human driver and can be assisted by warning and protection systems during driving.

[0062] Level 1 (Driving Assistance): Driving assistance is provided for one of the operations of steering and acceleration and deceleration through the driving environment. Other driving actions are performed by human drivers.

[0063] Level 2 (partial automation): Driving assistance is provided for multiple operations such as steering and acceleration and deceleration through the driving environment, and other driving actions are performed by human drivers.

[0064] Level 3 (conditional automation): All driving operations are completed by the unmanned driving system. The human driver provides appropriate responses based on system requests.

[0065] Level 4 (High Automation): All driving operations are completed by the unmanned driving system. Depending on the system request, the human driver does not necessarily need to respond to all system requests, that is, the vehicle drives completely autonomously within a specific geographical range (geofence) or conditions without driver intervention.

[0066] Level 5 (Full Automation): All driving operations are performed by the unmanned driving system. The human driver takes over when possible. The vehicle is capable of autonomous driving under all road and environmental conditions.

[0067] like Figure 1 As shown, this embodiment provides a driver attention detection method, which includes the following steps:

[0068] Step S1, obtaining vehicle automatic driving level information and vehicle driving speed;

[0069] Step S2: when the vehicle's autonomous driving level is lower than level L2 and the vehicle's driving speed reaches a first threshold speed, a first fatigue determination step is performed; when the vehicle's autonomous driving level is between level L2 and level L4 and the vehicle's driving speed reaches a second threshold speed, a distraction determination step and / or a second fatigue determination step is performed;

[0070] Step S3, performing different alarm reminder operations and system operations according to the execution results of the first fatigue determination step, the distraction determination step and the second fatigue determination step.

[0071] The reasons for different levels of autonomous driving in the above steps are:

[0072] When the vehicle's autonomous driving level is at level L0 and L1, according to the functional safety definition and ADA, ASIL, and QM functional requirements, the driver monitoring function only needs to be responsible for reminders, and the system does not need to take subsequent operations of the vehicle system. When the vehicle's autonomous driving level is at level L2 and above (except L5), the premise of this level of autonomous driving is that the driver is ready to take over the vehicle at any time. Therefore, the system needs to monitor the driver's status at any time and take certain system operations when the driver is severely fatigued to minimize the harm to the driver.

[0073] The above ADA is the abbreviation of Advanced Driver Assistance, which means driving assistance. The above ASIL is the abbreviation of Automotive Safety Integrity Level, which means automobile safety integrity level. The above QM is the abbreviation of Quality Management, which means quality management.

[0074] The setting of the first threshold speed and the second threshold speed is used to better realize the start-up timing of fatigue determination and distraction determination. Preferably, the first threshold speed and the second threshold speed are 60km / h. Under urban road conditions, it is generally difficult to increase the driving speed to 60km / h, and drivers rarely suffer from severe fatigue in daily urban road conditions, so the speed is set to 60km / h to prevent reminders from interfering with the driver's driving during daily commuting. In addition, speeds above 60km / h are already high-speed driving, which requires a high level of driver status, so the system is limited to 60km / h, while at the same time protecting the driver to the greatest extent possible to prevent safety problems caused by fatigue and distraction.

[0075] like Figure 2 As shown, the first fatigue determination step includes:

[0076] Step S11, obtaining eye feature information, mouth feature information and behavior feature information of the driver, and analyzing the obtained feature information;

[0077] Step S12: when the Perclos value exceeds the reference value B31, or the time the driver keeps closing his eyes exceeds the reference value B41, or the driver is identified as yawning, it is determined that the driver is in a fatigue driving state based on camera recognition; when the Perclos value is not higher than the reference value B31, and the time the driver keeps closing his eyes is not higher than the reference value B41, and the driver is not identified as yawning, it is determined that the driver is in a non-fatigue driving state based on camera recognition;

[0078] Step S13, obtaining continuous driving time data and a message signal triggered by the driver;

[0079] Step S14: when the continuous driving time does not exceed 4 hours and there is a message signal triggered by the driver, it is determined that the driver is in a non-fatigue driving state based on behavior judgment; when the continuous driving time is more than 4 hours, or no message signal triggered by the driver is detected within the continuous time C1, it is determined that the driver is in a fatigue driving state based on behavior judgment;

[0080] Step S15: When it is determined that the driver is in a non-fatigue driving state based on camera recognition and the driver is in a non-fatigue driving state based on behavior judgment, the driver is finally determined to be in a non-fatigue driving state; when it is determined that the driver is in a fatigue driving state based on camera recognition or the driver is in a fatigue driving state based on behavior judgment, the driver is finally determined to be in a fatigue driving state.

[0081] When the driver is judged to be in a fatigue driving state, the HMI is driven to provide continuous text warnings and sound warnings at the same time. HMI is the abbreviation of Human-Machine Interface, which means human-machine interaction interface.

[0082] Description of relevant parameters in the first fatigue determination step. The eye parameters include the Perclos value and the time the driver keeps his eyes closed. The Perclos value represents the proportion of time that the eyelids meet the PB2 condition within any continuous time B1. The PB2 condition means that the area of ​​the pupil covered by the eyelids exceeds B2, which is considered as eye closure. B2 is generally a percentage. The Perclos value here corresponds to the reference value B31. For B1, the OEM can define it according to the user's sensitivity needs. The reference value B31 is preferably 10%, and the OEM can define it according to the user's sensitivity needs. The reference value B41 of the time the driver keeps his eyes closed (EyesClosed Duration) is preferably 2 seconds, and the OEM can also use other values ​​according to user needs. In addition, the mouth parameters include the mouth width-to-height ratio, which is the ratio of the vertical height of the inner edge of the lips to the horizontal width of the corners of the mouth. When the mouth width-to-height ratio is greater than B5 and the duration is greater than B6, it is determined that the driver has yawned. Preferably, the value of B5 is 0.6.

[0083] The continuous driving time in step S14 refers to the time since the last time the vehicle was powered off. According to Article 62 of the "Road Traffic Safety Law Implementation Regulations": If a motor vehicle is driven continuously for more than 4 hours, it should be stopped for rest, and the rest time should be no less than 20 minutes. Therefore, the threshold is set to 4 hours. The message signal triggered by the driver in step S14 includes but is not limited to one or more of the steering controller message signal, the turn signal message signal, the accelerator pedal message signal, the comfort system message signal, and the infotainment system message signal. Take any continuous unit time C1 to determine whether there is one or more of the steering controller message signal, the turn signal message signal, the accelerator pedal message signal, the comfort system message signal, and the infotainment system message signal.

[0084] In step S15, when it is determined that the driver is in a non-fatigued driving state based on camera recognition and when it is determined that the driver is in a non-fatigued driving state based on behavior judgment, it is natural to determine that the driver is not fatigued. Similarly, when it is determined that the driver is in a fatigued driving state based on camera recognition and when it is determined that the driver is in a fatigued driving state based on behavior judgment, it can be directly determined that the driver is fatigued. When the two determination results are inconsistent, in order to reduce the situation where the driver is fatigued but not recognized, as long as either party determines that the driver is fatigued, then the driver will eventually be determined to be fatigued, thereby maximizing the safety of the system. The determination of step S15 is based on Figure 6As shown in Table 1.

[0085] like Figure 3 As shown, the second fatigue determination step includes:

[0086] Step S21, obtaining eye feature information, mouth feature information and behavior feature information of the driver, and analyzing the obtained feature information;

[0087] Step S22: when the Perclos value is not higher than the reference value B32, and the driver's continuous eye-closing time is not higher than the reference value B42, it is determined that the driver is in a non-fatigue driving state based on camera recognition;

[0088] When the Perclos value exceeds the reference value B32 and is not higher than the reference value B33, and the driver's continuous closing time is not higher than the reference value B42, or when the driver is identified to be yawning, it is determined that the driver is in a state of mild fatigue driving based on camera recognition;

[0089] When the Perclos value exceeds the reference value B33 and is not higher than the reference value B34, and the driver's continuous eye-closing time is not higher than the reference value B42, the driver is determined to be in a moderate fatigue driving state based on camera recognition;

[0090] When the Perclos value exceeds the reference value B34, or the driver's continuous closing time is higher than the reference value B42, the driver is judged to be in a state of severe fatigue driving based on camera recognition;

[0091] Step S23, obtaining continuous driving time data and a message signal triggered by the driver;

[0092] Step S24: when the continuous driving time does not exceed 4 hours and there is a message signal triggered by the driver, it is determined that the driver is in a non-fatigue driving state based on behavior judgment; when the continuous driving time is more than 4 hours, or no message signal triggered by the driver is detected for the continuous time C2, it is determined that the driver is in a fatigue driving state based on behavior judgment;

[0093] Step S25: when it is determined that the driver is in a non-fatigue driving state based on camera recognition and the driver is in a non-fatigue driving state based on behavior judgment, it is finally determined that the driver is in a non-fatigue driving state;

[0094] When the driver is determined to be in a non-fatigue driving state based on camera recognition and is determined to be in a fatigue driving state based on behavior judgment, the driver is finally determined to be in a mildly fatigued driving state;

[0095] When it is determined that the driver is in a state of mild driving fatigue based on camera recognition, the driver is finally determined to be in a state of mild driving fatigue;

[0096] When it is determined that the driver is in a moderate fatigue driving state based on camera recognition, the driver is finally determined to be in a moderate fatigue driving state;

[0097] When the driver is determined to be in a state of severe driving fatigue based on camera recognition and is determined to be in a state of non-fatigue driving based on behavior judgment, the driver is finally determined to be in a state of moderate driving fatigue;

[0098] When it is determined that the driver is in a severe fatigue driving state based on camera recognition and the driver is in a fatigue driving state based on behavior judgment, it is finally determined that the driver is in a severe fatigue driving state.

[0099] When it is determined that the driver is in a non-fatigued driving state based on camera recognition and is in a non-fatigued driving state based on behavior judgment, it is natural to determine that the driver is not fatigued. At this time, the system will not have any reminders or behaviors.

[0100] When the driver is determined to be in a state of severe fatigue driving based on camera recognition and the driver is determined to be in a state of fatigue driving based on behavior judgment, the driver is directly determined to be severely fatigued. At this time, the vehicle will continue to give text, sound and vibration reminders to try to wake up the driver. If the driver is still detected to be severely fatigued after the alarm prompt, the system will provide the driver with the possibility of self-rescue or rescue by others to the greatest extent. Close the windows and sunroof, brake the car and the infotainment system repeatedly gives the driver voice prompts to try to wake the driver. Tighten the seat belt to prevent injury to the driver during the braking process. Turn on the double flash and continue to sound the horn to remind surrounding vehicles to pay attention to avoid. Turn on the interior lighting so that surrounding vehicles can pay attention to the status of the driver in the car and help take alarm measures. At the same time, slow down the vehicle until it stops to prevent the car from continuing to drive, turn off the automatic driving assistance system, and finally stop the vehicle.

[0101] The definitions of the four different levels of fatigue are as follows: When the driver is very awake and does not feel any fatigue, the driver is in a non-fatigue driving state. When the driver feels fatigued but it does not affect driving, the driver is in a mild fatigue driving state. When the driver is trying to stay awake and it affects driving, the driver is in a moderate fatigue driving state. When the driver is considered to be incapable of driving, the driver is in a severe fatigue driving state. The state of fatigue has a gradual increase, from never tired to mild fatigue to moderate fatigue and finally severe fatigue. There will not be a situation where the state of never tired goes directly to the state of severe fatigue. However, from a state of high fatigue, any state of low fatigue can be switched at any time. For example, the state of severe fatigue can be converted into any state of moderate fatigue, mild fatigue or no fatigue. Assuming that the driver is in severe fatigue and has no driving ability at this time, the driver may become awake after the reminder and operation of the vehicle system, so the driver's state may become non-fatigue at this time. The same applies to other states.

[0102] When the driver is identified as mildly fatigued, the driver just feels tired and does not affect the driving system, so only a text reminder is needed to remind the driver to adjust the fatigue state. When the driver is identified as moderately fatigued, the driver's efforts to stay awake will affect driving. Therefore, the vehicle system will continue to provide text and voice reminders. If the driver is still detected to be moderately fatigued after the alarm prompt, the infotainment system will repeatedly give the driver voice prompts to try to further remind the driver; turn off the automatic driving assistance system, and the system brakes and slows down to prevent accidents caused by excessive speed.

[0103] When it is finally determined that the driver is in a state of mild fatigue driving, the HMI is driven to provide continuous text warnings.

[0104] When it is finally determined that the driver is in a moderate fatigue driving state, the HMI is driven to provide continuous text warnings and sound warnings at the same time. After the alarm prompts, the system operation is executed to turn off the automatic driving assistance system and start engine braking.

[0105] When it is finally determined that the driver is in a state of severe driving fatigue, text warnings, sound warnings and steering wheel vibration warnings are continuously issued. After the alarm prompts, system operations are executed, including turning off the automatic driving assistance system, braking the vehicle, tightening the seat belts, closing the windows and sunroof, turning on the double flash, turning on the interior lighting, continuously honking the horn, and slowing the vehicle down until it stops.

[0106] The continuous driving time in step S24 refers to the time since the last time the vehicle was powered off. According to Article 62 of the "Road Traffic Safety Law Implementation Regulations": If a motor vehicle is driven continuously for more than 4 hours, it should be stopped for rest, and the rest time should be no less than 20 minutes. Therefore, the threshold is set to 4 hours. The message signal triggered by the driver in step S24 includes but is not limited to one or more of the steering controller message signal, the turn signal message signal, the accelerator pedal message signal, the comfort system message signal, and the infotainment system message signal. Take any continuous unit time C2 to determine whether there is one or more of the steering controller message signal, the turn signal message signal, the accelerator pedal message signal, the comfort system message signal, and the infotainment system message signal.

[0107] Description of relevant parameters in the second fatigue determination step. The eye parameters include the Perclos value and the time the driver keeps his eyes closed. The Perclos value represents the proportion of time that the eyelids meet the PB2 condition within any continuous time B1. The PB2 condition means that the area of ​​the pupil covered by the eyelids exceeds B2, which is considered as eye closure. B2 is generally a percentage value. The Perclos value here corresponds to the reference value B32, reference value B33 and reference value B34. For B1, the OEM can define it according to the user's sensitivity needs. The reference value B32 is preferably 10%, the reference value B33 is preferably 20%, and the reference value B34 is preferably 40%. The OEM can define it according to the user's sensitivity needs. The reference value B41 of the time the driver keeps his eyes closed (EyesClosed Duration) is preferably 2 seconds, and the OEM can also use other values ​​according to user needs. In addition, the mouth parameters include the mouth width-to-height ratio, which is the ratio of the vertical height of the inner edge of the lips to the horizontal width of the mouth corners. When the ratio of the height to width of the open mouth is greater than B5 and the duration is greater than B6, it is determined that the driver yawns. Preferably, the value of B5 is 0.6.

[0108] In addition, the definitions of Perclos and PRC are based on the indicator information within any continuous time. When the vehicle speed just reaches 60km / h and the driver attention detection is just started, if the requirement of any continuous time cannot be met, it is considered that when the driver accelerates the vehicle to 60km / h, the driver will not be extremely tired. Therefore, at this stage, we assume that the Perclos value is ≤ the reference value B31 and the PRC value is ≥ the reference value A2.

[0109] The determination in step S25 is based on Figure 7 As shown in Table 2.

[0110] like Figure 4 As shown, the distraction determination steps include:

[0111] Step S31, obtaining eye feature information, mouth feature information and behavior feature information of the driver, and analyzing the obtained feature information;

[0112] Step S32: when the PRC value is less than the PRC reference value A2, or when the driver does not look at the road for a period of time exceeding the reference value A3, or when a smoking action is detected within a fixed time period, or when a drinking action is detected within a fixed time period, or when a hand-held phone call action is detected within a fixed time period, it is determined that the driver is in a distracted driving state;

[0113] When the PRC value is not lower than the PRC reference value A2, and when the driver's continuous time without looking at the road is not higher than the reference value A3, and no smoking, drinking, or hand-held phone calls are detected within a fixed time period, the driver is determined to be in a non-distracted driving state.

[0114] When the driver is determined to be in a distracted driving state, the HMI is driven to provide continuous text warnings and sound warnings at the same time. HMI is the abbreviation of Human-Machine Interface, which means human-machine interaction interface.

[0115] Distraction is defined as the driver's attention not being focused on driving. Not distracted is defined as the driver's attention being focused on driving. The distraction determination step has only two results: distracted and not distracted. The reason for only distinguishing between distracted and not distracted levels is that the driver is only distracted, not incapable of action. The driver can take over the steering wheel at any time. Therefore, in the case of distraction, the vehicle only needs to serve as a reminder to the driver. If the system intervenes too much, it will cause user complaints and will also affect the driver's driving to a certain extent. The same reason applies to L0 and L1 level autonomous driving.

[0116] The above PRC is the abbreviation of Percent Road Center, which represents the percentage of time the driver looks at the road within a continuous unit time of A1 seconds. Regarding A1, the OEM can define it according to the user's sensitivity needs. The PRC reference value A2 is preferably 60%, and the OEM can also use other values ​​according to user needs. The reference value A3 of the time the driver does not look at the road (EyesOffRoad Duration) is preferably 2s, and the OEM can also use other values ​​according to user needs.

[0117] Description of behavioral postures. Regarding smoking actions, when a column with cigarette characteristics is held in the mouth or a column with cigarette characteristics is held close to the mouth by the hand, it is determined as smoking behavior. Regarding drinking water actions, when a cup is held close to the mouth, partially or completely covering the mouth and drinking water, it is determined as drinking water behavior. Regarding handheld phone calls, when a smartphone is held close to the ear as if making a phone call, it is determined as a phone call behavior. The fixed time period is preferably 2s, and the OEM can also use other values ​​according to user needs.

[0118] The driver attention detection method provided in this embodiment has the following beneficial effects:

[0119] 1. The fatigue situation faced by car drivers is very complicated. The solution of the present invention takes into account sufficient indicators and reasonable judgment methods to determine whether the driver is fatigued and distracted.

[0120] 2. Driver distraction is also a cause of traffic accidents. The solution of the present invention not only takes into account the driver's fatigue, but the technical solution of the present invention can also detect whether the driver is in a distracted driving state.

[0121] 3. In the prior art, fatigue levels are divided into fatigue and non-fatigue, but there is no detailed division of the specific fatigue levels. The solution of the present invention divides the driver's fatigue state into four levels in detail, and each level has a clear definition.

[0122] 4. In the current prior art, the driver's fatigue detection function and distraction detection function are not applied to the vehicle warning system and vehicle operating system. However, the solution of the present invention achieves this. The solution of the present invention can warn the driver according to the detection results, and can also control the vehicle operating system to take emergency measures after the warning, so as to avoid accidents caused by driver fatigue or distraction.

[0123] Embodiment 2

[0124] The difference between this embodiment and the first embodiment is that: Figure 5 As shown, this embodiment provides a driver attention detection device, which includes a judgment control module 1, a camera module 2, a feature analysis module 3, a warning module 4, a vehicle data acquisition module 5 and various vehicle control modules 6.

[0125] The judgment control module 1 is respectively connected with the feature analysis module 3, the warning module 4, the vehicle data collection module 5 and the vehicle control modules 6. The camera module 2 is connected with the feature analysis module 3.

[0126] The judgment control module 1 is used to implement the logical judgment in the driver attention detection method described in the first embodiment.

[0127] The camera module 2 is used to obtain the driving status of the driver.

[0128] The feature analysis module 3 is used to extract and analyze the driver's feature information, and to perform a large amount of data training on the model and define basic parameters.

[0129] The warning module 4 is used to send warning reminder information to the driver.

[0130] The vehicle data acquisition module 5 is used to collect information from various vehicle controllers, including one or more of a steering controller message signal, a turn signal message signal, an accelerator pedal message signal, a comfort system message signal, and an infotainment system message signal.

[0131] The vehicle control modules 6 include brake, door, window, horn and other control modules.

[0132] The driver attention detection device can execute all the steps of the driver attention detection method in the first embodiment.

[0133] Embodiment 3

[0134] The difference between this embodiment and the first embodiment is that:

[0135] By adjusting the reference value B31, reference value B32, reference value B33, and reference value B34, the judgment criteria of the four fatigue levels can be changed. The OEM can make adjustments.

[0136] In addition, the reference values ​​such as A2, A3, B41, B42 and the values ​​such as A1, B1, B2, B5, B6 in this embodiment are the same as those described in the first embodiment and are also adjustable variables.

[0137] Embodiment 4

[0138] The difference between this embodiment and the first embodiment is that: this embodiment provides a computer-readable storage medium storing a computer program, which is used to implement all the steps of the driver attention detection method in the first embodiment when executed by a processor.

[0139] The above are only specific embodiments of the present invention, but the technical features of the present invention are not limited thereto. Any simple changes, equivalent replacements or modifications made based on the present invention to solve basically the same technical problems and achieve basically the same technical effects are all included in the protection scope of the present invention.

Claims

1. A driver attention detection method, characterized in that: It includes the following steps: Step S1, obtaining vehicle automatic driving level information and vehicle driving speed; Step S2: when the vehicle's autonomous driving level is lower than level L2 and the vehicle's driving speed reaches a first threshold speed, a first fatigue determination step is performed; when the vehicle's autonomous driving level is between level L2 and level L4 and the vehicle's driving speed reaches a second threshold speed, a distraction determination step and / or a second fatigue determination step is performed; Step S3, performing different alarm reminder operations and system operations according to the execution results of the first fatigue determination step, the distraction determination step and the second fatigue determination step.

2. A driver attention detection method as claimed in claim 1, characterized in that: The first fatigue determination step comprises: Step S11, obtaining eye feature information, mouth feature information and behavior feature information of the driver, and analyzing the obtained feature information; Step S12: when the Perclos value exceeds the reference value B31, or the time the driver keeps closing his eyes exceeds the reference value B41, or the driver is identified as yawning, it is determined that the driver is in a fatigue driving state based on camera recognition; when the Perclos value is not higher than the reference value B31, and the time the driver keeps closing his eyes is not higher than the reference value B41, and the driver is not identified as yawning, it is determined that the driver is in a non-fatigue driving state based on camera recognition; Step S13, obtaining continuous driving time data and a message signal triggered by the driver; Step S14: when the continuous driving time does not exceed 4 hours and there is a message signal triggered by the driver, it is determined that the driver is in a non-fatigue driving state based on behavior judgment; when the continuous driving time is more than 4 hours, or no message signal triggered by the driver is detected within the continuous time C1, it is determined that the driver is in a fatigue driving state based on behavior judgment; Step S15: When it is determined that the driver is in a non-fatigue driving state based on camera recognition and the driver is in a non-fatigue driving state based on behavior judgment, the driver is finally determined to be in a non-fatigue driving state; when it is determined that the driver is in a fatigue driving state based on camera recognition or the driver is in a fatigue driving state based on behavior judgment, the driver is finally determined to be in a fatigue driving state.

3. A driver attention detection method as claimed in claim 1, characterized in that: The second fatigue determination step comprises: Step S21, obtaining eye feature information, mouth feature information and behavior feature information of the driver, and analyzing the obtained feature information; Step S22: when the Perclos value is not higher than the reference value B32, and the driver's continuous eye-closing time is not higher than the reference value B42, it is determined that the driver is in a non-fatigue driving state based on camera recognition; When the Perclos value exceeds the reference value B32 and is not higher than the reference value B33, and the driver's continuous closing time is not higher than the reference value B42, or when the driver is identified to be yawning, it is determined that the driver is in a state of mild fatigue driving based on camera recognition; When the Perclos value exceeds the reference value B33 and is not higher than the reference value B34, and the driver's continuous eye-closing time is not higher than the reference value B42, the driver is determined to be in a moderate fatigue driving state based on camera recognition; When the Perclos value exceeds the reference value B34, or the driver's continuous closing time is higher than the reference value B42, the driver is judged to be in a state of severe fatigue driving based on camera recognition; Step S23, obtaining continuous driving duration data and a message signal triggered by the driver; Step S24: when the continuous driving time does not exceed 4 hours and there is a message signal triggered by the driver, it is determined that the driver is in a non-fatigue driving state based on behavior judgment; when the continuous driving time is more than 4 hours, or no message signal triggered by the driver is detected for the continuous time C2, it is determined that the driver is in a fatigue driving state based on behavior judgment; Step S25: when it is determined that the driver is in a non-fatigue driving state based on camera recognition and the driver is in a non-fatigue driving state based on behavior judgment, it is finally determined that the driver is in a non-fatigue driving state; When the driver is determined to be in a non-fatigue driving state based on camera recognition and is determined to be in a fatigue driving state based on behavior judgment, the driver is finally determined to be in a mildly fatigued driving state; When it is determined that the driver is in a state of mild driving fatigue based on camera recognition, the driver is finally determined to be in a state of mild driving fatigue; When it is determined that the driver is in a moderate fatigue driving state based on camera recognition, the driver is finally determined to be in a moderate fatigue driving state; When the driver is determined to be in a state of severe driving fatigue based on camera recognition and is determined to be in a state of non-fatigue driving based on behavior judgment, the driver is finally determined to be in a state of moderate driving fatigue; When it is determined that the driver is in a severe fatigue driving state based on camera recognition and the driver is in a fatigue driving state based on behavior judgment, it is finally determined that the driver is in a severe fatigue driving state.

4. A driver attention detection method as claimed in claim 3, characterized in that: By adjusting the reference value B32, the reference value B33, or the reference value B34, the evaluation standard of the fatigue degree can be changed.

5. A driver attention detection method as claimed in claim 2 or 3, characterized in that: The message signal triggered by the driver includes one or more of a steering controller message signal, a turn signal message signal, an accelerator pedal message signal, a comfort system message signal, and an infotainment system message signal.

6. A driver attention detection method as claimed in claim 1, characterized in that: The distraction determination step comprises: Step S31, obtaining eye feature information, mouth feature information and behavior feature information of the driver, and analyzing the obtained feature information; Step S32: when the PRC value is less than the PRC reference value A2, or when the driver does not look at the road for a period of time exceeding the reference value A3, or when a smoking action is detected within a fixed time period, or when a drinking action is detected within a fixed time period, or when a hand-held phone call action is detected within a fixed time period, it is determined that the driver is in a distracted driving state; When the PRC value is not lower than the PRC reference value A2, and when the driver's continuous time without looking at the road is not higher than the reference value A3, and no smoking, drinking, or hand-held phone calls are detected within a fixed time period, the driver is determined to be in a non-distracted driving state.

7. A driver attention detection method as claimed in claim 1, characterized in that: The alarm reminder operation includes at least one of a continuous text warning and a sound warning, and the system operation includes one or more of turning off the automatic driving assistance system, braking the vehicle, tightening the seat belt, closing the windows and sunroof, turning on the hazard lights, turning on the interior lights, continuously honking the horn, and slowing the vehicle down until it stops.

8. A driver attention detection device, characterized in that: It is used to execute a driver attention detection method as described in any one of claims 1 to 7.

9. A driver attention detection device as claimed in claim 8, characterized in that: It includes a judgment control module, a camera module, a feature analysis module, a warning module, and a vehicle data collection module, wherein the feature analysis module, the warning module, and the vehicle driving data collection module are respectively connected to the control module; the judgment control module is used to implement the logical judgment in the driver attention detection method; the camera module is used to obtain the driver's driving status; The feature analysis module is used to extract and analyze the driver's feature information, and is used to perform data training on the model and define basic parameters; the vehicle data acquisition module is used to collect information from various vehicle controllers; and the warning module is used to send alarm reminder information to the driver.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it is used to implement a driver attention detection method as described in any one of claims 1 to 7.

Images