Fatigue driving monitoring methods, devices, traffic equipment and storage media

Abstract

A fatigue driving monitoring method, device, traffic equipment, and storage medium are disclosed. The method includes: acquiring the driver's current facial information (S101); controlling an electronic device to output a stimulus signal to stimulate the driver, and acquiring stress information of the driver's stress response based on the stimulus signal (S102); and determining the driver's current fatigue level based on the facial information and the stress information (S103).

Description

Technical Field

[0001] This application relates to the field of traffic safety technology, and in particular to a fatigue driving monitoring method, device, traffic equipment, and storage medium. Background Technology

[0002] Nowadays, vehicles have become an indispensable means of transportation for people, making safe driving especially important. When drivers are fatigued, their ability to perceive their surroundings, judge situations, and control the vehicle all decline to varying degrees, making them more prone to traffic accidents. Therefore, timely detection of driver fatigue is of paramount importance.

[0003] Currently, fatigue driving behavior is generally assessed by analyzing images of the driver, estimating head posture, and detecting eye and mouth closure. However, because this method relies on facial or eye detection results, the accuracy of fatigue driving detection becomes compromised when facial or eye information is obscured, such as when the driver is wearing opaque sunglasses and their eyes are not visible.

[0004] Therefore, improving the accuracy of fatigue driving detection has become an urgent problem to be solved. Summary of the Invention

[0005] Based on this, this application provides a fatigue driving monitoring method, device, traffic equipment, and storage medium to improve the accuracy of fatigue driving detection.

[0006] Firstly, this application provides a method for monitoring fatigued driving, including:

[0007] Obtain the driver's current facial information;

[0008] The electronic control device outputs a stimulus signal to stimulate the driver and acquires stress information of the driver's stress response based on the stimulus signal.

[0009] The driver's current level of fatigue is determined based on the facial information and the stress information.

[0010] Secondly, this application also provides a fatigue driving monitoring device, which includes a memory and a processor;

[0011] The memory is used to store computer programs;

[0012] The processor is configured to execute the computer program and, when executing the computer program, perform the following steps:

[0013] Obtain the driver's current facial information;

[0014] The electronic control device outputs a stimulus signal to stimulate the driver and acquires stress information of the driver's stress response based on the stimulus signal.

[0015] The driver's current level of fatigue is determined based on the facial information and the stress information.

[0016] Thirdly, this application also provides a traffic device, wherein the remote controller includes the fatigue driving monitoring device as described above.

[0017] Fourthly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the fatigue driving monitoring method described above.

[0018] The fatigue driving monitoring method, device, traffic equipment, and storage medium disclosed in this application acquire the driver's current facial information and control electronic devices to output stimulation signals to the driver, thereby acquiring stress information of the driver's stress response based on the stimulation signals. The stress information corresponding to the driver's stress response is different under different fatigue levels. By combining facial information and stress information, the driver's current fatigue level can be determined. Compared with detection results that rely on facial or eye recognition, the accuracy of fatigue driving detection is improved.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of the structure of a transportation device provided in an embodiment of this application;

[0022] Figure 2 This is a schematic flowchart illustrating the steps of a fatigue driving monitoring method provided in an embodiment of this application;

[0023] Figure 3 This is a schematic flowchart illustrating the steps of another fatigue driving monitoring method provided in this application embodiment;

[0024] Figure 4 This is a schematic flowchart illustrating the steps for determining the current fatigue level of a driver based on the facial information, stress information, and physical state information, as provided in this application embodiment.

[0025] Figure 5 This is a schematic flowchart illustrating the steps of another fatigue driving monitoring method provided in this application embodiment;

[0026] Figure 6 This is a schematic block diagram of a fatigue driving monitoring device provided in an embodiment of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0029] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0030] It should also be understood that the term "and / or" as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0031] The following detailed description of some embodiments of this application is provided in conjunction with the accompanying drawings. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0032] Embodiments of this application provide a fatigue driving monitoring method, device, traffic equipment, and storage medium to improve the accuracy of fatigue driving detection.

[0033] Please see Figure 1 , Figure 1 This is a structural schematic diagram of a transportation device provided as an embodiment of this application. Figure 1As shown, the traffic equipment 1000 may include a power unit 100 and a fatigue driving monitoring device 200. The power unit 100 is used to drive and traction of the traffic equipment 1000, and the fatigue driving monitoring device 200 is used to detect the fatigue level of the driver of the traffic equipment 1000.

[0034] The transportation equipment 1000 may include automobiles, trams, trucks, etc. Of course, the transportation equipment 1000 may also be other types of mobile transportation such as boats, and the embodiments of this application are not limited thereto.

[0035] The fatigue driving monitoring device 200 is used to acquire the driver's current facial information and control electronic devices such as smart bracelets to output stimulation signals to the driver, acquire the stress information of the driver's stress response based on the stimulation signals, and combine the facial information and stress information to determine the driver's current fatigue level, thus improving the accuracy of fatigue driving detection.

[0036] It is understood that the naming of the various components of the traffic equipment 1000 is merely for identification purposes and does not limit the embodiments of this application.

[0037] The fatigue driving monitoring method provided by the embodiments of this application will be described in detail below based on traffic equipment 1000 and fatigue driving monitoring device 200. It should be noted that... Figure 1 The traffic equipment 1000 and the fatigue driving monitoring device 200 mentioned are only used to explain the fatigue driving monitoring method provided in the embodiments of this application, but do not constitute a limitation on the application scenarios of the fatigue driving monitoring method provided in the embodiments of this application.

[0038] Please see Figure 2 , Figure 2 This is a schematic flowchart illustrating a fatigue driving monitoring method provided in an embodiment of this application. This method can be used in the traffic equipment provided in the above embodiment, as well as in other equipment that includes a fatigue driving monitoring device. This application does not limit the application scenario of this method. Based on this fatigue driving monitoring method, the accuracy of fatigue driving detection can be improved, thereby enhancing driving safety.

[0039] like Figure 2 As shown, the fatigue driving monitoring method specifically includes steps S101 to S103.

[0040] S101. Obtain the driver's current facial information.

[0041] For example, a camera device is installed in front of the driver. The camera device can capture images and / or videos of the driver's face, and obtain the driver's facial information by performing facial information extraction operations on the facial images and / or videos. For example, the facial information includes, but is not limited to, information about the driver's eyes, head posture, and mouth.

[0042] S102. Control the electronic device to output a stimulus signal to stimulate the driver, and acquire stress information of the driver's stress response based on the stimulus signal.

[0043] Generally, drivers exhibit varying degrees of stress response to stimuli depending on their level of fatigue. Based on this, in addition to acquiring the driver's facial information, electronic devices are controlled to output stimulus signals. These electronic devices include, but are not limited to, wearable devices such as smart bracelets and smartwatches. The driver will generate a stress response based on these stimulus signals, and the system acquires the corresponding stress information. This stress information includes at least one of the following: stress response time and stress response intensity.

[0044] For example, controlling the electronic device to output a stimulus signal to the driver may include: sending a control signal to the electronic device so that the electronic device outputs the stimulus signal upon receiving the control signal.

[0045] By establishing a communication connection with electronic devices, such as wireless communication connections like WiFi or Bluetooth, the driver sends control signals to these devices during driving. Upon receiving these control signals, the electronic devices output corresponding stimulus signals. The driver will produce a stress response to these stimulus signals, and the electronic devices can collect this stress information.

[0046] For example, control signals are transmitted to a smart bracelet worn by the driver via wireless communication methods such as WiFi and Bluetooth. Upon receiving the control signal, the smart bracelet outputs stimulation signals such as vibration, for example, high-frequency, low-peak-value, short-duration vibrations. Furthermore, the smart bracelet collects stress information such as the driver's stress response intensity (peak value of the stress response after being stimulated, max) and stress response time (time t from being stimulated to the response reaching its peak value, max), and feeds this stress information back to the driver.

[0047] In some embodiments, sending a control signal to the electronic device may include periodically sending the control signal to the electronic device according to a preset periodic time.

[0048] For example, a pre-set cycle time of 5 minutes is used. During driving, control signals are sent to electronic devices every 5 minutes based on this cycle time. Whenever the electronic devices receive a control signal, they output stimulation signals such as vibration and collect stress information about the driver's stress response based on the stimulation signals. It is understood that this cycle time can be flexibly set according to actual conditions and is not limited here.

[0049] Furthermore, since drivers typically begin to feel fatigued after driving for a period of time, a timer can be started after the driver begins driving. Once the preset duration is reached, fatigue monitoring is activated, and control signals are periodically sent to the driver's smart bracelet or other electronic device based on the scheduled time intervals. The preset duration can be flexibly set according to actual conditions; for example, it can be set to 1 hour, or other durations, without any restrictions.

[0050] S103. Determine the driver's current level of fatigue based on the facial information and the stress information.

[0051] After obtaining the driver's facial information and stress information based on the stimulus signals, the driver's current level of fatigue is determined by combining the facial information and stress information.

[0052] For example, based on the acquired facial information, key points of the human eye are detected, and the opening of the human eye is calculated based on these key points. The calculation method is as follows:

[0053] left ratio =abs|left top -left down |

[0054] right ratio =abs|right top -right down |

[0055] Where, left ratio It refers to the opening of the left eye. top It is the highest point when the left eye is open. down It is the lowest position when the left eye is open, right ratio It refers to the opening of the right eye. top It is the highest point when the right eye is open. down It is the lowest position of the right eye when it is open, and abs is the absolute value of the difference between the two.

[0056] Based on the opening of the left and right eyes ratio、 right ratioCalculate the perclos score. The perclos score can be calculated using three metrics: P70, P80, and EM.

[0057] It should be noted that the above is just one of the methods for calculating perclos. Other methods can also be used to calculate perclos, such as using CNN to predict the eye state and then calculating perclos, or using traditional image processing methods to predict the eye state and then calculate perclos. This embodiment does not specifically limit the method for calculating perclos.

[0058] For example, based on stress information such as stress response intensity and stress response time, the corresponding score for the stress information is determined, where different stress response intensities and stress response times correspond to different scores.

[0059] Next, the obtained Perclos score and the scores corresponding to stress information are weighted and summed to obtain a weighted score. Based on this weighted score, the preset mapping relationship between fatigue level and score is queried to determine the fatigue level corresponding to the weighted score. In other words, the driver's fatigue level is determined by combining the driver's facial information and stress information.

[0060] In some embodiments, such as Figure 3 As shown, step S104 may be included before step S103, and step S103 may include sub-step S1031.

[0061] S104. Obtain the driver's physical condition information.

[0062] The driver's physical condition information includes at least one of the following: heart rate, IMU (Inertial Measurement Unit) information, and temperature information. It is understood that physical condition information may also include other information besides heart rate, IMU, and temperature.

[0063] In some embodiments, obtaining the driver's physical state information may include: receiving the physical state information collected by sensors, wherein the sensors include at least one of a pulse sensor, an IMU sensor, and a temperature sensor.

[0064] For example, sensors such as pulse sensors, IMU sensors, and temperature sensors used to collect information about the driver's physical condition can be installed in wearable devices such as smart bracelets and smartwatches, or in areas of the cockpit that the driver's body comes into contact with. For instance, they can be embedded inside the car steering wheel, monitoring and collecting the driver's heart rate, IMU information, temperature information, and other physical condition information when the driver is holding the steering wheel; or they can be installed on the back of the cockpit, directly contacting the driver's cervical spine to monitor and collect various physical condition information.

[0065] S1031. Determine the driver's current level of fatigue based on the facial information, the stress information, and the physical state information.

[0066] A person's physical condition information differs when they are awake and fatigued. For example, heart rate information shows that a person's heart rate is generally between 60 and 100 beats per minute when awake and normal, while it is between 40 and 60 beats per minute when asleep. Therefore, combining facial and stress information with physical condition information to determine the driver's current level of fatigue can further improve the accuracy of fatigue driving detection.

[0067] In some embodiments, such as Figure 4 As shown, step S1031 may include sub-steps S10311 to S10315.

[0068] S10311. Perform perclos calculation based on the facial information to obtain the first score.

[0069] The perclos calculation based on facial information can be referred to in the above embodiment, and will not be repeated here. For ease of distinction, the score obtained by the perclos calculation will be referred to as the first score, score_1, in the following text.

[0070] S10312. Quantify and classify the body state information to obtain the second score corresponding to the body state information.

[0071] In some embodiments, when the body status information includes heart rate information, obtaining the second score corresponding to the body status information may include: determining the heart rate level corresponding to the heart rate information according to a preset mapping relationship between heart rate and quantification level; and determining the score corresponding to the heart rate level as the second score according to a preset mapping relationship between heart rate level and score.

[0072] For example, the pre-set mapping relationship between heart rate (heart_rate) and quantization level (grade) is as follows:

[0073]

[0074] The mapping relationship between heart rate grade and score is as follows:

[0075]

[0076] After obtaining the driver's heart rate information, the system queries the preset mapping relationship between heart rate (heart_rate) and quantification level (grade) to determine which level (1 to 6) the driver's heart rate information corresponds to. For example, if the obtained heart rate information is 63, the heart rate level (grade) is determined to be level 2. Then, it substitutes the heart rate level (grade) into the above mapping relationship with the score (score) to determine the corresponding score as 0.1, and uses the obtained score of 0.1 as the second score (score_2).

[0077] In some embodiments, when the body state information includes IMU information, quantizing and classifying the body state information to obtain a second score corresponding to the body state information may include: obtaining a first acceleration, a second acceleration, and a third acceleration in three directions based on the IMU information; calculating the sum of the absolute values ​​of the first acceleration, the second acceleration, and the third acceleration to obtain an acceleration sum value; determining the IMU level corresponding to the acceleration sum value according to a preset mapping relationship between acceleration and quantization level; and determining the score corresponding to the IMU level as the second score according to the preset mapping relationship between IMU level and score.

[0078] If the first acceleration in the three directions is r_x, the second acceleration is r_y, and the third acceleration is r_z, calculate the sum of the absolute values ​​of the first acceleration r_x, the second acceleration r_y, and the third acceleration r_z to obtain the sum of accelerations r_all: r_all = abs|r_x| + abs|r_y| + abs|r_z|.

[0079] In some embodiments, after obtaining the first acceleration, second acceleration, and third acceleration corresponding to the three directions based on the IMU information, the process may include: performing correction processing on the first acceleration, second acceleration, and third acceleration to obtain a first corrected acceleration, a second corrected acceleration, and a third corrected acceleration; calculating the sum of the absolute values ​​of the first acceleration, second acceleration, and third acceleration to obtain the acceleration sum value may include: determining the sum of the absolute values ​​of the first corrected acceleration, second corrected acceleration, and third corrected acceleration as the acceleration sum value.

[0080] To further improve the accuracy of fatigue driving detection, the first acceleration r_x, the second acceleration r_y, and the third acceleration r_z are corrected to remove the influence of gravitational acceleration, resulting in the first corrected acceleration r_x', the second corrected acceleration r_y', and the third corrected acceleration r_z'. Then, the sum of the absolute values ​​of these three accelerations is calculated to obtain the acceleration sum r_all: r_all = abs|r_x'| + abs|r_y'| + abs|r_z'|.

[0081] For example, the pre-defined mapping relationship between acceleration r_all and quantization level grade (IMU level) is as follows:

[0082]

[0083] The mapping relationship between IMU grade and score is as follows:

[0084]

[0085] After calculating the acceleration sum value r_all, the preset mapping relationship between acceleration r_all and quantization level grade is queried to determine which level from 1 to 6 in the mapping relationship corresponds to the acceleration sum value r_all. For example, if the obtained acceleration sum value r_all is 0.44, then the heart rate level grade is determined to be 4. Then, substituting it into the above mapping relationship between heart rate level grade and score, the corresponding score is determined to be 0.8, and the obtained score of 0.8 is used as the second score score_2.

[0086] In some embodiments, when the body state information includes temperature information, quantifying and classifying the body state information to obtain a second score corresponding to the body state information may include: determining the temperature level corresponding to the temperature information according to a preset mapping relationship between temperature and quantization level; and determining the score corresponding to the temperature level as the second score according to a preset mapping relationship between temperature level and score.

[0087] For example, the pre-defined mapping relationship between temperature and quantization grade (temperature level) is as follows:

[0088]

[0089] The mapping relationship between temperature grade and score is as follows:

[0090]

[0091] After obtaining the driver's temperature, the system queries the preset mapping relationship between temperature and quantification level (grade) to determine which level (1-4) the driver's temperature corresponds to. For example, if the driver's temperature is 36.6°C, the heart rate level (grade) is determined to be level 2. Then, by substituting this into the mapping relationship between temperature level (grade) and score, the corresponding score is determined to be 0.4. This score of 0.4 is then used as the second score, score_2.

[0092] In some embodiments, when the body state information includes multiple components, multiple scores are determined based on the multiple body state information. Then, a second score, score_2, is determined based on the multiple scores. For example, the multiple scores are weighted and summed to obtain the second score, score_2.

[0093] For example, when the body status information includes heart rate, IMU, and temperature, then three corresponding scores are determined based on the heart rate, IMU, and temperature information, respectively. If these are respectively... heart score r score tem Score these three values heart score r score tem After performing a weighted summation, the second score, score_2, is obtained as follows:

[0094] in, These are the corresponding weighting coefficients, and their specific values ​​can be flexibly set according to the actual situation. No specific restrictions are imposed here.

[0095] S10313. The stress information is quantified and graded to obtain the third score corresponding to the stress information.

[0096] For example, stress information includes stress response time and stress response intensity, and a corresponding third score, score_3, is obtained based on the stress response time and stress response intensity.

[0097] In some embodiments, quantifying and classifying the stress information to obtain a third score corresponding to the stress information may include: determining the stress level corresponding to the stress reaction time based on a preset mapping relationship between reaction time and quantification level; determining the stress coefficient corresponding to the stress reaction intensity based on a preset mapping relationship between reaction intensity and quantification coefficient; and determining the third score based on the stress level and the stress coefficient.

[0098] For example, taking the IMU signal as the stress signal of the driver's stress response, the stress response time t of the driver's stress response and the corresponding measurement values ​​a, b, and c of the IMU sensor in the three-axis direction at time t are obtained.

[0099] For example, the pre-defined mapping relationship between reaction time t and quantification level (stress level) is as follows:

[0100]

[0101] Based on the obtained driver's stress response time t, the preset mapping relationship between reaction time t and quantification level grade is queried to determine which level from 1 to 6 in the mapping relationship corresponds to the driver's stress response time t. For example, if the obtained stress response time t is 0.5, then the stress level grade is determined to be "from".

[0102] For example, the pre-defined mapping relationship between response intensities a, b, and c and the quantification coefficient alpha (stress coefficient) is as follows:

[0103]

[0104] Based on the measured values ​​a, b, and c of the driver in the three axial directions at time t, the value of |a|+|b|+|c| is calculated, and the preset mapping relationship between the reaction intensities a, b, and c and the quantization coefficient alpha is looked up to determine the stress coefficient alpha corresponding to the measured values ​​a, b, and c of the driver in the three axial directions at time t. For example, if the calculated value of |a|+|b|+|c| is 0.6, then the corresponding stress coefficient alpha is determined to be 0.6.

[0105] Next, based on the obtained stress level (grade) and stress coefficient (alpha), the corresponding third score (score_3) is determined. For example, substituting these values ​​into the formula score_3 = (1 / grade) * alpha, the third score score_3 is calculated. For instance, in the example above, if the grade is 3 and the alpha is 0.6, substituting these values ​​into the formula score_3 = (1 / grade) * alpha, the calculated third score score_3 is 0.2.

[0106] It should be noted that the stress signals of a driver's stress response are not limited to IMU signals; stress information can also be measurements obtained from other physiological signals.

[0107] S10314. Perform a weighted summation of the first score, the second score, and the third score to obtain a comprehensive score.

[0108] After obtaining the first score_1, the second score_2, and the third score_3 as described above, the first score_1, the second score_2, and the third score_3 are weighted and summed to obtain the comprehensive score: in, These are the corresponding weighting coefficients, and their specific values ​​can be flexibly set according to the actual situation. No specific restrictions are imposed here.

[0109] For example, if the second score, score_2, is determined by score heart score r score tem The scores are calculated from multiple values, based on score_1, score_2, and score_3. heart score r score tem The score is calculated by weighted summation of score_3, resulting in the score: score = α·score_1 + β·score heart +γ·score r +δ·score tem +θ·score_3.

[0110] Where α, β, γ, δ, and θ are the corresponding weighting coefficients. For example, if α, β, γ, δ, and θ are set to 0.4, 0.2, 0.2, 0.1, and 0.1 respectively, then the weighted summation yields the score as follows:

[0111] score = 0.4score_1 + 0.2score heart +0.2 score r +0.1 score tem +0.1score_3

[0112] It should be noted that the specific values ​​of α, β, γ, δ, and θ can be flexibly set according to the actual situation, and no specific restrictions are imposed here.

[0113] S10315. Determine the fatigue level corresponding to the comprehensive score based on the preset mapping relationship between fatigue level and score.

[0114] For example, the mapping relationship between fatigue level and score includes, but is not limited to, a mapping table. For instance, the pre-set mapping relationship between fatigue level and score is shown in Table 1.

[0115] Table 1

[0116] Score fatigue level >0.7 Already asleep >0.5&&<＝0.7 Severe fatigue >0.25&&<＝0.5 Moderate fatigue >0.15&&<＝0.25 Mild fatigue <＝0.15 wide awake

[0117] For example, if the calculated score is 0.3, by looking up Table 1, it can be determined that the fatigue level corresponding to a score of 0.3 is moderate fatigue, that is, the driver's current fatigue level is determined to be moderate fatigue.

[0118] In some embodiments, such as Figure 5 As shown, step S105 may be included after step S103.

[0119] S105. Output corresponding alarm information according to the fatigue level, wherein different fatigue levels correspond to different alarm information.

[0120] The alarm information includes at least one of vibration alerts and voice prompts. That is, different vibration alerts and / or voice prompts are given based on the driver's level of fatigue. For example, voice prompts can be provided using the car's center console audio system.

[0121] In some embodiments, outputting the corresponding alarm information may include: sending an alarm command to a wearable device, so that the wearable device outputs the alarm information according to the alarm command.

[0122] For example, based on the driver's level of fatigue, corresponding alarm commands can be sent to wearable devices such as smart bracelets and smartwatches, which can then provide corresponding vibration and / or voice prompts.

[0123] For example, different levels of fatigue correspond to different prompting parameters for vibration prompts and / or voice prompts, wherein the prompting parameters include, but are not limited to, vibration frequency, vibration amplitude, vibration duration, and voice volume.

[0124] For example, the pre-set mapping relationship between fatigue level, score, and alarm information is shown in Table 2.

[0125] Table 2

[0126]

[0127] For example, if it is determined that the driver's current fatigue level is moderate, the corresponding alarm information can be determined by referring to Table 2, and the smart bracelet (watch) can be controlled to vibrate in a short-term, high-amplitude manner to provide a reminder.

[0128] The above embodiments acquire the driver's current facial information and control the electronic device to output a stimulus signal to stimulate the driver, thereby acquiring the driver's stress information based on the stimulus signal. The stress information corresponding to the driver's stress response is different under different fatigue levels. By combining facial information and stress information, the driver's current fatigue level can be determined. Compared with detection results that rely on facial or eye recognition, the accuracy of fatigue driving detection is improved, thereby improving the driver's driving safety.

[0129] Please see Figure 6 , Figure 6 This is a schematic block diagram of a fatigue driving monitoring device provided in one embodiment of this application.

[0130] like Figure 6 As shown, the fatigue driving monitoring device 200 may include a processor 211 and a memory 212, which are connected by a bus, such as an I2C (Inter-integrated Circuit) bus.

[0131] Specifically, the processor 211 can be a microcontroller unit (MCU), a central processing unit (CPU), or a digital signal processor (DSP), etc.

[0132] Specifically, the memory 212 can be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a portable hard drive, etc. The memory 212 stores various computer programs for the processor 211 to execute.

[0133] The processor is configured to run a computer program stored in memory, and to perform the following steps when executing the computer program:

[0134] Obtain the driver's current facial information;

[0135] The electronic control device outputs a stimulus signal to stimulate the driver and acquires stress information of the driver's stress response based on the stimulus signal.

[0136] The driver's current level of fatigue is determined based on the facial information and the stress information.

[0137] In some embodiments, the stress information includes at least one of stress response time and stress response intensity.

[0138] In some embodiments, when the processor implements the output of a stimulation signal to the driver by the control electronic device, it specifically implements the following:

[0139] A control signal is sent to the electronic device so that the electronic device outputs the stimulation signal upon receiving the control signal.

[0140] In some embodiments, when the processor implements the sending of control signals to the electronic device, it specifically implements the following:

[0141] The control signal is periodically sent to the electronic device according to a preset periodic interval.

[0142] In some embodiments, before determining the driver's current fatigue level based on the facial information and the stress information, the processor further implements:

[0143] Obtain the driver's physical condition information;

[0144] When the processor determines the driver's current fatigue level based on the facial information and the stress information, it specifically implements the following:

[0145] The driver's current level of fatigue is determined based on the facial information, stress information, and physical condition information.

[0146] In some embodiments, the body status information includes at least one of heart rate information, IMU information, and temperature information.

[0147] In some embodiments, when the processor acquires the driver's physical state information, it specifically implements the following:

[0148] The body state information collected by the sensors includes at least one of a pulse sensor, an IMU sensor, and a temperature sensor.

[0149] In some embodiments, the sensor is disposed within a wearable device, or the sensor is disposed in a cockpit portion that is in contact with the driver's body.

[0150] In some embodiments, when the processor determines the driver's current fatigue level based on the facial information, the stress information, and the physical state information, it specifically implements the following:

[0151] Based on the facial information, a perclos calculation is performed to obtain a first score;

[0152] The physical state information is quantified and graded to obtain a second score corresponding to the physical state information;

[0153] The stress information is quantified and graded to obtain a third score corresponding to the stress information;

[0154] The first score, the second score, and the third score are weighted and summed to obtain a comprehensive score.

[0155] The fatigue level corresponding to the comprehensive score is determined based on the preset mapping relationship between fatigue level and score.

[0156] In some embodiments, the body state information includes heart rate information, and the processor, when performing the quantification and grading processing of the body state information to obtain the second score corresponding to the body state information, specifically implements the following:

[0157] The heart rate level corresponding to the heart rate information is determined based on the preset mapping relationship between heart rate and quantization level.

[0158] Based on the preset mapping relationship between heart rate levels and scores, the score corresponding to the heart rate level is determined as the second score.

[0159] In some embodiments, the body state information includes IMU information, and the processor, when performing quantization and grading processing on the body state information to obtain a second score corresponding to the body state information, specifically implements the following:

[0160] Based on the IMU information, the first acceleration, second acceleration, and third acceleration corresponding to the three directions are obtained;

[0161] Calculate the sum of the absolute values ​​of the first acceleration, the second acceleration, and the third acceleration to obtain the sum of acceleration values;

[0162] Based on the preset mapping relationship between acceleration and quantization level, the IMU level corresponding to the acceleration and value is determined;

[0163] Based on the preset mapping relationship between IMU levels and scores, the score corresponding to the IMU level is determined as the second score.

[0164] In some embodiments, after the processor obtains the first acceleration, second acceleration, and third acceleration corresponding to the three directions based on the IMU information, it further implements:

[0165] The first acceleration, the second acceleration, and the third acceleration are corrected to obtain a first corrected acceleration, a second corrected acceleration, and a third corrected acceleration.

[0166] When the processor calculates the sum of the absolute values ​​of the first acceleration, the second acceleration, and the third acceleration to obtain the acceleration sum value, it specifically implements the following:

[0167] The sum of the absolute values ​​of the first corrected acceleration, the second corrected acceleration, and the third corrected acceleration is determined as the acceleration sum value.

[0168] In some embodiments, the body state information includes temperature information, and the processor, when performing the quantization and grading processing on the body state information to obtain the second score corresponding to the body state information, specifically implements the following:

[0169] The temperature level corresponding to the temperature information is determined based on the preset mapping relationship between temperature and quantization level.

[0170] Based on the preset mapping relationship between temperature levels and scores, the score corresponding to the temperature level is determined as the second score.

[0171] In some embodiments, the stress information includes stress response time and stress response intensity. When the processor performs quantification and grading of the stress information to obtain a third score corresponding to the stress information, the specific implementation is as follows:

[0172] The stress level corresponding to the stress response time is determined based on the preset mapping relationship between the reaction time and the quantification level.

[0173] Based on the preset mapping relationship between the response intensity and the quantification coefficient, the stress coefficient corresponding to the stress response intensity is determined;

[0174] The third score is determined based on the stress level and the stress coefficient.

[0175] In some embodiments, after determining the driver's current level of fatigue, the processor further implements:

[0176] Based on the fatigue level, corresponding alarm information is output, where different fatigue levels correspond to different alarm information.

[0177] In some embodiments, when the processor implements the alarm information corresponding to the output, it specifically implements:

[0178] An alarm command is sent to the wearable device, so that the wearable device can output the alarm information according to the alarm command.

[0179] In some embodiments, the alarm information includes at least one of vibration alert information and voice alert information.

[0180] In some embodiments, different levels of fatigue correspond to different prompting parameters of the vibration prompting information and / or the voice prompting information, and the prompting parameters include vibration amplitude, vibration duration, and voice volume.

[0181] The embodiments of this application also provide a computer-readable storage medium storing a computer program, the computer program including program instructions, and the processor executing the program instructions to implement the steps of the fatigue driving monitoring method provided in the embodiments of this application.

[0182] The computer-readable storage medium can be an internal storage unit of the traffic equipment or fatigue driving monitoring device described in the foregoing embodiments, such as a hard disk or memory of the traffic equipment or fatigue driving monitoring device. The computer-readable storage medium can also be an external storage device of the traffic equipment or fatigue driving monitoring device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the traffic equipment or fatigue driving monitoring device.

[0183] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for monitoring driver fatigue, characterized in that, include: Obtain the driver's current facial information; During driving, the electronic device controls the output of a stimulus signal to the driver and acquires stress information of the driver's stress response based on the stimulus signal; the electronic device includes a wearable device; the stress information includes at least one of stress response time and stress response intensity. Obtain the driver's physical condition information; determine the driver's current level of fatigue based on the facial information, stress information, and physical condition information; Determining the driver's current fatigue level based on the facial information, stress information, and physical state information includes: A first score is obtained by performing perclos calculation based on the facial information; a second score is obtained by quantifying and classifying the body state information; a third score is obtained by quantifying and classifying the stress information; a comprehensive score is obtained by weighted summation of the first, second, and third scores; and the fatigue level corresponding to the comprehensive score is determined based on a preset mapping relationship between fatigue level and score. After determining the driver's current level of fatigue, the process includes: Based on the fatigue level, corresponding alarm information is output, where different fatigue levels correspond to different alarm information.

2. The method according to claim 1, characterized in that, The control electronic device outputs a stimulus signal to the driver, including: A control signal is sent to the electronic device so that the electronic device outputs the stimulation signal upon receiving the control signal.

3. The method according to claim 2, characterized in that, Sending control signals to the electronic device includes: The control signal is periodically sent to the electronic device according to a preset periodic interval.

4. The method according to claim 1, characterized in that, The body status information includes at least one of heart rate information, IMU information, and temperature information.

5. The method according to claim 1, characterized in that, The acquisition of the driver's physical condition information includes: The body state information collected by the sensors includes at least one of a pulse sensor, an IMU sensor, and a temperature sensor.

6. The method according to claim 5, characterized in that, The sensor is located inside a wearable device, or the sensor is located in the cockpit area that the driver's body contacts.

7. The method according to claim 1, characterized in that, The body state information includes heart rate information. The step of quantifying and classifying the body state information to obtain a second score corresponding to the body state information includes: The heart rate level corresponding to the heart rate information is determined based on the preset mapping relationship between heart rate and quantization level. Based on the preset mapping relationship between heart rate levels and scores, the score corresponding to the heart rate level is determined as the second score.

8. The method according to claim 1, characterized in that, The body state information includes IMU information. The step of quantifying and classifying the body state information to obtain a second score corresponding to the body state information includes: Based on the IMU information, the first acceleration, second acceleration, and third acceleration corresponding to the three directions are obtained; Calculate the sum of the absolute values ​​of the first acceleration, the second acceleration, and the third acceleration to obtain the sum of acceleration values; Based on the preset mapping relationship between acceleration and quantization level, the IMU level corresponding to the acceleration and value is determined; Based on the preset mapping relationship between IMU levels and scores, the score corresponding to the IMU level is determined as the second score.

9. The method according to claim 8, characterized in that, After obtaining the first acceleration, second acceleration, and third acceleration corresponding to the three directions based on the IMU information, the process includes: The first acceleration, the second acceleration, and the third acceleration are corrected to obtain a first corrected acceleration, a second corrected acceleration, and a third corrected acceleration. The calculation of the sum of the absolute values ​​of the first acceleration, the second acceleration, and the third acceleration to obtain the acceleration sum value includes: The sum of the absolute values ​​of the first corrected acceleration, the second corrected acceleration, and the third corrected acceleration is determined as the acceleration sum value.

10. The method according to claim 1, characterized in that, The body state information includes temperature information. The step of quantifying and classifying the body state information to obtain a second score corresponding to the body state information includes: The temperature level corresponding to the temperature information is determined based on the preset mapping relationship between temperature and quantization level. Based on the preset mapping relationship between temperature levels and scores, the score corresponding to the temperature level is determined as the second score.

11. The method according to claim 1, characterized in that, The stress information includes stress response time and stress response intensity. The step of quantifying and classifying the stress information to obtain a third score corresponding to the stress information includes: The stress level corresponding to the stress response time is determined based on the preset mapping relationship between the reaction time and the quantification level. Based on the preset mapping relationship between the response intensity and the quantification coefficient, the stress coefficient corresponding to the stress response intensity is determined; The third score is determined based on the stress level and the stress coefficient.

12. The method according to any one of claims 1 to 11, characterized in that, The alarm information corresponding to the output includes: An alarm command is sent to the wearable device, so that the wearable device can output the alarm information according to the alarm command.

13. The method according to any one of claims 1 to 11, characterized in that, The alarm information includes at least one of vibration alerts and voice alerts.

14. The method according to claim 13, characterized in that, Different levels of fatigue correspond to different prompting parameters of the vibration prompting information and / or the voice prompting information, and the prompting parameters include vibration amplitude, vibration duration, and voice volume.

15. A fatigue driving monitoring device, characterized in that, The fatigue driving monitoring device includes a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program and, when executing the computer program, perform the following steps: Obtain the driver's current facial information; During driving, the electronic device controls the output of a stimulus signal to the driver and acquires stress information of the driver's stress response based on the stimulus signal; the electronic device includes a wearable device; the stress information includes at least one of stress response time and stress response intensity. Obtain the driver's physical condition information; determine the driver's current level of fatigue based on the facial information, stress information, and physical condition information; When the processor determines the driver's current fatigue level based on the facial information, stress information, and physical state information, it specifically implements the following: A first score is obtained by performing perclos calculation based on the facial information; a second score is obtained by quantifying and classifying the body state information; a third score is obtained by quantifying and classifying the stress information; a comprehensive score is obtained by weighted summation of the first, second, and third scores; and the fatigue level corresponding to the comprehensive score is determined based on a preset mapping relationship between fatigue level and score. After determining the driver's current level of fatigue, the processor also implements: Based on the fatigue level, corresponding alarm information is output, where different fatigue levels correspond to different alarm information.

16. The apparatus according to claim 15, characterized in that, When the processor implements the output of a stimulation signal to the driver by the control electronic device, it specifically implements the following: A control signal is sent to the electronic device so that the electronic device outputs the stimulation signal upon receiving the control signal.

17. The apparatus according to claim 16, characterized in that, When the processor implements the sending of control signals to the electronic device, it specifically implements the following: The control signal is periodically sent to the electronic device according to a preset periodic interval.

18. The apparatus according to claim 15, characterized in that, The body status information includes at least one of heart rate information, IMU information, and temperature information.

19. The apparatus according to claim 15, characterized in that, When the processor acquires the driver's physical state information, it specifically implements the following: The body state information collected by the sensors includes at least one of a pulse sensor, an IMU sensor, and a temperature sensor.

20. The apparatus according to claim 19, characterized in that, The sensor is located inside a wearable device, or the sensor is located in the cockpit area that the driver's body contacts.

21. The apparatus according to claim 15, characterized in that, The body state information includes heart rate information. When the processor performs quantification and grading of the body state information to obtain a second score corresponding to the body state information, it specifically implements the following: The heart rate level corresponding to the heart rate information is determined based on the preset mapping relationship between heart rate and quantization level. Based on the preset mapping relationship between heart rate levels and scores, the score corresponding to the heart rate level is determined as the second score.

22. The apparatus according to claim 15, characterized in that, The body state information includes IMU information. When the processor performs quantization and grading processing on the body state information to obtain the second score corresponding to the body state information, it specifically implements the following: Based on the IMU information, the first acceleration, second acceleration, and third acceleration corresponding to the three directions are obtained; Calculate the sum of the absolute values ​​of the first acceleration, the second acceleration, and the third acceleration to obtain the sum of acceleration values; Based on the preset mapping relationship between acceleration and quantization level, the IMU level corresponding to the acceleration and value is determined; Based on the preset mapping relationship between IMU levels and scores, the score corresponding to the IMU level is determined as the second score.

23. The apparatus according to claim 22, characterized in that, After implementing the process of obtaining the first acceleration, second acceleration, and third acceleration corresponding to the three directions based on the IMU information, the processor also implements: The first acceleration, the second acceleration, and the third acceleration are corrected to obtain a first corrected acceleration, a second corrected acceleration, and a third corrected acceleration. When the processor calculates the sum of the absolute values ​​of the first acceleration, the second acceleration, and the third acceleration to obtain the acceleration sum value, it specifically implements the following: The sum of the absolute values ​​of the first corrected acceleration, the second corrected acceleration, and the third corrected acceleration is determined as the acceleration sum value.

24. The apparatus according to claim 15, characterized in that, The body state information includes temperature information. When the processor performs quantification and grading processing on the body state information to obtain the second score corresponding to the body state information, it specifically implements the following: The temperature level corresponding to the temperature information is determined based on the preset mapping relationship between temperature and quantization level. Based on the preset mapping relationship between temperature levels and scores, the score corresponding to the temperature level is determined as the second score.

25. The apparatus according to claim 15, characterized in that, The stress information includes stress response time and stress response intensity. When the processor performs quantification and grading of the stress information to obtain the third score corresponding to the stress information, it specifically implements the following: The stress level corresponding to the stress response time is determined based on the preset mapping relationship between the reaction time and the quantification level. Based on the preset mapping relationship between the response intensity and the quantification coefficient, the stress coefficient corresponding to the stress response intensity is determined; The third score is determined based on the stress level and the stress coefficient.

26. The apparatus according to any one of claims 15 to 25, characterized in that, When the processor implements the alarm information corresponding to the output, it specifically implements the following: An alarm command is sent to the wearable device, so that the wearable device can output the alarm information according to the alarm command.

27. The apparatus according to any one of claims 15 to 25, characterized in that, The alarm information includes at least one of vibration alerts and voice alerts.

28. The apparatus according to claim 27, characterized in that, Different levels of fatigue correspond to different prompting parameters of the vibration prompting information and / or the voice prompting information, and the prompting parameters include vibration amplitude, vibration duration, and voice volume.

29. A transportation device, characterized in that, The transportation equipment includes a fatigue driving monitoring device as described in any one of claims 15 to 28.

30. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to implement the fatigue driving monitoring method as described in any one of claims 1 to 14.

Images