Methods, devices, equipment and storage media for early warning of driver fatigue

Abstract

This application discloses a method, device, equipment, and storage medium for fatigue driving warning, belonging to the field of vehicle control technology. The method includes: collecting driver status information at a reference frequency; determining the driver's driving state based on the driver status information; if fatigue driving is determined based on the driver's driving state, determining a fatigue driving warning method by combining at least one of the driver's driving state and vehicle interior environmental information; and issuing a warning according to the fatigue driving warning method. Because the driver's driving state and vehicle interior environmental information are combined in determining the fatigue driving warning method, a warning method suitable for the driver under the current environment can be selected, improving the efficiency and safety of fatigue driving warnings.

Description

Technical Field

[0001] This application relates to the field of vehicle control technology, and in particular to a method, device, equipment and storage medium for early warning of fatigued driving. Background Technology

[0002] As vehicles become an indispensable part of our lives, the risk of traffic accidents caused by fatigued driving is also increasing. Fatigue driving warnings remind drivers to stay alert, reducing the accident rate caused by fatigued driving, and are an important means of ensuring safe driving. Summary of the Invention

[0003] This application provides a method, device, equipment, and storage medium for early warning of driver fatigue. The technical solution is as follows:

[0004] On one hand, embodiments of this application provide a method for warning of fatigued driving, the method comprising:

[0005] The driver's state information is collected according to a reference frequency, and the driver's state information includes at least one of the driver's blink frequency, pupil diameter length, or body bending angle.

[0006] Based on the length of the driver's pupil diameter when the vehicle starts and the length of the driver's pupil diameter after the vehicle starts, a first degree of change in the driver's pupil diameter is determined;

[0007] Based on the driver's body bending angle after the vehicle starts and the driver's body bending angle when the vehicle starts, a second degree of change in the driver's body bending angle is determined;

[0008] The driver's driving state is determined based on the driver's blink frequency, the first degree of change, and the second degree of change;

[0009] If fatigue driving is determined based on the driver's driving status, a fatigue driving warning method is determined by combining at least one of the driver's driving status and the environmental information inside the vehicle.

[0010] The warning will be issued according to the aforementioned method for warning of fatigued driving.

[0011] On the other hand, a fatigue driving warning device is provided, the device comprising:

[0012] The acquisition module is used to acquire the driver's state information according to a reference frequency. The driver's state information includes at least one of the driver's blink frequency, pupil diameter length, or body bending angle.

[0013] The first determining module is used to determine a first degree of change in the driver's pupil diameter based on the length of the driver's pupil diameter when the vehicle starts and the length of the driver's pupil diameter after the vehicle starts; to determine a second degree of change in the driver's body bending angle based on the driver's body bending angle after the vehicle starts and the driver's body bending angle when the vehicle starts; and to determine the driver's driving state based on the driver's blinking frequency, the first degree of change, and the second degree of change.

[0014] The second determining module is used to determine a fatigue driving warning method by combining at least one of the driver's driving state and the vehicle's internal environmental information when it is determined that fatigue driving exists based on the driver's driving state.

[0015] The warning module is used to issue warnings according to the aforementioned fatigue driving warning method.

[0016] On the other hand, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor, so that the computer device implements any of the above-described methods for warning of fatigued driving.

[0017] On the other hand, a computer-readable storage medium is also provided, wherein at least one computer program is stored in the computer-readable storage medium, the at least one computer program being loaded and executed by a processor to enable the computer to implement any of the above-described methods for warning of fatigued driving.

[0018] On the other hand, a computer program product or computer program is also provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform any of the above-described methods for warning of fatigued driving.

[0019] The technical solution provided in this application has at least the following beneficial effects:

[0020] This application collects driver status information according to a reference frequency; determines the driver's driving status based on the driver status information; and combines the driver's driving status and the vehicle's internal environmental information in the process of determining the early warning method for fatigue driving. Therefore, it can select an early warning method suitable for the driver under the current environment, thereby improving the early warning efficiency and safety of fatigue driving. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of an implementation environment provided in an embodiment of this application;

[0023] Figure 2 This is a flowchart of a fatigue driving early warning method provided in an embodiment of this application;

[0024] Figure 3 This is a schematic diagram of the structure of a fatigue driving warning device provided in an embodiment of this application;

[0025] Figure 4 This is a schematic diagram of the structure of a server provided in an embodiment of this application;

[0026] Figure 5 This is a schematic diagram of the structure of a fatigue driving warning device provided in an embodiment of this application. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0028] This application provides a method for early warning of driver fatigue. Please refer to the following embodiments. Figure 1 The diagram illustrates the implementation environment of the method provided in this application embodiment. This implementation environment may include: a vehicle 11 and a vehicle control system 12.

[0029] The vehicle 11 is equipped with a display screen, audio system, and fragrance system, capable of playing sound and video information and releasing odorous gases within the vehicle 11. The vehicle control system 12, combining at least one of the driver's driving state and environmental information within the vehicle 11, determines a fatigue driving warning method. Based on this warning method, the vehicle control system 12 controls at least one of the vehicle 11's display screen, audio system, and fragrance system to issue a fatigue warning to the driver. The vehicle control system 12 can store the driver's personal preference information and prioritize implementing fatigue driving warnings according to the driver's set personal preferences.

[0030] Optionally, the vehicle control system 12 can be a single server, a server cluster consisting of multiple servers, or a cloud computing service center. The vehicle 11 and the vehicle control system 12 establish a communication connection via a wired or wireless network.

[0031] Optionally, the implementation environment may also include server 13, which may be a single server, a server cluster consisting of multiple servers, or a cloud computing service center. Vehicle 11 and vehicle control system 12 establish communication connections with server 13 via wired or wireless networks.

[0032] Those skilled in the art should understand that vehicle 11 and vehicle control system 12 are merely examples, and other existing or future terminals or servers that are applicable to this application should also be included within the scope of protection of this application, and are hereby incorporated by reference.

[0033] Based on the above Figure 1 The implementation environment shown in this application provides a method for early warning of fatigued driving. Figure 2 As shown, the fatigue driving warning method can be executed by the vehicle control system, and the method includes steps 201-204.

[0034] In step 201, the driver's status information is collected according to the reference frequency.

[0035] This application does not limit the device used to collect driver status information. Taking a face recognition detector and an infrared sensor installed inside the vehicle as an example, a face recognition detector can be installed in front of the driver's face and an infrared sensor can be installed on the side of the driver's seat. The face recognition detector can collect the driver's facial expression changes and the size of his facial features, while the infrared sensor can collect the driver's body bending angle.

[0036] In one possible implementation, driver status information is collected according to a reference frequency, including: collecting the driver's blink frequency and pupil size using a facial recognition detector at the reference frequency; the pupil size collection includes collecting the length of the driver's pupil diameter. The driver's body bending angle is also collected using an infrared sensor at the reference frequency.

[0037] This application does not limit the reference frequency; it can be set based on experience or adjusted based on the scenario. The unit for the driver's blinking frequency collected by the facial recognition detector can be times per minute, the unit for the driver's pupil diameter can be millimeters, and the unit for the driver's body bending angle collected by the infrared sensor can be degrees.

[0038] In step 202, the driver's driving status is determined based on the driver's status information.

[0039] The driver's state information includes at least one of the driver's blink frequency, pupil diameter length, or body flexion angle. Determining the driver's driving state based on the driver's state information includes: when the driver's state information includes the driver's blink frequency, determining a first degree of change in the driver's pupil diameter based on the length of the driver's pupil diameter when the vehicle starts and the length of the driver's pupil diameter after the vehicle starts; when the driver's state information includes the length of the pupil diameter, determining a second degree of change in the driver's body flexion angle based on the angle of the driver's body flexion angle after the vehicle starts and the angle of the driver's body flexion angle when the vehicle starts; and when the driver's state information includes the body flexion angle, determining the driver's driving state based on the driver's blink frequency, the first degree of change, and the second degree of change.

[0040] When the vehicle control system detects that the vehicle is starting, it takes the diameter of the driver's pupil at that moment as the diameter of the driver's pupil when the vehicle is starting, and takes the angle of the driver's body bending collected by the infrared sensor at that moment as the angle of the driver's body bending when the vehicle is starting.

[0041] The calculation and processing of the driver's state information after vehicle startup, based on the length of the pupil diameter at vehicle startup and the driver's body bending angle at vehicle startup, includes: dividing the length of the driver's pupil diameter at vehicle startup by the length of the driver's pupil diameter collected after vehicle startup to obtain the first degree of change; and dividing the bending angle of the driver's upper body collected after vehicle startup by the bending angle of the driver's upper body at vehicle startup to obtain the second degree of change.

[0042] The driver's driving state is determined to be Level 1 fatigue when any one of the following conditions is met: the driver's blink frequency is less than the blink threshold, the first degree of change is greater than the first degree of change, or the second degree of change is greater than the second degree of change. The driver's driving state is determined to be Level 2 fatigue when any two of the following conditions are met: the driver's blink frequency is less than the blink threshold, the first degree of change is greater than the first degree of change, or the second degree of change is greater than the second degree of change. The driver's driving state is determined to be Level 3 fatigue when all three conditions are met: the driver's blink frequency is less than the blink threshold, the first degree of change is greater than the first degree of change, or the second degree of change is greater than the second degree of change.

[0043] Among level 1, level 2, and level 3 fatigue, the degree of fatigue increases with the increase of the fatigue level; that is, the degree of fatigue of level 3 fatigue is greater than that of level 2 fatigue, and the degree of fatigue of level 2 fatigue is greater than that of level 1 fatigue. This application does not limit the first and second change thresholds; they can be set based on experience or adjusted based on the scenario.

[0044] In step 203, if fatigue driving is determined based on the driver's driving state, a fatigue driving warning method is determined by combining at least one of the driver's driving state and the environmental information inside the vehicle.

[0045] In one possible implementation, the environmental information inside the vehicle includes at least one of the following: the vehicle's interior temperature, the vehicle's interior light intensity, and the vehicle's interior volume. This application does not limit the equipment used to collect the vehicle's interior environmental information. Taking the collection of vehicle interior environmental information using a temperature sensor, a light intensity sensor, and a sound detector installed on the vehicle as an example, these sensors can be installed inside the vehicle. If it is determined that the driver is fatigued, the temperature inside the vehicle can be collected using the temperature sensor, the light intensity inside the vehicle can be collected using the light intensity sensor, and the volume inside the vehicle can be collected using the sound detector.

[0046] The temperature sensor can measure temperature in degrees Celsius, the light intensity sensor can measure light intensity in lux, and the sound detector can measure volume in decibels.

[0047] The vehicle's internal environmental information includes at least one of the following: internal temperature, internal light intensity, or internal volume. Combining this information with the driver's driving state and at least one of the internal environmental factors, a fatigue driving warning method is determined. This includes: based on the driver's level one or two fatigue status and the internal temperature being below a temperature threshold, the warning method is determined to be at least one of auditory and visual methods; based on the internal light intensity being above a light threshold, the warning method is determined to be at least one of auditory and olfactory methods; based on the internal volume being above a volume threshold, the warning method is determined to be at least one of visual and olfactory methods; and based on the driver's level three fatigue status, the warning method is determined to be at least one of olfactory methods.

[0048] When a driver is experiencing Level 1 or Level 2 fatigue, and the vehicle's interior temperature is below a temperature threshold, airflow is slow. To improve the efficiency of fatigue warning, at least one of auditory and visual methods is used to warn the driver. When the light intensity inside the vehicle exceeds a light threshold, visual information is obstructed. To improve the efficiency of fatigue warning, at least one of auditory and olfactory methods is used to warn the driver. When the volume inside the vehicle exceeds a volume threshold, audio information is obstructed. To improve the efficiency of fatigue warning, at least one of visual and olfactory methods is used to warn the driver.

[0049] This application does not impose restrictions on temperature thresholds, light thresholds, and volume thresholds; these can be set based on experience or adjusted based on the scenario.

[0050] When a driver is in a state of level three fatigue, choosing auditory or visual warning methods to warn the driver may easily cause the driver to be in a state of stress. In order to improve driving safety and the effectiveness of fatigue driving warning, olfactory warning methods are chosen to warn the driver.

[0051] In step 204, a warning is issued to the driver in accordance with the warning method for fatigued driving.

[0052] In one possible implementation, after determining the fatigue driving warning method for different situations by combining the driver's driving state and the environmental conditions inside the vehicle, the driver is warned according to the fatigue driving warning method. This includes, if the fatigue driving warning method includes auditory warning, warning the driver by playing sound information inside the vehicle; if the fatigue driving warning method includes visual warning, warning the driver by playing video information inside the vehicle; if the fatigue driving warning method includes olfactory warning, warning the driver by releasing odorous gas inside the vehicle; if the driver's driving state is at level three fatigue, warning the driver by releasing odorous gas at level one concentration inside the vehicle; and if the driver's driving state is at level one or level two fatigue, warning the driver by releasing odorous gas at level two concentration inside the vehicle.

[0053] This application does not restrict the primary and secondary concentrations, but the primary concentration must be greater than the secondary concentration. This application does not restrict the type of gas selected, but the selected gas must have an odor, the odor must increase with the increase of gas concentration, and it must be harmless to the human body at any concentration.

[0054] This application does not limit the devices used to play sound or video information or release odorous gases inside a vehicle; these can be achieved through in-vehicle displays, audio systems, and fragrance systems. When the fatigue driving warning method includes an auditory warning, the in-vehicle audio system plays sound information to remind the driver to stay alert. When the fatigue driving warning method includes a visual warning, the in-vehicle display plays video information to remind the driver to stay alert. When the fatigue driving warning method includes an olfactory warning and the driver's fatigue level is level three, the in-vehicle fragrance system releases a level one concentration of a harmless, irritating gas to warn the driver. When the fatigue driving warning method includes an olfactory warning and the driver's fatigue level is level one or two, the in-vehicle fragrance system releases a level two concentration of a harmless, irritating gas to warn the driver.

[0055] In one possible implementation, the method provided in this application further includes: feeding back warning information to the driver. For example, after the vehicle's in-vehicle display screen, audio system, and fragrance system begin implementing a warning for driver fatigue, the vehicle control system transmits the warning information to the vehicle's display screen, which then displays the warning information. The warning information includes the driver's driving status, the type of fatigue driving warning, and the progress of the warning action.

[0056] After receiving the warning information displayed on the screen, the driver can set personal preference information based on their individual driving habits. This personal preference information includes the maximum driving reminder duration and preferred warning method. Upon receiving this personal preference information, the vehicle control system prioritizes implementing fatigue driving warnings according to the driver's settings. This includes: if the vehicle control system determines that the driver's driving state is not fatigued, but the driving time exceeds the driver's set maximum driving reminder duration, a fatigue driving warning will still be activated; if the driver's fatigue state is determined to be Level 1 or Level 2 fatigue, the driver's preferred warning method will be used.

[0057] This application collects driver status information according to a reference frequency; determines the driver's driving status based on the driver status information; and combines the driver's driving status and the vehicle's internal environmental information in the process of determining the early warning method for fatigue driving. Therefore, it can select an early warning method suitable for the driver under the current environment, thereby improving the early warning efficiency and safety of fatigue driving.

[0058] See Figure 3 This application provides a fatigue driving warning device, which includes:

[0059] The data acquisition module 301 is used to collect the driver's status information according to a reference frequency;

[0060] The first determining module 302 is used to determine the driver's driving status based on the driver's status information;

[0061] The second determining module 303 is used to determine a fatigue driving warning method by combining at least one of the driver's driving state and the environmental information inside the vehicle when it is determined that fatigue driving exists based on the driver's driving state.

[0062] The warning module 304 is used to issue warnings according to the warning method for fatigued driving.

[0063] In one possible implementation, the driver's state information includes at least one of the driver's blink frequency, pupil diameter length, or body bending angle; the first determining module 302 is used to determine a first degree of change in the driver's pupil diameter based on the length of the driver's pupil diameter when the vehicle starts and the length of the driver's pupil diameter after the vehicle starts; to determine a second degree of change in the driver's body bending angle based on the driver's body bending angle after the vehicle starts and the driver's body bending angle when the vehicle starts; and to determine the driver's driving state based on the driver's blink frequency, the first degree of change, and the second degree of change.

[0064] In one possible implementation, the environmental information inside the vehicle includes at least one of the following: the temperature inside the vehicle, the light intensity inside the vehicle, or the volume inside the vehicle; the second determining module 303 is used to determine, based on the temperature inside the vehicle being lower than a temperature threshold, that the driver is being warned by at least one of auditory and visual means; based on the light intensity inside the vehicle being higher than a light threshold, that the driver is being warned by at least one of auditory and olfactory means; and based on the volume inside the vehicle being higher than a volume threshold, that the driver is being warned by at least one of visual and olfactory means.

[0065] In one possible implementation, the driver's driving state includes three levels of fatigue; the three levels of fatigue are the driver's driving state when the driver's blinking frequency is less than the blinking threshold, the first degree of change is greater than the first degree of change, and the second degree of change is greater than the second degree of change; the second determining module 303 is used to determine the fatigue driving warning method as a warning to the driver through smell based on the driver's driving state being three levels of fatigue.

[0066] In one possible implementation, the warning module 304 is used to warn the driver by playing sound information inside the vehicle when the warning method for fatigue driving includes an auditory warning method; to warn the driver by playing video information inside the vehicle when the warning method for fatigue driving includes a visual warning method; and to warn the driver by releasing odorous gas inside the vehicle when the warning method for fatigue driving includes an olfactory warning method.

[0067] In one possible implementation, the warning module 304 is used to issue a warning to the driver via olfactory means by releasing a first-level concentration of odorous gas inside the vehicle when the driver's driving state is at level three fatigue. Level three fatigue is defined as the driver's driving state when the driver's blinking frequency is less than a blinking threshold, the first degree of change is greater than the first degree of change, and the second degree of change is greater than the second degree of change. When the driver's driving state is at level one or level two fatigue, the warning is issued to the driver via olfactory means by releasing a second-level concentration of odorous gas inside the vehicle. The first-level concentration is higher than the second-level concentration. Level one fatigue is defined as the driver's driving state when any one of the following conditions is met: the driver's blinking frequency is less than a blinking threshold, the first degree of change is greater than the first degree of change, and the second degree of change is greater than the second degree of change. Level two fatigue is defined as the driver's driving state when any two of the following conditions are met: the driver's blinking frequency is less than a blinking threshold, the first degree of change is greater than the first degree of change, and the second degree of change is greater than the second degree of change.

[0068] The aforementioned device collects the driver's status information according to a reference frequency; it determines the driver's driving status based on the driver's status information. In determining the warning method for fatigue driving, it combines the driver's driving status with the environmental information inside the vehicle. Therefore, it can select a warning method suitable for the driver under the current environment, thereby improving the warning efficiency and safety of fatigue driving.

[0069] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0070] Figure 4This is a schematic diagram of a server structure provided in an embodiment of this application. The server can vary significantly due to differences in configuration or performance. It may include one or more processors 901 and one or more memories 902. The one or more memories 902 store at least one computer program, which is loaded and executed by the one or more processors 901 to enable the server to implement the fatigue driving warning method provided in the above-described method embodiments. Of course, the server may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server may also include other components for implementing device functions, which will not be elaborated upon here.

[0071] Figure 5 This is a schematic diagram of a fatigue driving warning device provided in an embodiment of this application. The device can be a terminal, such as a smartphone, tablet, media player, laptop, or desktop computer. The terminal may also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, or other names.

[0072] Typically, a terminal includes a processor 1501 and a memory 1502.

[0073] Processor 1501 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1501 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1501 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 1501 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, processor 1501 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0074] The memory 1502 may include one or more computer-readable storage media, which may be non-transitory. The memory 1502 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 1502 is used to store at least one instruction, which is executed by the processor 1501 to cause the terminal to implement the fatigue driving warning method provided in the method embodiments of this application.

[0075] In some embodiments, the terminal may also optionally include: a peripheral device interface 1503 and at least one peripheral device. The processor 1501, memory 1502, and peripheral device interface 1503 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 1503 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of: a radio frequency circuit 1504, a display screen 1505, a camera assembly 1506, an audio circuit 1507, and a power supply 1508.

[0076] Peripheral interface 1503 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 1501 and memory 1502. In some embodiments, processor 1501, memory 1502 and peripheral interface 1503 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 1501, memory 1502 and peripheral interface 1503 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0077] The radio frequency (RF) circuit 1504 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 1504 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 1504 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 1504 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 1504 can communicate with other terminals through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: metropolitan area networks (MANs), various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks (WLANs), and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 1504 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0078] Display screen 1505 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 1505 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 1501 for processing. In this case, display screen 1505 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, display screen 1505 can be a single screen, located on the front panel of the terminal; in other embodiments, display screen 1505 can be at least two screens, respectively located on different surfaces of the terminal or in a folded design; in other embodiments, display screen 1505 can be a flexible display screen, located on a curved or folded surface of the terminal. Furthermore, display screen 1505 can be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. Display screen 1505 can be made of materials such as LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode).

[0079] The camera assembly 1506 is used to acquire images or videos. Optionally, the camera assembly 1506 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 1506 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0080] The audio circuit 1507 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 1501 for processing, or input to the radio frequency circuit 1504 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each positioned at a different location on the terminal. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from the processor 1501 or the radio frequency circuit 1504 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 1507 may also include a headphone jack.

[0081] Power supply 1508 is used to power the various components in the terminal. Power supply 1508 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 1508 includes a rechargeable battery, the rechargeable battery can support wired or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0082] In some embodiments, the terminal further includes one or more sensors 1509. The one or more sensors 1509 include, but are not limited to: an acceleration sensor 1510, a gyroscope sensor 1511, a pressure sensor 1512, an optical sensor 1513, and a proximity sensor 1514.

[0083] Accelerometer 1510 can detect the magnitude of acceleration along the three coordinate axes of a coordinate system established by the terminal. For example, accelerometer 1510 can be used to detect the components of gravitational acceleration along the three coordinate axes. Processor 1501 can control display screen 1505 to display the user interface in either a landscape or portrait view based on the gravitational acceleration signal acquired by accelerometer 1510. Accelerometer 1510 can also be used for games or for acquiring user motion data.

[0084] The gyroscope sensor 1511 can detect the terminal's orientation and rotation angle. The gyroscope sensor 1511 can work in conjunction with the accelerometer sensor 1510 to collect the user's 3D movements on the terminal. Based on the data collected by the gyroscope sensor 1511, the processor 1501 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0085] The pressure sensor 1512 can be disposed on the side bezel of the terminal and / or the lower layer of the display screen 1505. When the pressure sensor 1512 is disposed on the side bezel of the terminal, it can detect the user's grip signal on the terminal, and the processor 1501 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 1512. When the pressure sensor 1512 is disposed on the lower layer of the display screen 1505, the processor 1501 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 1505. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0086] Optical sensor 1513 is used to collect ambient light intensity. In one embodiment, processor 1501 can control the display brightness of display screen 1505 based on the ambient light intensity collected by optical sensor 1513. Specifically, when the ambient light intensity is high, the display brightness of display screen 1505 is increased; when the ambient light intensity is low, the display brightness of display screen 1505 is decreased. In another embodiment, processor 1501 can also dynamically adjust the shooting parameters of camera assembly 1506 based on the ambient light intensity collected by optical sensor 1513.

[0087] The proximity sensor 1514, also known as a distance sensor, is typically installed on the front panel of the terminal. The proximity sensor 1514 is used to detect the distance between the user and the front of the terminal. In one embodiment, when the proximity sensor 1514 detects that the distance between the user and the front of the terminal is gradually decreasing, the processor 1501 controls the display screen 1505 to switch from a screen-on state to a screen-off state; when the proximity sensor 1514 detects that the distance between the user and the front of the terminal is gradually increasing, the processor 1501 controls the display screen 1505 to switch from a screen-off state to a screen-on state.

[0088] Those skilled in the art will understand that Figure 5 The structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0089] In an exemplary embodiment, a computer device is also provided, comprising a processor and a memory storing at least one computer program. The at least one computer program is loaded and executed by one or more processors to enable the computer device to implement any of the aforementioned methods for providing a driver fatigue warning.

[0090] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one computer program that is loaded and executed by a processor of a computer device to enable the computer to implement any of the above-described methods for warning of fatigued driving.

[0091] In one possible implementation, the aforementioned computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0092] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the above-described methods for providing a fatigue driving warning.

[0093] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the driver's status information and the driver's personal preference information involved in this application were obtained with full authorization.

[0094] It should be understood that "multiple" as used in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0095] It should be noted that the terms "first," "second," etc. (if applicable) in the specification and claims of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0096] The above description is merely an exemplary embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A method for early warning of driver fatigue, characterized in that, The method includes: The driver's state information is collected according to a reference frequency, including the driver's blink frequency, pupil diameter, or body bending angle. Based on the length of the driver's pupil diameter when the vehicle starts and the length of the driver's pupil diameter after the vehicle starts, a first degree of change in the driver's pupil diameter is determined; Based on the driver's body bending angle after the vehicle starts and the driver's body bending angle when the vehicle starts, a second degree of change in the driver's body bending angle is determined; The driver's driving state is determined based on the driver's blink frequency, the first degree of change, and the second degree of change; The environmental information inside the vehicle includes at least one of the following: the temperature inside the vehicle, the light intensity inside the vehicle, or the volume of sound inside the vehicle. If fatigue driving is determined based on the driver's driving state, and the temperature inside the vehicle is below a temperature threshold, the fatigue driving warning method is determined to be to warn the driver using at least one of auditory and visual methods. Based on the fact that the light intensity inside the vehicle is higher than the light threshold, the fatigue driving warning method is determined to be to warn the driver using at least one of the senses of hearing and smell. Based on the fact that the volume inside the vehicle is greater than the volume threshold, the fatigue driving warning method is determined to be to warn the driver using at least one of visual and olfactory methods. The warning will be issued according to the aforementioned method for warning of fatigued driving.

2. The method according to claim 1, characterized in that, The driver's driving state includes three levels of fatigue; the three levels of fatigue are the driver's driving state when the driver's blinking frequency is less than the blinking threshold, the first degree of change is greater than the first degree of change threshold, and the second degree of change is greater than the second degree of change threshold. By combining at least one of the driver's driving status and the vehicle's internal environmental information, a fatigue driving warning method is determined, including: Based on the driver's driving state being classified as Level 3 fatigue, the early warning method for fatigued driving is determined to be to provide an early warning to the driver through the sense of smell.

3. The method according to claim 1, characterized in that, The method of issuing a warning according to the fatigue driving warning method includes: When the fatigue driving warning method includes an auditory warning method, the driver is warned by playing sound information inside the vehicle. When the fatigue driving warning method includes a visual warning method, the driver is warned by playing video information inside the vehicle. When the fatigue driving warning method includes an olfactory warning method, the driver is warned by releasing an odorous gas inside the vehicle.

4. The method according to claim 3, characterized in that, The method of issuing a warning to the driver by releasing an odorous gas inside the vehicle includes: When the driver's driving state is at level three fatigue, an odorous gas of level one concentration is released inside the vehicle to provide an olfactory warning to the driver. Level three fatigue is the driver's driving state when the driver's blinking frequency is less than a blinking threshold, the first degree of change is greater than a first degree of change threshold, and the second degree of change is greater than a second degree of change threshold. When the driver's driving state is at level one or level two fatigue, an olfactory warning is issued to the driver by releasing a level two concentration of odorous gas inside the vehicle. The level one concentration is higher than the level two concentration. Level one fatigue is the driver's driving state when any one of the following conditions is met: the driver's blinking frequency is less than a blinking threshold, the first degree of change is greater than a first degree of change, and the second degree of change is greater than a second degree of change. Level two fatigue is the driver's driving state when any two of the following conditions are met: the driver's blinking frequency is less than a blinking threshold, the first degree of change is greater than a first degree of change, and the second degree of change is greater than a second degree of change.

5. A fatigue driving warning device, characterized in that, The device includes: The acquisition module is used to acquire the driver's state information according to a reference frequency. The driver's state information includes the driver's blink frequency, pupil diameter length, or body bending angle. The first determining module is used to determine a first degree of change in the driver's pupil diameter based on the length of the driver's pupil diameter when the vehicle starts and the length of the driver's pupil diameter after the vehicle starts; to determine a second degree of change in the driver's body bending angle based on the driver's body bending angle after the vehicle starts and the driver's body bending angle when the vehicle starts; and to determine the driver's driving state based on the driver's blinking frequency, the first degree of change, and the second degree of change. The environmental information inside the vehicle includes at least one of the following: the temperature inside the vehicle, the light intensity inside the vehicle, or the volume inside the vehicle. The second determining module is configured to, when determining that fatigue driving exists based on the driver's driving state, determine the fatigue driving warning method as follows: if the temperature inside the vehicle is below a temperature threshold, determine the fatigue driving warning method as at least one of auditory and visual methods; if the light intensity inside the vehicle is above a light threshold, determine the fatigue driving warning method as at least one of auditory and olfactory methods; if the volume inside the vehicle is above a volume threshold, determine the fatigue driving warning method as at least one of visual and olfactory methods. The warning module is used to issue warnings according to the aforementioned fatigue driving warning method.

6. The apparatus according to claim 5, characterized in that, The driver's driving condition includes three levels of fatigue; The three levels of fatigue refer to the driver's driving state when the driver's blinking frequency is less than the blinking threshold, the first degree of change is greater than the first degree of change, and the second degree of change is greater than the second degree of change. The second determining module is used to determine, based on the driver's driving state being level three fatigue, the fatigue driving warning method as issuing a warning to the driver through smell.

7. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to enable the computer device to implement the fatigue driving warning method as described in any one of claims 1 to 4.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one computer program, which is loaded and executed by a processor to enable the computer to implement the fatigue driving warning method as described in any one of claims 1 to 4.

Images