Driving assistance method, driving assistance program, and driving assistance system
The driving support system adjusts detection ranges for hazards based on driver alertness, providing timely and relevant warnings to enhance safety and reduce annoyance, addressing the issue of bothersome notifications in existing systems.
Patent Information
- Application Number
- JP2020129328
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-07-30
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2040-07-30
AI Technical Summary
Existing driving assistance systems that detect drowsiness and other hazards can be bothersome, leading drivers to disable the features, and there is a need for a method that supports safe driving without frequent notifications.
A driving support system that adjusts the detection range for potential hazards based on the driver's wakefulness state, using facial image data to determine alertness and set appropriate warning thresholds.
The system provides timely and relevant warnings, reducing driver annoyance and enhancing safety by tailoring notifications to the driver's alertness level, thereby supporting safe driving without unnecessary disturbances.
Smart Images

Figure 0007589400000001 
Figure 0007589400000002 
Figure 0007589400000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a driving assistance method, a driving assistance program, and a driving assistance system. [Background technology]
[0002] Falling asleep while driving a vehicle has become a social problem. There is known a device that detects a driver's signs of falling asleep while driving and alerts the driver (see, for example, Patent Document 1). [Prior art documents] [Patent documents]
[0003] [Patent Document 1] Special Publication No. 2016-539446 Summary of the Invention [Problem to be solved by the invention]
[0004] Driving support devices that detect not only drowsiness but also pedestrians jumping out into the road and traffic signs are becoming more common. While driving support devices that warn drivers of various dangers in traffic conditions are appreciated by drivers, frequent warnings can be annoying, and some drivers may turn off the driving support functions.
[0005] The present invention has been made to solve such problems, and provides a driving assistance method and the like that appropriately assists a driver in safe driving without being bothersome to the driver. [Means for solving the problem]
[0006] A driving assistance method in a first aspect of the present invention includes a first acquisition step of acquiring facial image data capturing an image of the face of a driver driving a vehicle, a determination step of judging the driver's state of alertness from the facial image data, a second acquisition step of acquiring forward image data capturing an image of the area in front of the vehicle, and a warning step of issuing a warning of a warning event detected within a detection range that is set by expanding or contracting the depth along the vehicle's traveling direction relative to the forward image of the forward image data based on the driver's state of alertness.
[0007] A driving assistance method in a second aspect of the present invention includes a first acquisition step of acquiring facial image data capturing an image of the face of a driver driving a vehicle, a determination step of judging the driver's state of alertness from the facial image data, a second acquisition step of acquiring forward image data capturing an image of the area in front of the vehicle, and a warning step of issuing a warning of a warning event detected for an object appearing in the forward image of the forward image data according to a threshold value set based on the driver's state of alertness.
[0008] A driving assistance program according to a third aspect of the present invention is a program for causing a computer to realize the above driving assistance method.
[0009] A driving assistance system in a fourth aspect of the present invention includes an acquisition unit that acquires facial image data that captures the face of a driver driving a vehicle and forward image data that captures the area in front of the vehicle, a determination unit that determines the driver's state of alertness from the facial image data, and a warning unit that warns of a warning event detected within a detection range that is set by expanding or contracting the depth along the vehicle's traveling direction relative to the forward image of the forward image data based on the driver's state of alertness. Effect of the Invention
[0010] According to the present invention, it is possible to provide a driving support program or the like that is not bothersome to the driver and that appropriately supports safe driving. [Brief description of the drawings]
[0011] [Figure 1] FIG. 2 is a schematic diagram showing a state in which the traveling direction is observed from inside the cabin of the host vehicle. [Diagram 2] FIG. 1 is a block diagram showing a configuration of a smartphone. [Diagram 3] 1A and 1B are diagrams illustrating a process of detecting feature points from a face image. [Figure 4] 11A and 11B are diagrams illustrating a procedure for calculating an eye opening rate from facial feature points. [Diagram 5] 11A and 11B are diagrams illustrating a procedure for calculating an aperture ratio from facial feature points. [Figure 6] 11A and 11B are diagrams illustrating a procedure for calculating the head center of gravity from facial feature points. [Figure 7] FIG. 13 is a diagram showing a procedure for detecting a warning event from a forward image when the warning level is 1. [Figure 8] FIG. 13 is a diagram showing a procedure for detecting a warning event from a forward image when the warning level is 2. [Figure 9] FIG. 13 is a diagram showing a procedure for detecting a warning event from a forward image when the warning level is 3. [Figure 10] FIG. 4 is a flow diagram showing a processing flow of driving assistance. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0012] The present invention will be described below through embodiments of the invention, but the invention according to the claims is not limited to the following embodiments. Furthermore, not all of the configurations described in the embodiments are necessarily essential as means for solving the problems.
[0013] FIG. 1 is a schematic diagram showing a state observed in the traveling direction from inside the vehicle cabin. This diagram shows a state in which the vehicle is traveling on a road, excluding the driver. The smartphone 100 is, for example, a smartphone that the driver carries with him or her on a daily basis, and is placed in a holder or the like installed on the dashboard when driving the vehicle. A driving assistance program is installed in the smartphone 100, and the smartphone 100 executes the driving assistance program in response to an instruction from the driver or when the smartphone 100 detects the traveling of the vehicle. The smartphone 100 functions as a driving assistance system by executing the driving assistance program.
[0014] The smartphone 100 includes an in-camera unit 120 that captures an image of the space on the side where the display 140 is provided, and an out-camera unit 130 that captures an image of the space on the opposite side. When the driver places the smartphone 100 on a holder or the like, the in-camera unit 120 can capture an image of the face of the driver sitting in the driver's seat, and the out-camera unit 130 can capture an image of the front, which is the traveling direction of the vehicle, through the windshield.
[0015] The smartphone 100 judges the wakefulness state of the driver from the face image of the driver captured by the in-camera unit 120. Then, for the forward image captured by the out-camera unit 130, a detection range is set based on the determined wakefulness state of the driver, and it is detected whether or not there is a warning event that should be warned within the detection range, and if there is, it warns the driver. As will be described in detail later, when a pedestrian 901 is detected within the detection range set in a certain wakefulness state, for example, as shown in the figure, the smartphone 100 highlights the pedestrian in the forward image displayed on the display 140 by surrounding it with a frame. In addition, the presence of the pedestrian is notified to the driver by a warning sound or voice.
[0016] 2 is a block diagram showing the configuration of the smartphone 100. The smartphone 100 is mainly composed of a processing unit 110, an in-camera unit 120, an out-camera unit 130, a display 140, a speaker 150, an input device 160, a memory 170, and a communication unit 18. The processing unit 110 is a processor (CPU: Central Processing Unit) that controls the smartphone 100 and executes programs. The processor may be configured to cooperate with an arithmetic processing chip such as an ASIC (Application Specific Integrated Circuit) or a GPU (Graphics Processing Unit). The processing unit 110 reads out a driving assistance program stored in the memory 170 and executes various processes related to driving assistance.
[0017] When the smartphone 100 functions as a driving assistance system, the in-camera unit 120 is used as a camera unit for capturing an image of the driver's face as described above. When the in-camera unit 120 receives an image capture instruction from the processing unit 110, it causes the imaging sensor to output an image signal, generates face image data from the image signal, and transfers it to the processing unit 110. When the smartphone 100 functions as a driving assistance system, the out-camera unit 130 is used as a camera unit for capturing an image of the front of the vehicle as described above. When the out-camera unit 130 receives an image capture instruction from the processing unit 110, it causes the imaging sensor to output an image signal, generates front image data from the image signal, and transfers it to the processing unit 110.
[0018] The display 140 is, for example, a display equipped with a liquid crystal panel, and displays a forward image captured by the outer camera unit 130 while the vehicle is traveling, and displays a warning when a warning event is detected. The speaker 150 issues a warning sound or voice when a warning event is detected. The input device 160 is a device that accepts user input, and is, for example, a touch panel superimposed on the display panel. It may also include a voice recognition unit that accepts instructions by the user's voice.
[0019] The memory 170 is a non-volatile storage medium, and is configured, for example, by a flash memory. The memory 170 stores a program for controlling and processing the smartphone 100, as well as a trained model for detecting warning events and parameters used for various calculations. The communication unit 180 transmits and receives data to and from a communication base station and the Internet network. The communication unit 180 is configured, for example, by a communication modem or a wireless LAN unit.
[0020] The processing unit 110 also plays a role as a functional calculation unit that executes various calculations according to the processing supported by the driving assistance program. The processing unit 110 can function as an acquisition unit 111, a determination unit 112, a detection unit 113, and a warning unit 114. The acquisition unit 111 acquires face image data generated by the in-camera unit 120 and forward image data generated by the out-camera unit 130. The determination unit 112 detects feature points of the driver's face from the face image of the face image data, and determines the driver's wakefulness state by observing the driver's eye opening, yawning, and head shaking based on the feature points. The determination unit 112 includes an eye opening calculation unit 112a that performs calculations to observe the driver's eye opening, a yawn calculation unit 112b that performs calculations to observe the yawning, and a shaking calculation unit 112c that performs calculations to observe the head shaking. Specific calculation processes will be described later.
[0021] The detection unit 113 detects a warning event that the driver should be aware of in order to drive the vehicle safely from the forward image of the forward image data. The detection unit 113 includes a range setting unit 113a that sets a detection range for detecting the warning event. Specific calculation processing will be described later. When the detection unit 113 detects a warning event, the warning unit 114 warns the driver of the warning event. For example, a warning is displayed on the display 140 or a warning sound is generated via the speaker 150.
[0022] 3 is a diagram for explaining the process of detecting feature points from a face image. When the acquisition unit 111 acquires face image data from the in-camera unit 120, the determination unit 112 identifies the driver's face area from the developed face image and trims a rectangular area including the face area. Then, the image processing is performed to extract the eyebrow boundary line, upper and lower eyelid boundary line, nose outline, lip boundary line, and chin outline, determine feature points at set intervals, and acquire the coordinates of these feature points.
[0023] FIG. 4 is a diagram for explaining a procedure for calculating the eye-opening rate from the facial feature points. The eye-opening calculation unit 112a selects the feature points along the boundary between the upper and lower eyelids as the eye feature points from among the facial feature points. Then, for each of the right and left eyes, the eye-opening rate=(a+b) / c is calculated. Here, a and b correspond to the vertical width at each position where the horizontal width of the eye is divided into three equal parts, and are calculated as the distance between the selected feature points. Also, c corresponds to the horizontal width of the eye, and is calculated as the distance between the most distant feature points. In addition, in order to avoid the influence of blinking, the calculation of the eye-opening rate may be performed by averaging the values calculated in each of the most recently acquired images, or by eliminating values that change at a rate equal to or greater than a threshold value compared to the previous and next eye-opening rates.
[0024] The smaller the eye opening rate of the right eye and the left eye calculated in this way is, the more likely the driver is to close his / her eyelids and to be in a state close to dozing. Therefore, the eye opening calculation unit 112a calculates the eye opening rate evaluation value Eα by substituting the calculated eye opening rate into a conversion formula determined in advance according to such a criterion. The conversion formula used here outputs a larger eye opening rate evaluation value Eα as the input eye opening rate is smaller.
[0025] FIG. 5 is a diagram for explaining the procedure for calculating the mouth opening ratio from facial feature points. The yawn calculation unit 112b selects the feature points along the lip boundary as the mouth feature points from among the facial feature points. Then, it calculates the mouth opening ratio==(a+b) / c. Here, a and b correspond to the vertical width at each position where the width of the mouth is divided into three equal parts, and are calculated as the distance between the selected feature points. Also, c corresponds to the width of the mouth, and is calculated as the distance between the feature points that are the furthest from each other. The mouth opening ratio may be calculated by averaging the values calculated for each of several images acquired within the duration of a standard yawn, for example.
[0026] When the calculated aperture ratio changes significantly in a short time, the driver is presumed to have yawned and is considered to be in a state close to drowsiness. Therefore, the yawn calculation unit 112b calculates the aperture ratio evaluation value Eβ by substituting the calculated aperture ratio into a conversion formula that is determined in advance according to such a criterion. The conversion formula used here outputs a large aperture ratio evaluation value Eβ when the aperture ratio changes significantly within a unit time, or when such changes are repeated.
[0027] 6 is a diagram for explaining the procedure for calculating the head center of gravity from the facial feature points. The swing calculation unit 112c selects the feature points along the boundary between the upper and lower eyelids and the feature points at the lower part of the nose outline as the head swing feature points. Then, the center of gravity of these feature points is calculated to be the head center of gravity.
[0028] If the value of the y-coordinate, which is the vertical coordinate of the head center of gravity calculated in this way, changes significantly in a short time, it is estimated that the driver is dozing off by repeatedly tilting his head back and forth, and is considered to be in a state close to drowsiness. Therefore, the sway calculation unit 112c calculates the sway evaluation value Eγ by substituting the calculated head center of gravity into a conversion formula that is predetermined according to such a criterion. The conversion formula used here outputs a large sway evaluation value Eγ when the value of the y-coordinate of the head center of gravity changes significantly within a unit time, or when such changes are repeated.
[0029] The determination unit 112 calculates the degree of awakening E as a numerical value representing the wakefulness state of the driver. Specifically, the degree of awakening E is calculated as follows: degree of awakening E=eye opening rate evaluation value Eα+eye opening rate evaluation value Eβ+sway evaluation value Eγ. The degree of awakening E calculated in this manner can be evaluated as being more likely to be dozing off as the value is larger. The selection of the feature points used to calculate the respective evaluation values described with reference to FIG. 4 to FIG. 6 is not limited to the above-mentioned case, and for example, feature points along the eyebrow boundary line or feature points along the jawline may be selected and used. In addition, in this embodiment, the degree of awakening E is calculated using three evaluation values, but evaluation values may be calculated by capturing other physical changes that may be inferred as dozing off, and used to calculate the degree of awakening E. In addition, instead of using all of the above-mentioned three evaluation values, some of them may be used. In addition, weighting may be adjusted so that changes that may strongly infer dozing off, such as the eye opening rate, contribute more to the calculation of the degree of awakening than other evaluation values.
[0030] Now, if the driver is estimated to be dozing, the smartphone 100 should immediately call the driver's attention, for example by issuing a warning sound. On the other hand, the state in which a person falls asleep is gradual, and even before the state is estimated to be dozing, if the driver is close to dozing, the driver's attention may already be reduced. In other words, when a safety-related event such as a pedestrian jumping out into the direction of travel of the vehicle occurs while driving, even if the event can be dealt with immediately if the driver is awake, the driver's reaction may be delayed if the driver's attention is reduced, which may lead to an accident.
[0031] On the other hand, driving support devices that detect safety-related events occurring in the vehicle's travel direction and warn the driver are becoming more widespread. However, when the driver is awake, he or she often senses such events early on without being warned, and the driver may find frequent warnings annoying. Some drivers may turn off such driving support functions.
[0032] In this embodiment, the detection range for detecting whether or not a warning event that may hinder safe driving has occurred in the vehicle's travel direction is changed based on the driver's alertness. More specifically, the depth of the detection range along the vehicle's travel direction is expanded or contracted depending on the driver's alertness. This will be described below with reference to the drawings.
[0033] 7 is a diagram showing a procedure for detecting a warning event from a forward image when the awakening level is warning level 1. Here, it is assumed that the vehicle is traveling in an urban area and the outer camera unit 130 captures a forward image as shown in the above figure.
[0034] When the determination unit 112 calculates the wakefulness level E as described above, the detection unit 113 classifies the wakefulness state of the driver into, for example, four warning levels according to the value. If the wakefulness level E is less than E1, the driver is considered to be in an awake state and is classified into warning level 1. If the wakefulness level E is equal to or greater than E1 and less than E2, the driver is considered to be in a state where early signs of drowsiness are visible and is classified into warning level 2. If the wakefulness level E is equal to or greater than E2 and less than E3, the driver is considered to be in a transitional state to drowsiness and is classified into warning level 3. If the wakefulness level E is E3 or greater, the driver is considered to be in a drowsy state and is classified into warning level 4.
[0035] When the warning level is 1, the range setting unit 113a sets a detection range 201 that is relatively short in depth with respect to the traveling direction of the host vehicle. In the lower diagram of Fig. 7, the road surface is shown by a mesh line, and the detection unit 113 detects a warning event by setting the region of the mesh line as the detection range 201. For example, as shown in the figure, when a pedestrian 902 moves from the sidewalk side to the roadway side, the event is detected as a warning event.
[0036] When the detection unit 113 detects a warning event, it passes the event event together with information about the event area to the warning unit 114. The warning unit 114 generates a warning frame 211 according to the event area information, and displays the warning frame 211 superimposed on the forward image displayed on the display 140. The warning frame 211 is highlighted on the display 140. For example, it may be displayed blinking, the frame color may be changed, or it may be displayed together with text information such as "Caution!". The warning unit 114 also issues a warning sound or a voice such as "Please be careful" via the speaker 150.
[0037] Fig. 8 is a diagram showing a procedure for detecting a warning event from a forward image in the case of warning level 2. As in Fig. 7, it is assumed that the vehicle is traveling in an urban area and the out-camera unit 130 captures a forward image as shown in the above figure. Here, a pedestrian 903 is entering the roadway near a traffic light.
[0038] In the case of warning level 2, range setting unit 113a sets detection range 201 that is deeper than in the case of warning level 1. In the lower diagram of FIG. 8, the road surface is indicated by a mesh line, and detection unit 113 detects a warning event by setting the mesh line area as detection range 201. For example, as shown in the figure, when pedestrian 903 moves from the sidewalk to the roadway, the event is detected as a warning event. When detection unit 113 detects a warning event, it passes it over to warning unit 114 together with information on the event area as described above. Warning unit 114 warns of the warning event via display 140 and speaker 150.
[0039] Fig. 9 is a diagram showing a procedure for detecting a warning event from a forward image in the case of warning level 3. As in Fig. 8, it is assumed that the vehicle is traveling in an urban area and the outer camera unit 130 captures a forward image as shown in the above figure. Here, a pedestrian 904 is walking on the road ahead of the traffic light.
[0040] In the case of warning level 3, the range setting unit 113a sets a detection range 201 with a greater depth than in the case of warning level 2. In the lower diagram of FIG. 9, the road surface is indicated by a mesh line, and the detection unit 113 detects a warning event by setting the mesh line area as the detection range 201. For example, when a pedestrian 904 is walking on the roadway, the event is detected as a warning event. When the detection unit 113 detects a warning event, it passes the event along with information on the event area to the warning unit 114 as described above. The warning unit 114 warns of the warning event via the display 140 or the speaker 150. At this time, if the vehicle is traveling and the traffic light ahead is red, the event is also detected as a warning event separate from the pedestrian warning event, and the warning unit 114 also issues a warning to the traffic light. By setting the detection range 201 according to the warning level in this way, a warning of a warning event can be issued at an earlier stage when the vehicle is close to falling asleep, and a warning can be issued only to warning events occurring nearby when the vehicle is awake. Therefore, the warning adjusted in this manner is not bothersome to the driver and can appropriately support safe driving.
[0041] When the detection unit 113 classifies the driver's wakefulness as warning level 4, the result is passed to the warning unit 114, and the warning unit 114 immediately issues a warning to call attention regardless of the detection result of the warning event. The warning unit 114, for example, flashes the message "Watch out, don't fall asleep!" on the display 140 with high brightness or makes the speaker 150 speak at a volume louder than usual.
[0042] The range setting unit 113a may expand or contract the depth of the detection range 201 according to the speed of the vehicle. Specifically, for the same warning level, the greater the speed, the longer the depth is set. The speed of the vehicle may be calculated using GPS information received by the smartphone 100, or may be estimated from a movement vector between frame images. In addition, when a curve is detected in the traveling direction by a white line on the road surface or the like, the detection range 201 may be set to match the curve.
[0043] The detection unit 113 may not only detect pedestrians running out as described above, but may also detect red lights and stop signs, the approach of oncoming vehicles and preceding vehicles, objects fallen on the road surface, and other traffic signs. Anything that detects an event related to the safety of vehicle travel may be a detection target. The detection unit 113 may employ various methods when detecting such a warning event. For example, as described above, a forward image may be input to a trained model to output an area where a warning event occurs, or the presence or absence of a specific event may be recognized by matching the forward image with a pattern image prepared in advance.
[0044] 10 is a flow diagram showing a process flow of driving assistance executed by the smartphone 100. The flow starts, for example, when the driver issues an instruction to start the flow via the input device 160. Alternatively, the flow may start when the smartphone 100 detects that the vehicle has started to travel.
[0045] In step S101, the acquisition unit 111 acquires face image data from the in-camera unit 120 and transfers it to the determination unit 112. In step S112, the determination unit 112 detects the driver's facial feature points from the face image of the face image data as described with reference to FIG. 3. In step S103, the eye opening calculation unit 112a calculates the eye opening rate from the driver's facial feature points to calculate the eye opening rate evaluation value Eα as described with reference to FIG. 4. In step S104, the yawn calculation unit 112b calculates the mouth opening rate from the driver's facial feature points to calculate the mouth opening rate evaluation value Eβ as described with reference to FIG. In step S105, the shaking calculation unit 112c calculates the head center of gravity from the driver's facial feature points to calculate the shaking evaluation value Eγ as described with reference to FIG. It should be noted that the order of steps S103 to S105 may be changed, or steps may be processed in parallel.
[0046] In step S106, the determination unit 112 calculates the level of wakefulness E from the eye open rate evaluation value Eα, the eye open rate evaluation value Eβ, and the sway evaluation value Eγ, and passes the level of wakefulness E to the detection unit 113. In step S107, the detection unit 113 checks whether the level of wakefulness E is equal to or greater than a threshold value E3. The threshold value E3 is set in advance as a level of wakefulness corresponding to a reference state above which it can be determined that the person is in a drowsy state. If it is confirmed that the level of wakefulness E is equal to or greater than the threshold value E3, the process proceeds to step S108, and the warning unit 114 issues an emergency warning corresponding to the above-mentioned warning level 4. Thereafter, the process proceeds to step S109. If it is confirmed that the level of wakefulness E is not equal to or greater than the threshold value E3, the process proceeds to step S109 without passing through step S108.
[0047] In step S109, the acquisition unit 111 acquires forward image data from the out-camera unit 130 and transfers it to the detection unit 113. In step S110, the range setting unit 113a sets a detection range for detecting a warning event in the forward image according to the warning level classified by the awakening level E. In step S111, the detection unit 113 detects whether a warning event exists in the set detection range. Then, in step S112, the detection unit 113 checks whether at least one warning event has been detected, and if it is confirmed that one has been detected, the process proceeds to step S113, and if none has been detected, the process proceeds to step S114.
[0048] In step S113, warning unit 114 executes the above-mentioned warning process by referring to the warning event and the event area information received from detection unit 113. Then, the process proceeds to step S114.
[0049] In step S114, the processing unit 110 checks whether an end command has been issued. An end command is issued, for example, when the driver issues an end command using the input device 160, or when the smartphone 100 detects that the vehicle has stopped. If it is confirmed that an end command has been issued, the processing unit 110 ends the series of processes. If it is confirmed that an end command has not been issued, the processing unit 110 returns to step S101 and continues to repeat the series of processes.
[0050] Although the present embodiment has been described above, the device that executes the driving assistance program to function as the driving assistance system is not limited to the smartphone 100, and may be, for example, a tablet terminal, a car navigation system, a drive recorder, etc. When using a car navigation system or a drive recorder, face image data may be obtained from a camera unit that is installed in the vehicle and capable of capturing an image of the driver's face, and forward image data may be obtained from a camera unit that is also installed in the vehicle and capable of capturing an image of the forward view.
[0051] In addition, in the present embodiment, the smartphone 100 includes the detection unit 113, but the processing of the detection unit 113 may be executed by a server connected to a network. In this case, the processing unit 110 transmits the acquired forward image data and the result of processing by the determination unit 112 to the server via the communication unit 180. Then, the server receives information on the warning event detected by the server and its event area via the communication unit 180, and the warning unit 114 issues a warning in response to the information.
[0052] In addition, the driving assistance system according to the present embodiment may warn of a detected warning event according to a threshold value set based on the wakefulness state for an object appearing in the forward image of the forward image data. Specifically, when a red light or a stop sign is detected, it is determined whether or not to warn as a warning event based on a threshold value (a threshold value set based on the wakefulness state) corresponding to the warning levels 1 to 3. For example, since the wakefulness level is lower at warning level 2 than at warning level 1, the threshold value is lower than at warning level 1. Therefore, at warning level 2, a red light or the like having a smaller area than at warning level 1 is treated as a warning event. As a result, at warning level 2, a red light or the like at the back of the screen that was not treated as a warning event at warning level 1 is also warned as a warning event. [Explanation of symbols]
[0053] 100...smartphone, 110...processing unit, 111...acquisition unit, 112...determination unit, 112a...eye opening calculation unit, 112b...yawning calculation unit, 112c...sway calculation unit, 113...detection unit, 113a...range setting unit, 114...warning unit, 120...in-camera unit, 130...out-camera unit, 140...display, 150...speaker, 160...input device, 170...memory, 180...communication unit, 201...detection range, 211...warning frame, 901, 902, 904...pedestrian, 903...traffic light
Claims
1. A driving assistance method for a vehicle driver, comprising: A first acquisition step in which an acquisition unit acquires face image data obtained by capturing an image of a face of a driver who drives a vehicle; a determination step in which a determination unit determines a wakefulness level of the driver from the face image data; A second acquisition step in which the acquisition unit acquires forward image data obtained by capturing an image of a forward area of the vehicle; a warning step in which a warning unit detects whether or not a warning event that may hinder safe driving of the vehicle has occurred in a forward image of the forward image data by increasing a depth along the traveling direction of the vehicle as the awakening level decreases, and when the warning event is detected, highlighting the warning event in the captured image to warn the driver; A driving assistance method comprising:
2. The driving assistance method according to claim 1 , wherein the warning step includes issuing a warning of a warning event detected for an object captured in a forward image of the forward image data in accordance with the degree of awakening.
3. 3. The driving assistance method according to claim 1, wherein the warning step issues a warning to call attention when the wakefulness level exceeds a reference state, regardless of a detection result within the detection range.
4. 4. The driving assistance method according to claim 1, wherein the determination step determines the degree of alertness by observing the degree of eye opening, yawning, and head movement of the driver based on feature points detected from a facial image of the facial image data.
5. A driving assistance program for causing a computer to implement the driving assistance method according to any one of claims 1 to 4.
6. 6. The driving assistance program according to claim 5, which is installed in a computer of a smartphone having a first camera unit and a second camera unit, acquiring the facial image data from the first camera unit; acquiring the forward image data from the second camera unit; Driver assistance programs.
7. An acquisition unit that acquires face image data that captures an image of a face of a driver who drives a vehicle and front image data that captures an image of a front of the vehicle; a determination unit for determining a degree of wakefulness of the driver from the facial image data; a warning unit that, in a detection range for detecting whether or not a warning event that may hinder safe driving of the vehicle has occurred in a forward image of the forward image data, increases a depth along the traveling direction of the vehicle as the awakening level decreases, and when the warning event is detected, highlights the warning event in a captured image to warn the driver; A driving assistance system equipped with
Citation Information
Patent Citations
Alarm system
JP2003317197A
Visual field support device, visual field support method, and visual field support program
JP2015144407A
Vehicle notification system, notification controller, and notification device
JP2016024778A
Two-stage drowsy driving prevention device through recognition of movement, face, eye and mouth shape
JP2016539446A
System and method responding to driver's behavior
JP2017200822A
