Driver monitoring system, vehicle, and driver monitoring method for determining whether a driver's driving condition is appropriate

Abstract

To provide a technique capable of accurately determining appropriateness of a driving state of a driver in accordance with the driver in a case where vehicle driving tendency differs from driver to driver.SOLUTION: A driver monitoring system 10 that determines appropriateness of a driving state of a driver who is driving a vehicle, comprises: a database 12 that stores driving state determination values for determining appropriateness in association with respective driver information of a plurality of driver; and a processor 14 that determines appropriateness during driving of a first driver identified on the basis of driver information by using a first driving state determination value which is stored in the database 12 associated with first driver information for the first driver.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a driver monitoring system, a vehicle, and a driver monitoring method for determining whether a driver's driving state is appropriate. [Background technology]

[0002] 2. Description of the Related Art A driver monitor system that determines whether a driver's driving state is appropriate is known (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-15549 Summary of the Invention [Problem to be solved by the invention]

[0004] In some cases, the driving habits of each driver may differ. In such cases, there is a need for a technology that can accurately determine whether the driver's driving condition is appropriate for each driver. [Means for solving the problem]

[0005] A first aspect of the present disclosure is a driver monitor system that determines whether the driving state of a driver driving a vehicle is appropriate, and includes a database that stores driving state judgment values ​​for determining suitability in association with individual driver information of multiple drivers, and a processor that determines whether the first driver is appropriate while driving using a first driving state judgment value stored in the database in association with first driver information of a first driver identified based on the driver information.

[0006] A second aspect of the present disclosure is a driver monitor system according to the first aspect, wherein the processor acquires driving state parameters representing the driving state of a first driver, updates a first driving state judgment value based on the acquired driving state parameters, and stores the updated first driving state judgment value in a database in association with the first driver information.

[0007] A third aspect of the present disclosure is a driver monitor system according to the second aspect, further comprising an imaging device that captures an image of the face of the driver while driving, and the processor acquires driving state parameters of the first driver based on the facial image of the first driver captured by the imaging device.

[0008] A fourth aspect of the present disclosure is a driver monitor system described in the second or third aspect, in which the processor updates the first driving state judgment value based on a plurality of driving state parameters repeatedly acquired during driving by the first driver.

[0009] A fifth aspect of the present disclosure is a driver monitor system described in the fourth aspect, in which the processor calculates a new driving state judgment value from a plurality of driving state parameters by a predetermined calculation, and updates the first driving state judgment value with the new driving state judgment value.

[0010] A sixth aspect of the present disclosure is a driver monitor system described in any one of the second to fifth aspects, in which the processor estimates whether the first driver is looking ahead of the vehicle based on vehicle information representing the behavior of the vehicle while driving, and updates the first driving state judgment value based on driving state parameters acquired when it is estimated that the driver is looking ahead.

[0011] A seventh aspect of the present disclosure is a driver monitor system according to the sixth aspect, wherein the vehicle information includes at least one of a steering angle of the vehicle, a speed of the vehicle, an environmental image captured around the vehicle, and position information of the vehicle.

[0012] An eighth aspect of the present disclosure is a driver monitor system according to any one of the second to seventh aspects, wherein the driving state parameter has the driver's facial direction or the degree of eye opening of the driver, and the driving state judgment value has a reference facial direction determined based on the facial direction of the driver looking ahead of the vehicle and a threshold value for the angle of the facial direction relative to the reference facial direction, or a reference opening degree determined based on the opening degree when the driver has their eyes open and a threshold value for the opening degree determined based on the reference opening degree.

[0013] A ninth aspect of the present disclosure is a driver monitor system described in any one of the first to eighth aspects, wherein the driver information includes a facial image of the driver, the driver monitor system further includes an imaging device that captures an image of the driver's face before driving, and the processor identifies the driver by comparing the facial image of the driver captured by the imaging device with a facial image previously stored in a database as driver information.

[0014] A tenth aspect of the present disclosure is a vehicle including the driver monitor system according to any one of the first to ninth aspects.

[0015] An eleventh aspect of the present disclosure is a driver monitoring method for determining whether the driving state of a driver driving a vehicle is appropriate, which includes preparing a database that stores driving state judgment values ​​for determining suitability in association with individual driver information of multiple drivers, and a processor that uses a first driving state judgment value stored in the database in association with first driver information of a first driver identified based on the driver information to determine whether the first driver is appropriate while driving. [Effects of the Invention]

[0016] According to the present disclosure, the processor can determine whether the driver's driving state is appropriate using a driving state judgment value specific to the driver, thereby making it possible to perform the determination of appropriateness with high accuracy depending on the driver. [Brief explanation of the drawings]

[0017] [Figure 1] FIG. 1 is a block diagram of a driver monitor system according to an embodiment. [Figure 2] 1 is a block diagram of a vehicle according to an embodiment; [Figure 3] 3 is a flowchart showing an example of an operation flow of the vehicle shown in FIG. 2. [Figure 4] 3 is an example of a face image of a first driver captured by the imaging device shown in FIG. 2. [Figure 5] 3 is an example of a face image of a second driver captured by the imaging device shown in FIG. 2. [Figure 6] 10 is a flowchart showing another example of the operation flow of the vehicle shown in FIG. DETAILED DESCRIPTION OF THE INVENTION

[0018] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In various embodiments described below, similar elements will be given the same reference numerals, and duplicated explanations will be omitted. First, a driver monitor system 10 according to one embodiment will be described with reference to FIG. 1. The driver monitor system 10 is a system that determines whether the driving state DC of a driver D who drives a vehicle is appropriate. Specifically, the driver monitor system 10 includes a database 12 and a processor 14. The database 12 stores data of a plurality of drivers D n Individual driver information DI (n=1, 2, 3, . . .) n In relation to this, the driving state judgment value DV n Store.

[0019] Driver Information DI n is Driver D n information specific to the driver D n For example, driver information DI n is Driver D n Identification code DI1 to identify n (number, character string, etc.), Driver D n Face image DI2 n , and Driver D n Biological information DI3 n(weight, fingerprint information, iris information, voice information, etc.) n is Driver D n Alternatively, the image data may be image data (for example, JPEG image data or RAW image data) of the face of the driver D extracted from the image data. n The driver information DI may be feature quantity information obtained by coding the facial feature points. n is a driver other than those exemplified here. n It may include any information that can identify the

[0020] On the other hand, the driving state judgment value DV n is the driver D who drives the vehicle. n Specifically, the driving state judgment value DV n is Driver D n In order to determine whether the driving condition DC is appropriate, the driver D n The operating state parameter PR is compared with the operating state DC of the vehicle.

[0021] This driving state parameter PR is, for example, the driver D n The face direction α of the driver D n The eye opening degree β of the driver D n The blinking frequency γ of the driver D n The mouth opening and closing frequency ε of the driver D n These driving state parameters PR include at least one of the following: n This is a parameter that quantitatively represents the operating state DC of the engine.

[0022] Operating state judgment value DV n is set for such an operating state parameter PR. As an example, the operating state determination value DV n is a driver D who looks ahead of the vehicle while driving. n a reference face direction α0 determined based on the face direction α of the driver D while driving, and n The threshold value φ of the angle φ of the face direction α thAs another example, the driving state determination value DV n is Driver D n a reference opening degree β0 determined based on the opening degree β when the eye is normally open, and a threshold value β of the opening degree β determined based on the reference opening degree β0. th It has.

[0023] As another example, the driving state determination value DV n is the normal driver D n A reference blink frequency γ0 determined based on the reference blink frequency γ, and a threshold value γ of the blink frequency γ0 determined based on the reference blink frequency γ0. th Further, as an example, the driving state determination value DV n is the normal driver D n a reference opening / closing frequency ε0 determined based on the mouth opening / closing frequency ε of the mouth, and a threshold value ε of the opening / closing frequency ε determined based on the reference opening / closing frequency ε0 th It has.

[0024] The database 12 includes a plurality of drivers D n Driver Information DI n Each driver D n The driving state judgment value DV n An example of the data structure of the database 12 is shown in Table 1 below.

[0025] [Table 1]

[0026] In the example shown in Table 1, the database 12 contains the driver D n Driver Information DI n In relation to the driving state judgment value DV n As a reference face direction α with respect to the face direction α, 0_n and threshold φ th_n , reference opening degree β for eye opening degree β 0_n and threshold β th_nFor example, the driver information DI1 of the first driver D1 stores the reference face direction α with respect to the face direction α as the driving state determination value DV1. 0_1 and φ th_1 , reference opening β for opening β 0_1 and threshold β th_1 etc. are stored in association with each other. This database 12 is prepared in advance by an operator and stored in a memory (ROM, RAM, etc.) provided in the vehicle or the management server.

[0027] In Table 1, the driving state determination value DV n As a reference face direction α with respect to the face direction α, 0_n and φ th_n and the reference opening degree β for the eye opening degree β 0_n and threshold β th_n However, in reality, the reference blink frequency γ 0_n and threshold γ th_n , the reference opening and closing frequency ε for the mouth opening and closing frequency ε 0_n and threshold ε th_n Other driving state judgment values ​​DV n Driver information DI n It should be understood that the data is stored in association with the

[0028] The processor 14 has a CPU or a GPU, and is capable of accessing the database 12. Specifically, the processor 14 accesses the driver information DI stored in the database 12. n Driver D, who drives a vehicle based on n Identify the identified driver D n Driver Information DI n The operating state determination value DI stored in the database 12 in association with n Using this, the driver D n During operation, the suitability of the operating state DC is judged.

[0029] For example, the processor 14 may use the driver information DI stored in the database 12. nIn this case, the processor 14 determines the driver of the vehicle as the first driver D1 based on the driver information DI1 of the determined driver D1. 0_1 and φ th_1 , standard opening degree β 0_1 and threshold β th_1 , the reference blink frequency γ 0_1 and threshold γ th_1 , reference switching frequency ε 0_1 and threshold ε th_1 etc.) are searched and obtained from the database 12.

[0030] The processor 14 then compares the driving state parameters PR (face direction α, opening degree β, blinking frequency γ, opening / closing frequency ε, etc.) acquired while the driver D1 is driving with the driving state determination value DV1 stored in the database 12 in association with the driver information DI1 of the identified driver D1, to determine whether the driving state DC of the driver D1 is appropriate. n The method for specifying the operating state DC and the method for determining whether the operating state DC is appropriate will be described in detail later.

[0031] As described above, in this embodiment, the driver monitor system 10 monitors the vehicle status of a plurality of drivers D. n Individual driver information DI n In relation to this, the driving state judgment value DV n and a database 12 that stores driver information DI. n The system further includes a processor 14 that determines whether the driving state DC of the first driver D1 is appropriate while the first driver D1 is driving, using a first driving state determination value DV1 stored in the database 12 in association with the first driver information DI1 of the first driver D1 identified based on the above. n The driving state judgment value DV n Using the driver D n Since it is possible to judge whether the driving state DC is appropriate or not, the judgment of the appropriateness is made by the driver D. n Depending on the calculation, high accuracy can be achieved.

[0032] Next, a vehicle 20 according to one embodiment will be described with reference to Fig. 2. The vehicle 20 is, for example, a four-wheeled automobile, and includes a vehicle body 22, an electronic control unit (ECU) 24, an imaging device 26, sensors 28, a human-machine interface (HMI) 30, and a communication device 32.

[0033] In addition to the ECU 24, imaging device 26, sensors 28, HMI 30, and communication device 32, the vehicle body 22 is equipped with various components related to the driving, braking, steering, infotainment, etc. of the vehicle 20 (engine, transmission, wheels, brake mechanism, steering, meter panel, rearview mirror, display, speaker, input device, etc.).

[0034] The ECU 24 controls the operation of the vehicle 20. Specifically, the ECU 24 is a computer having a processor 14, a memory 34, and an I / O interface 36. The memory 34 has RAM, ROM, or the like, and temporarily or permanently stores various data used in the arithmetic processing executed by the processor 14 and various data generated during the arithmetic processing. In this embodiment, the memory 34 stores the database 12.

[0035] The I / O interface 36 has, for example, a controller area network (CAN) port, an Ethernet (registered trademark) port, a USB port, an optical fiber connector, or an HDMI (registered trademark) terminal, and communicates data via wire or wirelessly with external devices such as the image capture device 26, the sensor 28, the HMI 30, and the communication device 32. The processor 14 communicates with the memory 34 and the I / O interface 36 (i.e., the image capture device 26, the sensor 28, the HMI 30, and the communication device 32) and performs arithmetic processing to execute various functions, including the function of determining whether the operating state DC is appropriate.

[0036] The imaging device 26 is nSpecifically, the imaging device 26 has, for example, a light emitting unit (such as an LED), an imaging sensor (such as a CCD or CMOS), and an optical lens (such as a collimator lens or a focus lens), and is incorporated into the interior of the vehicle body 22 (for example, an instrument panel or a rearview mirror). The imaging device 26 emits light (such as infrared light) from the light emitting unit, and captures the face of the driver D. n By focusing the light from the n The face is photographed.

[0037] The sensor 28 detects various data of the vehicle 20 while it is being driven. For example, the sensor 28 includes a steering angle sensor that detects the steering angle λ of the steering of the vehicle 20, a speed sensor that detects the speed V of the vehicle 20, and an external camera that detects environmental images IMs that capture the surrounding environment of the vehicle 20 (road signs such as lanes, other vehicles, pedestrians, obstacles, etc.).

[0038] The communication device 32 has at least one of a receiver and a transmitter, and communicates with devices external to the vehicle 20. For example, the communication device 32 may include a GPS receiver that communicates with a GPS satellite and receives a GPS signal from the GPS satellite, an inter-vehicle communication device that communicates with other vehicles and transmits and receives various data such as vehicle position information to and from the other vehicles, and a driver D n The mobile device (for example, a smart key or a smartphone) owned by the driver D communicates with the driver D. n The mobile phone has a mobile signal receiver that receives the identification code of the mobile phone.

[0039] The processor 14 acquires the steering angle λ, the speed V, and the environmental image IMs detected by the sensor 28 while the vehicle 20 is being driven as vehicle information CI representing the behavior of the vehicle 20 while being driven. The processor 14 also acquires position information PI of the vehicle 20 as the vehicle information CI based on the GPS signal received by the communication device 32 and map data pre-stored in the memory 34.

[0040] The processor 14 continuously (e.g., periodically) acquires the vehicle information CI (steering angle λ, speed V, environmental image IMs, and position information PI) while the vehicle 20 is being driven, and stores the collected vehicle information CI in the memory 34. Note that the vehicle information CI is not limited to the steering angle λ, speed V, environmental image IMs, and position information PI, and may include any other information that indicates the behavior of the vehicle 20.

[0041] HMI30 is the driver D n For example, the HMI 30 exchanges information with the driver D. n input devices (switches, push buttons, rotary dials, touch panels, etc.) that accept information input from the driver, a display that displays various information as images, a speaker that outputs various information as sound, and n It has a microphone that converts sound into an electrical signal.

[0042] Next, the operation of the vehicle 20 will be described with reference to Fig. 3. The flow shown in Fig. 3 is n The process begins when the driver D accesses (e.g., unlocks) the vehicle 20. In step S1, the processor 14 n As an example, the communication device 32 identifies the driver D n communicates with a portable device owned by the driver D, and n The processor 14 receives the identification code of the driver D through the communication device 32. n and obtains the identification code of the driver, and compares the identification code with the driver information DI stored in advance in the database 12. n Identification code DI1 included in n By comparing this, driver D n Identify.

[0043] As another example, the imaging device 26 may capture the image of a driver D getting into the vehicle 20. n The processor 14 captures the face of the driver D captured by the imaging device 26. n Face image IMf n and driver information DI stored in advance in the database 12. n Facial image DI2 included in nBy comparing this, driver D n Identify.

[0044] The processor 14 processes the facial image IMf n and the facial image DI2 stored in the database 12. n Alternatively, the image data may be compared with the facial image IMf n Driver D extracted from the image data n and obtains feature information obtained by coding the facial feature points of the face, and compares the feature information with the facial image DI2 stored in the database 12. n The feature amount information may be compared with the feature amount information of the first embodiment.

[0045] As yet another example, the communication device 32 may n communicates with a portable device owned by the driver D and inputs the information of the driver D into the portable device. n The processor 14 receives biometric information (e.g., fingerprint information) of the driver D through the communication device 32. n and comparing the biometric information with driver information DI stored in advance in the database 12. n Biometric information contained in DI3 n By comparing this, driver D n Identify.

[0046] As yet another example, the sensor 28 may be built into the driver's seat and detect the driver D seated therein. n In this case, the processor 14 may include a weight sensor for detecting the weight W of the driver D. n Biological information DI3 n The weight W detected by the weight sensor is acquired, and the weight W and the biological information DI3 are stored in the database 12. n The pre-stored weight W n By comparing this, driver D n may be specified.

[0047] In step S2, the processor 14 determines whether the driver D identified in step S1 n Operating state judgment value DV nSpecifically, the processor 14 acquires the driver D identified in step S1. n Driver Information DI n The driving state determination value DV n are retrieved from the database 12.

[0048] For example, if the first driver D1 in Table 1 is identified in step S1, the processor 14 determines in step S2 the driving state determination value DV1 (specifically, the reference face direction α 0_1 and threshold φ th_1 , standard opening degree β 0_1 and threshold β th_1 etc.) are searched and obtained from the database 12.

[0049] In step S3, the processor 14 determines whether or not driving of the vehicle 20 has started. For example, in step S3, the processor 14 may determine YES when the engine speed of the vehicle 20 or the speed of the vehicle 20 exceeds a predetermined threshold. If the processor 14 determines YES, the processor 14 proceeds to step S4, and if the processor 14 determines NO, the processor 14 proceeds to step S10.

[0050] In step S4, the processor 14 starts to acquire the driving state parameter PR and the vehicle information CI. Specifically, the processor 14 operates the imaging device 26 to capture the driver D while driving. n The face of the driver D is continuously (for example, periodically) captured by the image capturing device 26. n Face image IMf n (Specifically, based on the image data or feature amount information obtained by coding feature points extracted from the image data), n As the driving state parameter PR of the driver D n The facial direction α, eye opening degree β, blinking frequency γ, mouth opening / closing frequency ε, and gaze direction δ are acquired.

[0051] The facial direction α is determined by the facial image IMf captured by the imaging device 26.n Fig. 4 shows the facial image IMf n The processor 14 generates a facial image IMf n By analyzing the facial image IMf n Driver D in the photo n Specifically, the processor 14 calculates the facial direction α of the facial image IMf. n The outer edges of the eyes and the tip of the nose of the face in the photograph are extracted as feature points, and the facial direction α can be determined based on the positional relationship of the extracted feature points of the outer edges of the eyes and the tip of the nose.

[0052] On the other hand, the eye opening degree β is calculated as, for example, the distance between the eyelids or the area of ​​the eye. The blink frequency γ is calculated as the number of blinks in a predetermined time (for example, 10 seconds). The mouth opening / closing frequency ε is calculated as the number of times the mouth opens and closes in a predetermined time (for example, 10 seconds). The gaze direction δ is calculated as the number of times the face image IMf opens and closes, as shown in FIG. n Driver D in the photo n can be obtained from the pupil position.

[0053] The processor 14 also starts an operation of continuously (for example, periodically) acquiring, as vehicle information CI, the steering angle λ, speed V, and environmental image IMs detected by the sensor 28 while the vehicle 20 is being driven, and position information PI calculated from the GPS signal. In this way, the processor 14 continuously acquires the driving state parameters PR (i.e., face direction α, opening degree β, blinking frequency γ, opening / closing frequency ε, and line of sight δ) and the vehicle information CI (steering angle λ, speed V, environmental image IMs, and position information PI) while the vehicle 20 is being driven.

[0054] In step S5, the processor 14 calculates the most recently acquired driver D n The driving state parameter PR and the driver D set at this point n Operating state judgment value DV n Using the above, the driver D n 3. After starting the flow of FIG. 3, when executing step S5 for the first time, the processor 14 determines whether the driving state DC of the driver D obtained in step S2 is appropriate.n Operating state judgment value DV n is used to determine whether the operating state DC is appropriate.

[0055] As an example, the processor 14 may calculate a face direction α as the driving state parameter PR and a driving state determination value DV n The reference face direction α 0_n and threshold φ th_n and the reference face direction α 0_n Driver D n The angle φ of the face direction α is calculated, and the angle φ is set to a threshold value φ th_n Below (φ≦φ th_n ) is satisfied. th_n If so, driver D n is driving the vehicle 20 while properly viewing the area ahead (i.e., YES).

[0056] The processor 14 also calculates the eye opening degree β as the driving state parameter PR and the driving state determination value DV n Reference opening β as 0_n and threshold β th_n and the opening β is set to the threshold value β th_n or more (β ≧ β th_n ) is determined. th_n is the reference opening β 0_n Based on this, for example, β th_n =ζβ 0_n (where ζ is a coefficient of 0<ζ<1). th_n If so, driver D n The system determines that the driver is not dozing off and is driving with their eyes properly open (i.e., YES).

[0057] The processor 14 also calculates the blinking frequency γ and the opening / closing frequency ε as the driving state parameter PR, and the driving state determination value DV n Reference blink frequency γ as 0_n and threshold γ th_n , and the reference switching frequency ε 0_n and threshold ε th_n and the blink frequency γ and the opening / closing frequency ε are calculated based on the threshold γth_n and ε th_n Below (γ≦γ th_n , and ε≦ε th_n ) is determined.

[0058] Threshold γ th_n and ε th_n is the reference blink frequency γ 0_n and the reference switching frequency ε 0_n Based on, for example, γ th_n =ζγ 0_n , and ε th_n =ζε 0_n (where ζ is a coefficient in the range of 0<ζ<1). th_n , and ε≦ε th_n If so, driver D n The processor 14 determines that the driver is in an appropriate state of health and is driving (i.e., YES). If the determination is YES, the processor 14 proceeds to step S6, whereas if the determination is NO, the processor 14 proceeds to step S9.

[0059] In step S6, the processor 14 determines the driver D based on the vehicle information CI (steering angle λ, speed V, environmental image IMs, and position information PI). n Specifically, the processor 14 determines whether the vehicle 20 is traveling on a straight road based on at least one of the steering angle λ, the lane shown in the environmental image IMs, and the position information PI of the vehicle 20. When the vehicle 20 is traveling on a straight road, the driver D n can be assumed to be looking ahead of the vehicle 20.

[0060] The processor 14 also determines whether the speed V of the vehicle 20 exceeds a predetermined threshold V th or more (V≧V th When the vehicle 20 is traveling at high speed, the driver D n It can be assumed that the driver D is looking ahead of the vehicle 20. The processor 14 determines whether the driver D is looking ahead when the vehicle 20 is traveling on a straight road or at a high speed. nIf the processor 14 determines that the driver D is looking ahead of the vehicle 20 (i.e., YES), the process proceeds to step S7. On the other hand, if the processor 14 determines that the driver D is looking ahead of the vehicle 20 (i.e., YES), the process proceeds to step S8. n Based on the gaze direction δ of the driver D n It may be estimated whether the driver is looking ahead of the vehicle 20.

[0061] In step S7, the processor 14 determines whether the driver D n Operating state judgment value DV n As an example, the processor 14 updates the driving state determination value DV n The reference face direction α stored in the database 12 0_n is replaced with the facial direction α acquired as the driving state parameter PR when the determination in step S6 is YES, the reference facial direction α 0_n Furthermore, the processor 14 updates the updated reference face direction α 0_n Based on the threshold φ th_n By newly setting the threshold value φ th_n Update.

[0062] As another example, the processor 14 may calculate the driving state determination value DV n The reference opening β stored in the database 12 0_n is replaced with the opening degree β acquired as the operating state parameter PR when the determination in step S6 is YES, the reference opening degree β 0_n In addition, the processor 14 updates the updated reference opening β 0_n Based on this, for example, β th_n =ζβ 0_n From the formula, the threshold value β th_n By newly setting the threshold value β th_n Update.

[0063] Similarly, the processor 14 calculates the driving state determination value DV n as the reference blink frequency γ 0_n and threshold γth_n , and the reference switching frequency ε 0_n and threshold ε th_n may be updated by replacing with the blinking frequency γ and the opening / closing frequency ε acquired as the driving state parameters PR when the determination in step S6 is YES.

[0064] The processor 14 updates the driving state determination value DV n (Reference face direction α 0_n and threshold φ th_n , standard opening degree β 0_n and threshold β th_n , the reference blink frequency γ 0_n and threshold γ th_n , reference switching frequency ε 0_n and threshold ε th_n etc.) as driver information DI n The driving state determination value DV n is updated based on the operating state parameter PR.

[0065] In step S8, processor 14 determines whether or not an engine start button (not shown) provided in the interior of vehicle 20 has been turned off. If processor 14 determines YES, it ends the flow shown in FIG. 3, but if it determines NO, it returns to step S5.

[0066] On the other hand, if the determination in step S5 is NO, the processor 14 generates an alarm signal AL1 in step S9. For example, if the angle φ is greater than or equal to the threshold value φ th_n is greater than (φ>φ th_n ) and the result is NO, in step S9, an image or audio warning signal AL1 saying "There is a possibility of inattentive driving. Please look ahead of the vehicle" may be generated.

[0067] Alternatively, in step S5, the opening β is set to a threshold value β th_n is smaller than (β<β th_n) and determines the result as NO, the processor 14 may generate an image or sound warning signal AL1 saying "You may be falling asleep while driving. Please take a break" in this step S9. The processor 14 may notify the driver D through the display or speaker of the HMI 30 of the warning signal AL1 generated in the form of an image or sound. n Output to.

[0068] On the other hand, if the determination in step S3 is NO, then in step S10, processor 14 determines whether the engine start button has been turned OFF, similar to step S8 described above. If the determination is YES, processor 14 ends the flow shown in Fig. 3, but if the determination is NO, processor 14 returns to step S3.

[0069] As described above, in this embodiment, the processor 14 determines whether the driving state DC is appropriate by using the database 12 and the image captured by the imaging device 26. Therefore, the processor 14, the database 12, and the imaging device 26 constitute a driver monitor system 10 (FIG. 2) that determines whether the driving state DC is appropriate.

[0070] In this embodiment, the processor 14 calculates the driving state determination value DV based on the acquired driving state parameter PR. n The updated driving state judgment value DV n Driver information DI n The driving state determination value DV n Driver D n Taking into account the driving tendencies of the driver D n Can be updated every

[0071] More specifically, for example, when a first driver D1 is driving appropriately while looking ahead of the vehicle 20, the first driver D1 tends to have a driving tendency in which the facial direction α is directed toward the front of the vehicle 20, as shown in FIG. 4, whereas when a second driver D2 is driving appropriately while looking ahead of the vehicle 20, the second driver D2 tends to have a driving tendency in which the facial direction α is directed slightly to the side of the vehicle 20, as shown in FIG. 5.

[0072] In this way, the driving tendency of each driver D n In the case where the driving state determination value DV n Driver D n Driver D n The driving condition judgment value DV n According to this configuration, the driver D determines whether the driving state DC is appropriate in step S5. n Depending on the situation, it will be possible to perform the calculation with higher accuracy.

[0073] In this embodiment, the driver monitor system 10 monitors the driver D while driving. n The processor 14 is configured to detect the face of the driver D captured by the imaging device 26. n Face image IMf n Based on this, the driver D n The driving state parameters PR (face direction α, opening degree β, blinking frequency γ, opening / closing frequency ε, line of sight direction δ, etc.) of the driver D are acquired (step S4). n The operating state parameter PR, which highly represents the operating state DC of the vehicle, can be obtained.

[0074] In this embodiment, the processor 14 determines whether the driver D is in a driving state based on the vehicle information CI that indicates the behavior of the vehicle 20 while driving. n The driving state determination value DV is calculated based on the driving state parameter PR acquired when it is estimated that the driver is looking ahead of the vehicle 20 (i.e., when the result of the determination in step S6 is YES). n (Step S7). n The driving state judgment value DV n Therefore, the driving state determination value DV n Driver D n This can be improved by taking into account the driving tendencies of the driver.

[0075] In this embodiment, the vehicle information CI includes at least one of the steering angle λ of the steering of the vehicle 20, the speed V of the vehicle 20, an environmental image IMs obtained by capturing an image of the surroundings of the vehicle 20, and the position information PI of the vehicle 20. According to this configuration, in step S6, the driver D n This makes it possible to estimate with higher accuracy whether the driver is looking ahead of the vehicle 20.

[0076] In this embodiment, the driving state determination value DV n is a driver D looking ahead of the vehicle 20. n A reference face direction α is determined based on the face direction α. 0_n , and the reference face direction α 0_n The threshold value φ of the angle φ of the face direction α relative to th_n Or, the driving state determination value DV n is Driver D n The reference opening angle β is determined based on the opening angle β when the eye is open. 0_n , and the reference opening β 0_n The threshold value β of the opening β is determined based on th_n It has.

[0077] According to this configuration, the driving state determination value DV n As a result, Driver D n Reference face direction α highly correlated with driving tendency 0_n and threshold φ th_n , or reference opening β 0_n and threshold β th_n Driver D n This allows the driver D to determine whether the driving state DC is appropriate or not. n This will enable more accurate execution based on the driver's driving tendencies.

[0078] In this embodiment, the driver information DI n is Driver D n Face image IMf n The driver monitor system 10 monitors the driver D before driving. nThe processor 14 then converts the facial image IMf captured by the imaging device 26 into a n and driver information DI in database 12. n A face image DI2 is stored in advance as n By comparing this, driver D n According to this configuration, the driver D of the vehicle 20 n It is possible to determine with high accuracy whether the person is an authenticated person.

[0079] In step S7, the processor 14 n Based on a plurality of operating state parameters PR repeatedly acquired during operation of the n As an example, each time the determination in step S6 is YES, the processor 14 updates the facial direction directions α1, α2, α3, . . . α as the driving state parameter PR. m is repeatedly obtained.

[0080] Then, in step S7, the processor 14 calculates the face direction α m Using this, a new reference face direction α 0_n For example, the processor 14 may calculate the face direction α m Weighted average α of m ' is calculated, and the reference face direction α 0_n The weighted average α m Similarly, the processor 14 may update the reference opening β 0_n , the reference blink frequency γ 0_n , or the reference switching frequency ε 0_n etc., multiple opening degrees β m , frequency γ m , or frequency ε m Weighted average β of m ', γ m ', or ε m It may be updated by computing '.

[0081] In this manner, in this embodiment, the processor 14 determines a new operating state determination value α from the plurality of operating state parameters PR. m ', β m ', γ m ', or ε m ' is calculated by a predetermined calculation (for example, weighted average), and the new driving state determination value α m ', β m ', γ m ', or ε m ' determines the driving state judgment value DV n According to this configuration, the driver D n The driving condition judgment value DV n can be updated.

[0082] The processor 14 compares the driving state parameter PR acquired when the determination in step S6 is YES with the driver D captured when the determination in step S6 is YES. n Face image IMf n Alternatively, a weighting coefficient η may be assigned in accordance with the driving state parameter PR acquired when the determination in step S6 is YES, or the vehicle information CI acquired when the determination in step S6 is YES.

[0083] As an example, the processor 14 may n By analyzing the image, the driver D n The processor 14 determines whether the driver D is wearing glasses or sunglasses. n If it is determined that the driver D is wearing glasses or sunglasses, a first weighting coefficient η1 is assigned to the driving state parameter PR (for example, the face direction α) acquired when the determination in step S6 is YES. n If it is determined that the driver is not wearing glasses or sunglasses, a second weighting coefficient η2 (>η1) higher than the first weighting coefficient η1 is assigned to the driving state parameter PR.

[0084] As another example, the processor 14 may n By analyzing the image, the driver Dn (in other words, the facial image IMf n Driver D in the photo n If the processor 14 determines that the seating position is inappropriate, it assigns a first weighting coefficient η1 to the driving state parameter PR acquired when the determination in step S6 is YES, while it assigns a second weighting coefficient η1 to the driving state parameter PR acquired when the determination in step S6 is YES. n Face is face image IMf n If it is determined that the driving state parameter PR is located in the central region of the driving state parameter PR, a second weighting coefficient η2 higher than the first weighting coefficient η1 is assigned to the driving state parameter PR.

[0085] As yet another example, the processor 14 determines whether the driving state parameter PR (for example, the face direction α) acquired when the determination in step S6 is YES is equal to or greater than a predetermined reference value PR r_n If the driving state parameter PR is greater than (or smaller than), a first weighting coefficient η1 is assigned to the driving state parameter PR, while a predetermined reference value PR r_n If it is equal to or less than (or equal to or greater than), a second weighting coefficient η2 higher than the first weighting coefficient η1 is assigned to the operating state parameter PR.

[0086] More specifically, when the determination in step S6 is YES and the face direction α is acquired as the driving state parameter PR, the processor 14 determines the reference face direction α 0_n The angle φ of the face direction α with respect to the predetermined reference value φ r_n and the above-mentioned threshold φ th_n Within the following range (φ r_n <φ≦φ th_n ), the first weighting coefficient η1 is assigned to the acquired face direction α, while the angle φ is set to the reference value φ r_n Below (φ≦φ r_n ), a higher second weighting coefficient η2 is assigned to the acquired face direction α.

[0087] Alternatively, when the processor 14 determines YES in step S6 and acquires the eye opening degree β as the driving state parameter PR, the processor 14 determines whether the eye opening degree β is greater than or equal to the threshold value β described above. th_n or more, and a predetermined reference value β r_n Within the range smaller than (β th_n ≦β<β r_n ), the first weighting coefficient η1 is assigned to the acquired opening β, while the opening β is equal to or smaller than the reference value β r_n or more (β r_n ≦β), a higher second weighting coefficient η2 is assigned to the acquired opening β.

[0088] Reference value PR for weighting coefficient η r_n (For example, the reference value φ r_n , or the reference value β for the opening β r_n ) is Driver D n The operating state DC is equal to or lower than the threshold value (for example, the threshold value φ th_n or β th_n ) is more appropriate than r_n <φ th_n The value of or β th_n <β r_n In this way, the processor 14 assigns the weighting coefficient η1 or η2 corresponding to the driving state parameter PR acquired when the determination in step S6 is YES to the driving state parameter PR.

[0089] As another example, when the determination in step S6 is YES, the processor 14 determines whether the steering angle λ acquired as the vehicle information CI is equal to or greater than a predetermined reference value λ r_n (or, the velocity V is greater than a predetermined reference value V r_n If the steering angle λ is smaller than the reference value λ, the first weighting coefficient η1 is assigned to the driving state parameter PR acquired when the determination in step S6 is YES. r_n (or the speed V is less than the reference value V r_nIn this way, the processor 14 assigns the weighting coefficient η1 or η2 corresponding to the vehicle information CI acquired when the determination in step S6 is YES to the driving state parameter PR.

[0090] The reference value PR of the operating state parameter PR referred to in order to assign the weighting coefficient η is r_n (For example, the reference value φ of the face direction α r_n ), or a reference value of the vehicle information CI (for example, a reference value λ of the steering angle λ r_n ) is the driver information DI n The information may be further stored in the database 12 in association with the

[0091] That is, in this case, the database 12 stores the driving state determination value DV n and the reference value PR for the weighting coefficient η r_n or λ r_n However, driver information DI n Then, in step S7, the processor 14 calculates the driving state determination value DV by taking into account the weighting coefficients η individually assigned to the plurality of driving state parameters PR repeatedly acquired each time the determination in step S6 is YES. n may be updated.

[0092] For example, each time the determination in step S6 is YES, the processor 14 sets the face direction α as the driving state parameter PR. m (m=1,2,3...) and obtain the face direction α m In this case, the processor 14 assigns a weighting coefficient η1 or η2 to each of the facial directions α m Weighted average α of the value obtained by multiplying by the weighting coefficient η1 or η2 m ' is calculated, and the reference face direction α 0_n The weighted average α m You can update it by replacing it with '.

[0093] Alternatively, the processor 14 may set the opening β as the operating state parameter PR each time the determination in step S6 is YES. m (m=1,2,3...) and obtain the opening β m In this case, the processor 14 calculates the weighting coefficient η1 or η2 for each opening β m Weighted average β of the value obtained by multiplying by the weighting coefficient η1 or η2 m ' is calculated, and the reference opening β 0_n The weighted average β m In this way, the driving state determination value DV n By updating the driving state judgment value DV n can be optimized more effectively.

[0094] In addition, in the above-mentioned Table 1, the database 12 contains multiple drivers D n Driver Information DI n and the driving state judgment value DV n However, in the first stage, the driver information DI is already stored in the database 12. n and driving state judgment value DV n may not be stored.

[0095] In this case, a common driving state determination value DV0 may be stored in advance in the memory 34 as an initial value. n Driver Information DI n (For example, Driver D n The identification code acquired from the portable device of the driver D or the image of the driver D captured by the image capturing device 26 n Next, in step S2, processor 14 may obtain a driving state determination value DV0 as an initial value from memory 34, and in step S5, which is executed for the first time, may use the driving state determination value DV0 to determine whether or not the driving state DC is appropriate.

[0096] Then, in step S7, which is executed for the first time, the processor 14 updates the driving state determination value DV0 based on the driving state parameter PR acquired when the determination in the most recent step S6 was YES, and sets the updated driving state determination value DV n The driver D obtained in step S1 n Driver Information DI n The information may be stored in the database 12 in association with the above.

[0097] In step S7, the processor 14 updates the driving state determination value DV n may be temporarily stored in the RAM of the memory 34. In this case, while the processor 14 determines NO in step S8 and repeatedly executes the loop of steps S5 to S9, when executing step S5, the operating state determination value DV n If the determination in step S8 is YES, the driving state determination value DV n Driver information DI n The data may be stored in the database 12 in association with the above and stored in the ROM of the memory 34.

[0098] Next, another example of the operation of the vehicle 20 will be described with reference to Fig. 6. In the flow shown in Fig. 6, the same processes as those in the flow shown in Fig. 3 are assigned the same step numbers, and duplicated explanations will be omitted. After the flow shown in Fig. 6 starts, in step S11, the processor 14 calculates new driver information DI n+1 It is determined whether an input operation to register the

[0099] Specifically, new driver D n+1 However, by operating the HMI30, the driver information DI n+1 In this case, the processor 14 displays a new registration operation image IMe on the display of the HMI 30. n+1The driver operates an input device (for example, a touch panel) of the HMI 30 and inputs the driver information DI through a new registration operation image IMe displayed on the display. n+1 In step S11, the processor 14 inputs the driver information DI n+1 If the input operation is accepted, the determination is YES and the process proceeds to step S12, whereas if the determination is NO, the process proceeds to step S1.

[0100] In step S12, the processor 14 operates the imaging device 26 to capture the driver D n+1 At this time, the image capturing device 26 captures the face of the driver D. n+1 The processor 14 controls the driver D through the HMI 30 so that the front face of the driver D can be captured. n+1 The processor 14 may display guide information for directing the imaging device 26 in the direction of the driver D captured by the imaging device 26 as an image on the new registration operation image IMe or output it as sound from the speaker. n+1 Face image IMf n+1 (Image data or feature information obtained by coding feature points extracted from the image data) is used as driver information DI n+1 New facial image DI2 included in n+1 and registers it in the database 12 as a new entry.

[0101] In step S13, the processor 14 calculates the driving state determination value DV n+1 For example, the processor 14 sets the initial value of the driver D captured in step S12. n+1 Face image IMf n+1 (That is, driver information DI n+1 Newly registered face image DI2 n+1 ) to Driver D n+1 The face direction α and opening degree β of the driver are calculated, and the face direction α and opening degree β are used as the driving state determination value DV n+1 The reference face direction α 0_n+1 and reference opening β 0_n+1 Set it anew as

[0102] Furthermore, the processor 14 determines whether the newly set reference face direction α 0_n+1 new threshold φ th_n+1 and set the newly set reference opening β 0_n+1 A new threshold β th_n+1 The processor 14 may set the set driving state determination value DV n+1 The initial value of (for example, the reference face direction α 0_n+1 and threshold φ th_n+1 , standard opening degree β 0_n+1 and threshold β th_n+1 ) and driver information DI n+1 and stores the information in the database 12 in association with the above.

[0103] After step S13, the processor 14 sequentially executes steps S1 to S10 in the same manner as in the flow of FIG. 3, and in step S7, determines the driver D set at this time based on the driving state parameter PR acquired when the most recent step S6 is determined as YES. n+1 Operating state judgment value DV n+1 Update the initial value of

[0104] According to this embodiment, the processor 14 calculates the driver information DI n+1 Facial image IMf taken for new registration n+1 Based on this, the driving state determination value DV n+1 In step S7 of FIG. 6, the processor 14 automatically sets the initial value of the driving state determination value DV n While updating part of the driving state judgment value DV n Alternatively, some of the parameters may not be updated and the initial values ​​set in step S13 may be maintained.

[0105] For example, in step S7, the processor 14 determines the reference face direction α 0_n and φ th_n , the reference blink frequency γ 0_n and threshold γ th_n , and the reference switching frequency ε 0_n and threshold ε th_n While updating the reference opening β 0_n and threshold β th_nIn step S13, the driver D n The reference opening β set for 0_n and threshold β th_n The initial value of may be maintained.

[0106] Here, in step S12, a face image IMf n When capturing the image, the driver D n Therefore, in step S7, the reference opening angle β 0_n and threshold β th_n By maintaining the initial value without updating, it is possible to more accurately determine whether the operating state DC is appropriate based on the opening β.

[0107] 6, steps S13 and S2 may be omitted. In this case, the processor 14 determines the driving state determination value DV based on the acquired driving state parameter PR (for example, the face direction α) in step S7 that is executed for the first time. n (For example, the reference face direction α 0_n and threshold φ th_n Then, every time step S7 is executed from the second time onward, the processor 14 sets the initial value of the driving state determination value DV n Update.

[0108] In the above-described embodiment, the processor 14 determines whether the new driver D n+1 The driver information DI is displayed according to the input operation of n+1 However, the processor 14 does not register the driver D n+1 Without accepting the input operation of the driver information DI n+1 may be automatically registered.

[0109] Specifically, in step S1 of the flow of FIG. 3, the processor 14 operates the imaging device 26 to capture the image of a new driver D who has entered the vehicle 20. n+1 The face of the driver D is photographed. n+1 Face image IMf n+1 and driver information DI in database 12.n Facial image DI2 stored as n As a result, the processor 14 determines whether the driver D who got into the vehicle 20 n+1 However, the driver information DI stored in advance in the database 12 n Assume that you realize that it is not included in

[0110] In this case, the processor 14 converts the captured face image IMf n+1 New driver D n+1 Driver Information DI n+1 The processor 14 may then write the new information about this driver D n+1 After starting driving, when step S7 is executed for the first time, the driving state determination value DV is calculated based on the acquired driving state parameter PR (for example, the face direction α). n+1 (For example, the reference face direction α 0_n+1 and threshold φ th_n+1 ) and set the initial value of the driver information DI n+1 The information may be stored in the database 12 in association with the above.

[0111] In this case, the processor 14 n+1 However, driver D, who is registered in database 12, n Driver Information DI n Only if the driver has at least one of the following, new driver information DI n+1 may be automatically added to the database 12. For example, a new driver D n+1 However, the driver D1 has a portable device (for example, a smart key) that stores an identification code DI12 of a second driver D2 who is registered in the database 12 in advance.

[0112] In this case, in step S1, the processor 14 detects a new driver D who has entered the vehicle 20. n+1 is recognized as the second driver D2 registered in advance in the database 12, while the imaging device 26 is operated to recognize a new driver D n+1 The processor 14 captures the face of the driver D. n+1Face image IMf n+1 As a result of comparing the face image DI22 stored in the database 12 as the driver information DI2 of the second driver D2 with the face image DI22 stored in the database 12 as the driver information DI2 of the second driver D2, n+1 However, the driver D2 registered in the database 12 will recognize that the driver is a different person.

[0113] In this case, the processor 14 converts the captured face image IMf n+1 New driver D n+1 Driver Information DI n+1 The driver information DI may be newly written to the database 12. n+1 As, face image IMf n+1 and the identification code DI12 of the second driver D2 may be stored.

[0114] The processor 14 detects the new driver D n+1 Face image IMf n+1 and the driver information DI stored in the database 12. n Face image DI2 n If the driver D is not identified as a legitimate driver in step S1, n If the driver cannot be identified as the driver, the driver information DI is stored in the database 12. n Registered driver D n A visual or audio warning signal AL2 may be transmitted to the portable device through the communication device 32.

[0115] In the above embodiment, the driving state parameters PR are exemplified by the face direction α, the blink frequency γ, the mouth opening / closing frequency ε, and the line of sight δ. However, the driving state parameters PR may be, for example, the driving state parameters PR of the driver D while driving. n The amount of head movement of the driver D, or the amount of change over time of the steering angle λ (for example, the time differential value: Δλ / Δt), n The parameter may include any parameter that quantitatively represents the operating state DC of the vehicle.

[0116] The database 12 is not limited to being stored in the memory 34 of the vehicle 20, but may be stored, for example, in a management server (not shown) outside the vehicle 20. In this case, the processor 14 communicates with the management server via the communication device 32 and receives information from the management server about the database 12 (i.e., driver information DI n , and the driving state judgment value DV n ) may be obtained.

[0117] In this case, the processor 14 performs the following steps: n Face image IMf n is transmitted to the management server via the communication device 32, and the management server receives the facial image IMf n and a face image DI2 stored in advance in the database 12. n By comparing this, driver D n Although the present disclosure has been described above through the embodiments, the invention according to the claims is not limited to the above-described embodiments. [Explanation of symbols]

[0118] 10 Driver Monitoring System 12 Databases 14 processors 20 vehicles 24 ECU 26 Imaging device

Claims

1. A driver monitor system that determines whether a driving state of a driver who drives a vehicle is appropriate, a database that stores driving state determination values ​​for determining the suitability of the driving state determination values ​​in association with the driver information of each of the plurality of drivers; a processor that determines whether or not a first driver is suitable for driving by using a first driving state determination value that is stored in the database in association with first driver information of a first driver identified based on the driver information, The processor: estimating whether the first driver is looking ahead of the vehicle based on vehicle information representing a behavior of the vehicle while driving; When it is estimated that the first driver is looking ahead, a driving state parameter representing the driving state of the first driver is acquired; updating the first driving state determination value based on the driving state parameter acquired when it is estimated that the driver is looking ahead; The driver monitor system stores the updated first driving state determination value in the database in association with the first driver information.

2. Further, an imaging device is provided to capture an image of the face of the driver while driving, The driver monitor system according to claim 1 , wherein the processor acquires the driving state parameter of the first driver based on a face image of the first driver captured by the imaging device.

3. The driver monitor system according to claim 1 , wherein the processor updates the first driving state determination value based on a plurality of the driving state parameters repeatedly acquired while the first driver is driving.

4. 4. The driver monitor system according to claim 3, wherein the processor determines a new driving state determination value from the plurality of driving state parameters by a predetermined calculation, and updates the first driving state determination value with the new driving state determination value.

5. 2. The driver monitor system according to claim 1, wherein the vehicle information includes at least one of a steering angle of the vehicle, a speed of the vehicle, an environmental image captured around the vehicle, and position information of the vehicle.

6. The driving state parameter includes a face direction of the driver or an eye opening degree of the driver, The driving state determination value is a reference face direction determined based on the face direction of the driver looking ahead of the vehicle, and a threshold value for the angle of the face direction relative to the reference face direction; or 2. The driver monitor system according to claim 1, further comprising a reference opening degree determined based on the opening degree when the driver has the eyes open, and a threshold value for the opening degree determined based on the reference opening degree.

7. the driver information includes a facial image of the driver, The driver monitor system further includes an imaging device that images a face of the driver before driving, 2. The driver monitor system according to claim 1, wherein the processor identifies the driver by comparing a facial image of the driver captured by the imaging device with the facial image stored in advance as the driver information in the database.

8. A vehicle comprising the driver monitor system of claim 1.

9. A driver monitoring method for determining whether a driving state of a driver who drives a vehicle is appropriate, comprising: preparing a database that stores driving state determination values ​​for determining the suitability of the driving state determination values ​​in association with the driver information of each of the plurality of drivers; The processor: determining whether or not the first driver is driving using a first driving state determination value stored in the database in association with first driver information of a first driver identified based on the driver information; estimating whether the first driver is looking ahead of the vehicle based on vehicle information representing a behavior of the vehicle while driving; When it is estimated that the first driver is looking ahead, a driving state parameter representing the driving state of the first driver is acquired; updating the first driving state determination value based on the driving state parameter acquired when it is estimated that the driver is looking ahead; The updated first driving state determination value is stored in the database in association with the first driver information.

Images