Drunk-state determination device, drunk-state determination system, and drunk-state determination method

Abstract

A drunk-state determination device (3) according to the present disclosure comprises: an image data acquisition unit (11) that acquires image data representing a facial image from an imaging device (1) for capturing an image of a driver's face; an alertness level estimation unit (12) that estimates the driver's alertness level on the basis of the image data acquired by the image data acquisition unit (11); and a vital sign estimation unit (13) that estimates vital signs of the driver on the basis of either the image data acquired by the image data acquisition unit (11) or sensing data from a sensor (2) for sensing the vital signs of the driver. In addition, the drunk-state determination device (3) according to the present disclosure is also provided with a drunk-state determination unit (14) that determines the drunk state of the driver on the basis of the alertness level estimated by the alertness level estimation unit (12) and the vital signs estimated by the vital sign estimation unit (13).

Description

Drinking level determination device, drinking level determination system, and drinking level determination method

[0001] The present disclosure relates to a drinking level determination device, a drinking level determination system, and a drinking level determination method.

[0002] There is a drinking level determination device that determines the drinking level of a vehicle driver. Patent Document 1, for example, discloses a drunk driving prevention device that includes a memory unit, an imaging unit, and a determination unit. The memory unit stores photographed data showing an image of the driver's face when the driver was sober. The photographed data shows an image of the driver's face taken when an alcohol detector detects that the driver is not drinking alcohol after getting into the vehicle and before starting the engine. The imaging unit captures an image of the driver's face when the driver starts driving the vehicle and outputs the photographed data showing the image of the driver's face to the determination unit. The determination unit determines whether or not the driver's facial color has changed by comparing the photographed data output from the imaging unit with the photographed data stored in the memory unit. If the facial color changes, the determination unit determines that the driver has drunk alcohol, and if the facial color does not change, the determination unit determines that the driver has not drunk alcohol. If the determination unit determines that the driver has drunk alcohol, the drunk driving prevention device, for example, performs a vehicle stop process.

[0003] JP 2010-52635 A

[0004] The drunk driving prevention device disclosed in Patent Document 1 has a problem in that the determination unit determines whether the driver has drunk alcohol based on changes in facial color, which can lead to an erroneous determination of whether the driver has drunk alcohol. This is because the driver's facial color can change due to factors other than drinking. Factors other than drinking that can change the driver's facial color include, for example, sunlight shining into the vehicle or a decrease in brightness around the vehicle when entering a tunnel. In such cases, the driver's drinking level can be erroneously determined.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a drinking state determination device that can reduce erroneous determinations of whether or not a driver has been drinking compared to the drunk driving prevention device disclosed in Patent Document 1.

[0006] The drinking level determination device according to the present disclosure includes an image data acquisition unit that acquires image data representing an image of the driver's face from an imaging device that captures an image of the driver's face, an alertness estimation unit that estimates the driver's alertness based on the image data acquired by the image data acquisition unit, and a vital sign estimation unit that estimates the driver's vital signs based on either the image data acquired by the image data acquisition unit or sensing data from a sensor that senses the driver's vital signs. The drinking level determination device also includes a drinking level determination unit that determines the drinking level of the driver based on the alertness estimated by the alertness estimation unit and the vital signs estimated by the vital sign estimation unit.

[0007] According to the present disclosure, it is possible to reduce erroneous determination of whether or not a driver has been drinking compared to the drunk driving prevention device disclosed in Patent Document 1.

[0008] FIG. 1 is a configuration diagram showing a drinking level determination device 3 according to embodiment 1. FIG. 2 is a hardware configuration diagram showing the hardware of the drinking level determination device 3 according to embodiment 1. FIG. 3 is a hardware configuration diagram of a computer in the case where the drinking level determination device 3 is realized by software, firmware, or the like. FIG. 4 is a flowchart showing a drinking level determination method, which is a processing procedure of the drinking level determination device 3. FIG. 5 is a configuration diagram showing a drinking level determination device 3 according to embodiment 2. FIG. 6 is a hardware configuration diagram showing the hardware of the drinking level determination device 3 according to embodiment 2. FIG. 7 is a hardware configuration diagram showing a drinking level determination device 3 according to embodiment 3. FIG. 8 is a hardware configuration diagram showing the hardware of the drinking level determination device 3 according to embodiment 3. FIG. 9 is a configuration diagram showing a drinking level determination device 3 according to embodiment 4. FIG. 10 is a hardware configuration diagram showing the hardware of the drinking level determination device 3 according to embodiment 4. FIG. 11 is a configuration diagram showing a drinking level determination system according to embodiment 5.

[0009] In order to explain the present disclosure in more detail, embodiments of the present disclosure will be described below with reference to the accompanying drawings.

[0010] Embodiment 1 Fig. 1 is a configuration diagram showing a drinking level determination device 3 according to embodiment 1. Fig. 2 is a hardware configuration diagram showing the hardware of the drinking level determination device 3 according to embodiment 1. The drinking level determination device 3 shown in Fig. 1 includes an image data acquisition unit 11, an alertness level estimation unit 12, a vital sign estimation unit 13, and a drinking level determination unit 14.

[0011] 1, the imaging device 1 is installed, for example, on the instrument panel, the A-pillar, or the ceiling of the vehicle. The imaging device 1 captures an image of the driver's face and outputs image data representing the image of the face to the drinking level determination device 3.

[0012] The sensor 2 is installed, for example, on a vehicle seat, an instrument panel, an A-pillar, or a ceiling of the vehicle. The sensor 2 senses the driver's vital signs and outputs the sensing data of the vital signs to the drinking level determination device 3. The driver's vital signs include, for example, the driver's heart rate or blood pressure.

[0013] The image data acquisition unit 11 is realized by, for example, the image data acquisition circuit 21 shown in Fig. 2. The image data acquisition unit 11 acquires image data representing a face image from the imaging device 1. The image data acquisition unit 11 outputs the image data to each of the arousal level estimation unit 12 and the vital sign estimation unit 13.

[0014] The alertness estimation unit 12 is realized by, for example, an alertness estimation circuit 22 shown in Fig. 2. The alertness estimation unit 12 acquires image data from the image data acquisition unit 11. The alertness estimation unit 12 estimates the alertness of the driver based on the image data. The alertness estimation unit 12 outputs alertness data indicating the alertness of the driver to the drinking state determination unit 14.

[0015] The vital sign estimation unit 13 is realized by, for example, a vital sign estimation circuit 23 shown in FIG. 2 . The vital sign estimation unit 13 acquires at least one of image data acquired by the image data acquisition unit 11 and sensing data output from the sensor 2. When the vital sign estimation unit 13 acquires image data from the image data acquisition unit 11, it estimates the driver's vital signs based on the image data. When the vital sign estimation unit 13 acquires sensing data from the sensor 2, it estimates the driver's vital signs based on the sensing data. When the vital sign estimation unit 13 acquires both image data and sensing data, it estimates the driver's vital signs based on the image data and the sensing data. The vital sign estimation unit 13 outputs vital sign data indicating the driver's vital signs to the drinking level determination unit 14.

[0016] The drinking level determination unit 14 is realized, for example, by the drinking level determination circuit 24 shown in FIG. 2. The drinking level determination unit 14 includes a drinking score calculation unit 14a and a drinking level determination processing unit 14b. The drinking level determination unit 14 acquires alertness data from the alertness estimation unit 12 and vital data from the vital sign estimation unit 13. The drinking level determination unit 14 determines the drinking level of the driver based on the alertness indicated by the alertness data and the vital signs indicated by the vital sign data. The drinking level determination unit 14 outputs the drinking level determination result to, for example, a driver monitoring system (hereinafter referred to as "DMS") of the vehicle.

[0017] The drinking score calculation unit 14a acquires the alertness data from the alertness estimation unit 12 and the vital data from the vital sign estimation unit 13. The drinking score calculation unit 14a calculates a drinking score indicating the degree of drinking state of the driver from the alertness indicated by the alertness data and the vital signs indicated by the vital sign data. The drinking score calculation unit 14a outputs the drinking score to the drinking state determination processing unit 14b.

[0018] The drinking level determination processing unit 14b acquires the drinking score from the drinking score calculation unit 14a. The drinking level determination processing unit 14b compares the drinking score with a threshold, and determines whether the driver is in a drinking state based on the comparison result between the drinking score and the threshold.

[0019] 1, it is assumed that each of the components of the drinking state determination device 3, namely, the image data acquisition unit 11, the alertness estimation unit 12, the vital sign estimation unit 13, and the drinking state determination unit 14, is realized by dedicated hardware as shown in FIG. 2. That is, it is assumed that the drinking state determination device 3 is realized by an image data acquisition circuit 21, an alertness estimation circuit 22, a vital sign estimation circuit 23, and a drinking state determination circuit 24. Each of the image data acquisition circuit 21, the alertness estimation circuit 22, the vital sign estimation circuit 23, and the drinking state determination circuit 24 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination thereof.

[0020] The components of the drinking level determination device 3 are not limited to those realized by dedicated hardware, and the drinking level determination device 3 may be realized by software, firmware, or a combination of software and firmware. The software or firmware is stored as a program in the memory of a computer. The computer refers to hardware that executes a program, and includes, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, a processor, or a DSP (Digital Signal Processor).

[0021] 3 is a hardware configuration diagram of a computer when the drinking level determination device 3 is realized by software, firmware, etc. When the drinking level determination device 3 is realized by software, firmware, etc., a program for causing the computer to execute the processing procedures of the image data acquisition unit 11, the alertness estimation unit 12, the vital signs estimation unit 13, and the drinking level determination unit 14 is stored in a memory 31. A processor 32 of the computer then executes the program stored in the memory 31.

[0022] 2 shows an example in which each of the components of the drinking level determination device 3 is realized by dedicated hardware, while Fig. 3 shows an example in which the drinking level determination device 3 is realized by software, firmware, etc. However, this is merely an example, and some of the components of the drinking level determination device 3 may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, etc.

[0023] Next, the operation of the drinking level determination device 3 shown in Fig. 1 will be described. Fig. 4 is a flowchart showing a drinking level determination method, which is a processing procedure of the drinking level determination device 3. The imaging device 1 captures an image of the driver's face and outputs image data representing the face image to the drinking level determination device 3. The sensor 2 senses the driver's vital signs and outputs sensing data of the vital signs to the drinking level determination device 3.

[0024] The image data acquisition unit 11 acquires image data representing a face image from the imaging device 1 (step ST1 in FIG. 4 ). The image data acquisition unit 11 outputs the image data to the wakefulness estimation unit 12 and the vital sign estimation unit 13, respectively.

[0025] The alertness estimation unit 12 acquires image data from the image data acquisition unit 11. The alertness estimation unit 12 estimates the alertness AL of the driver based on the image data (step ST2 in FIG. 4 ). The alertness estimation unit 12 outputs alertness data indicating the alertness AL of the driver to the drinking state determination unit 14.

[0026] The wakefulness estimation process by the wakefulness estimation unit 12 will be specifically described below. The wakefulness estimation unit 12 detects the eyes of the driver by analyzing the image of the face shown in the image data. The eye detection process itself is a known technique, so a detailed description will be omitted. The wakefulness estimation unit 12 measures the time Ta during which the driver's eyes are open and the time Tb during which the driver's eyes are closed in a certain period FI. The eye open time Ta is calculated by calculating the time when the distance L between the upper eyelid and the lower eyelid in the certain period FI is equal to or smaller than a threshold value Th Ｌ The time Tb during which the eyes are closed is the total time during which the distance L between the upper and lower eyelids is equal to or greater than the threshold value Th Ｌ The total time that the threshold value Th is less than Ｌ may be stored in the internal memory of the alertness estimation unit 12, or may be provided from outside the drinking state determination device 3. The distance L between the upper eyelid and the lower eyelid may be the distance of the longest line segment in the vertical direction of the face that is approximately perpendicular to the line segment connecting the inner corner and outer corner of the eye. The fixed period FI is, for example, a period from the current time to a preset time ago.

[0027] Next, the wakefulness level estimation unit 12 calculates the ratio of the time Ta to the total time of the time Ta and the time Tb as the eye open rate R, as shown in the following formula (1).

[0028]

[0029] The wakefulness estimation unit 12 calculates the wakefulness AL of the driver based on the eye opening rate R as shown in the following equation (2): AL=R×c (2) In equation (2), c is a constant.

[0030] Here, the wakefulness estimation unit 12 calculates the wakefulness AL on the assumption that the wakefulness AL is proportional to the eye opening rate R. The wakefulness AL may be any value that increases as the eye opening rate R increases, and the wakefulness AL is not limited to being proportional to the eye opening rate R.

[0031] The vital sign estimation unit 13 acquires at least one of the image data acquired by the image data acquisition unit 11 and the sensing data output from the sensor 2. When the vital sign estimation unit 13 acquires image data from the image data acquisition unit 11, it estimates the driver's vital sign VT by analyzing the facial image represented by the image data (step ST3 in FIG. 4). The process of estimating vital signs by analyzing the facial image is a well-known technique, and therefore a detailed description thereof will be omitted. When the sensing data is acquired from the sensor 2, the vital sign estimation unit 13 estimates the driver's vital sign VT based on the sensing data.

[0032] Upon acquiring both the image data and the sensing data, the vital sign estimation unit 13 calculates, as the driver's vital sign VT, for example, a weighted sum of the vital signs estimated from the image data and the vital signs estimated from the sensing data, or an average value of the vital signs estimated from the image data and the vital signs estimated from the sensing data. The vital sign estimation unit 13 outputs the vital sign data indicating the driver's vital sign VT to the drinking state determination unit 14.

[0033] The drinking score calculation unit 14a of the drinking state determination unit 14 acquires the alertness data from the alertness estimation unit 12 and the vital data from the vital sign estimation unit 13. The drinking score calculation unit 14a calculates a drinking score Dsc indicating the degree of drinking state of the driver from the alertness AL indicated by the alertness data and the vital sign VT indicated by the vital sign data (step ST4 in FIG. 4).

[0034] The calculation process of the drinking score calculation unit 14a to calculate the drinking score Dsc will be described in detail below. The drinking score calculation unit 14a provides the alertness data and vital sign data to the learning model LM. The learning model LM may be included in the drinking score calculation unit 14a or may be provided externally to the drinking score calculation unit 14a. The learning model LM is realized, for example, by a neural network. The learning model LM is provided with alertness data, vital sign data, and teacher data during learning. The teacher data indicates the drinking score Dsc when the driver's alertness is AL and the driver's vital sign is VT. When the alertness data, vital sign data, and teacher data are provided, the learning model LM learns the drinking score Dsc corresponding to the alertness data and vital sign data. When the alertness data and vital sign data are provided, the learning model LM outputs the drinking score Dsc corresponding to the alertness data and vital sign data to the drinking score calculation unit 14a during inference. The drinking score calculation unit 14a acquires the drinking score Dsc from the learning model LM and outputs the drinking score Dsc to the drinking state determination processing unit 14b.

[0035] The alertness level AL indicated by the alertness data is likely to decrease as the driver drinks alcohol. The alertness level AL may decrease as the driver becomes more drowsy, even when the driver is not drinking alcohol. The heart rate included in the vital VT indicated by the vital data is likely to increase as the driver drinks alcohol. The heart rate included in the vital VT may decrease as the driver becomes more drowsy. The blood pressure included in the vital VT indicated by the vital data is likely to decrease as the driver drinks alcohol. Therefore, when the driver drinks alcohol, there is a high probability that both a decrease in the alertness level AL and an increase in the heart rate will occur. Furthermore, when the driver drinks alcohol, there is a high probability that both a decrease in the alertness level AL and a decrease in blood pressure will occur. On the other hand, when the driver is not drinking alcohol, even if the alertness level AL decreases, there is a low probability that an increase in the heart rate will occur. Furthermore, when the driver is not drinking alcohol, even if the alertness level AL decreases, there is a low probability that a decrease in blood pressure will occur. The level of alertness AL, heart rate, and blood pressure are hardly affected by, for example, sunlight shining into the vehicle or by a decrease in brightness around the vehicle when the vehicle enters a tunnel.

[0036] Here, the drinking score calculation unit 14a calculates the drinking score Dsc using the learning model LM. However, this is merely an example, and the drinking score calculation unit 14a may calculate the drinking score Dsc from the alertness data and the vital data by, for example, executing a random forest method or a logistic regression analysis.

[0037] The drinking level determination processing unit 14b acquires the drinking score Dsc from the drinking score calculation unit 14a. The drinking level determination processing unit 14b calculates the drinking score Dsc and the threshold value Th Ｄ Compare with the threshold value Th Ｄ may be stored in an internal memory of the drinking level determination processing unit 14b, or may be provided from outside the drinking level determination device 3. The drinking level determination processing unit 14b Ｄ If the drinking score Dsc is equal to or greater than the threshold value Th (step ST5 in FIG. 4: YES), the drinking state determination processing unit 14b determines that the driver is in a drinking state (step ST6 in FIG. 4).Ｄ If the alcohol intake is less than 1000 kcal (step ST5 in FIG. 4: NO), the drinking level determination processing unit 14b determines that the driver is not drinking alcohol (step ST7 in FIG. 4). The drinking level determination processing unit 14b outputs the result of the drinking level determination to, for example, the DMS.

[0038] Here, the drinking level determination processing unit 14b calculates the drinking score Dsc and the threshold value Th Ｄ However, this is only an example, and two or more different thresholds Th Ｄ The drinking level determination processing unit 14b calculates the drinking score Dsc and two or more thresholds Th Ｄ In this case, the drinking level determination processing unit 14b can not only determine whether the driver is in a drinking state, but also determine the degree of drinking of the driver. For example, Ｄ１ , Th Ｄ２ (Th Ｄ１ <Th Ｄ２ ) is prepared, the drinking level determination processing unit 14b determines whether the drinking score Dsc is greater than or equal to the threshold value Th Ｄ２ If the drinking score Dsc is equal to or greater than the threshold value Th Ｄ１ is equal to or greater than the threshold value Th Ｄ２ If the drinking score Dsc is less than the threshold value Th Ｄ１ If the reading is less than this, it is determined that the person is not in a drunk state.

[0039] If the result of the drinking state determination indicates that the driver is drunk, the DMS performs, for example, stop control to stop the vehicle or display processing to display a message urging the driver to stop the vehicle.If the result of the drinking state determination indicates that the driver is not drunk, the DMS allows the driver to continue driving the vehicle.

[0040] In the first embodiment described above, the drinking state determination device 3 is configured to include an image data acquisition unit 11 that acquires image data representing an image of the driver's face from an imaging device 1 that captures an image of the driver's face, an alertness estimation unit 12 that estimates the driver's alertness based on the image data acquired by the image data acquisition unit 11, and a vital sign estimation unit 13 that estimates the driver's vital signs based on either the image data acquired by the image data acquisition unit 11 or sensing data from a sensor 2 that senses the driver's vital signs. The drinking state determination device 3 also includes a drinking state determination unit 14 that determines the drinking state of the driver based on the alertness estimated by the alertness estimation unit 12 and the vital signs estimated by the vital sign estimation unit 13. Therefore, the drinking state determination device 3 can reduce erroneous determinations of whether or not the driver has been drinking compared to the drunk driving prevention device disclosed in Patent Document 1.

[0041] In the drinking level determination device 3 shown in FIG. 1 , the alertness estimation unit 12 calculates the eye open rate R and calculates the driver's alertness AL from the eye open rate R. However, this is merely an example, and the alertness estimation unit 12 may, for example, detect the number of times the driver yawns within a certain period FI by analyzing a facial image represented by image data. The yawn detection process itself is a well-known technique, and therefore a detailed description thereof will be omitted. The alertness estimation unit 12 may calculate the alertness AL so that the more times the driver yawns, the lower the alertness AL becomes.

[0042] In the drinking level determination device 3 shown in FIG. 1 , the drinking level determination unit 14 determines the drinking level of the driver based on the alertness estimated by the alertness estimation unit 12 and the vital signs estimated by the vital sign estimation unit 13. However, this is merely an example, and the drinking level determination unit 14 may determine the drinking level of the driver based on the alertness estimated by the alertness estimation unit 12, the vital signs estimated by the vital sign estimation unit 13, and changes in the driver's facial color. Alternatively, the drinking level determination unit 14 may determine the drinking level of the driver based on the alertness estimated by the alertness estimation unit 12, the vital signs estimated by the vital sign estimation unit 13, and the proportion of time the driver is gazing ahead. If the driver is drinking, there is a high possibility that the proportion of time the driver is gazing ahead will decrease.

[0043] Second Embodiment In a second embodiment, a drinking level determination device 3 will be described that includes a vehicle information acquisition unit 15 that acquires vehicle information indicating the driving operation of the driver from the vehicle being driven by the driver.

[0044] Fig. 5 is a configuration diagram showing a drinking level determination device 3 according to embodiment 2. In Fig. 5, the same reference numerals as in Fig. 1 indicate the same or corresponding parts, and detailed description thereof will be omitted. Fig. 6 is a hardware configuration diagram showing the hardware of the drinking level determination device 3 according to embodiment 2. In Fig. 6, the same reference numerals as in Fig. 2 indicate the same or corresponding parts, and detailed description thereof will be omitted. The drinking level determination device 3 shown in Fig. 5 includes an image data acquisition unit 11, an alertness level estimation unit 12, a vital sign estimation unit 13, a vehicle information acquisition unit 15, and a drinking level determination unit 16.

[0045] The vehicle information acquisition unit 15 is realized, for example, by a vehicle information acquisition circuit 25 shown in FIG. 6 . The vehicle information acquisition unit 15 acquires vehicle information indicating changes in the vehicle state caused by the driver's vehicle operation from the vehicle being driven. Specifically, if the driver's vehicle operation is a steering wheel operation, the vehicle information acquisition unit 15 acquires vehicle information indicating changes in the steering angle caused by the steering wheel operation as the vehicle information. If the driver's vehicle operation is either a braking operation or an accelerator operation, the vehicle information acquisition unit 15 acquires vehicle information indicating changes in vehicle speed caused by the braking operation as the vehicle information. The vehicle information acquisition unit 15 outputs the vehicle information to the drinking level determination unit 16.

[0046] The drinking state determination unit 16 is realized, for example, by the drinking state determination circuit 26 shown in FIG. 6 . The drinking state determination unit 16 includes a drinking score calculation unit 16a and a drinking state determination processing unit 16b. The drinking state determination unit 16 acquires alertness data from the alertness estimation unit 12 and vital sign data from the vital sign estimation unit 13. The drinking state determination unit 16 also acquires vehicle information from the vehicle information acquisition unit 15. The drinking state determination unit 16 determines the drinking state of the driver based on the alertness indicated by the alertness data, the vital signs indicated by the vital sign data, and the vehicle information. The drinking state determination unit 16 outputs the drinking state determination result to, for example, the vehicle's DMS.

[0047] The drinking score calculation unit 16a acquires alertness data from the alertness estimation unit 12 and vital data from the vital sign estimation unit 13. The drinking score calculation unit 16a also acquires vehicle information from the vehicle information acquisition unit 15. The drinking score calculation unit 16a calculates a drinking score indicating the degree of drinking state of the driver from the alertness indicated by the alertness data, the vital signs indicated by the vital sign data, and the vehicle information. The drinking score calculation unit 16a outputs the drinking score to the drinking state determination processing unit 16b.

[0048] The drinking level determination processing unit 16b acquires the drinking score from the drinking score calculation unit 16a. The drinking level determination processing unit 16b compares the drinking score with a threshold, and determines whether the driver is in a drinking state based on the comparison result between the drinking score and the threshold.

[0049] 5, it is assumed that each of the components of the drinking level determination device 3, namely, the image data acquisition unit 11, the alertness estimation unit 12, the vital sign estimation unit 13, the vehicle information acquisition unit 15, and the drinking level determination unit 16, is realized by dedicated hardware such as that shown in FIG. 6. That is, it is assumed that the drinking level determination device 3 is realized by an image data acquisition circuit 21, an alertness estimation circuit 22, a vital sign estimation circuit 23, a vehicle information acquisition circuit 25, and a drinking level determination circuit 26. Each of the image data acquisition circuit 21, the alertness estimation circuit 22, the vital sign estimation circuit 23, the vehicle information acquisition circuit 25, and the drinking level determination circuit 26 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof.

[0050] The components of the drinking level determination device 3 are not limited to those realized by dedicated hardware, and the drinking level determination device 3 may be realized by software, firmware, or a combination of software and firmware. When the drinking level determination device 3 is realized by software, firmware, or the like, a program for causing a computer to execute the processing procedures of the image data acquisition unit 11, the alertness estimation unit 12, the vital signs estimation unit 13, the vehicle information acquisition unit 15, and the drinking level determination unit 16 is stored in a memory 31 shown in Fig. 3. Then, a processor 32 shown in Fig. 3 executes the program stored in the memory 31.

[0051] 6 shows an example in which each of the components of the drinking level determination device 3 is realized by dedicated hardware, while Fig. 3 shows an example in which the drinking level determination device 3 is realized by software, firmware, etc. However, this is merely an example, and some of the components of the drinking level determination device 3 may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, etc.

[0052] Next, the operation of the drinking level determination device 3 shown in Fig. 5 will be described. Except for the vehicle information acquisition unit 15 and the drinking level determination unit 16, the drinking level determination device 3 shown in Fig. 1 is the same as that shown in Fig. 1. Therefore, only the operation of the vehicle information acquisition unit 15 and the drinking level determination unit 16 will be described here.

[0053] The vehicle information acquisition unit 15 acquires vehicle information indicating the driving operation of the vehicle by the driver from the vehicle being driven by the driver. If the driver's vehicle operation is a steering operation, the vehicle information acquisition unit 15 acquires vehicle information indicating a change in steering angle accompanying the steering operation as the vehicle information. If the driver's vehicle operation is either a braking operation or an accelerator operation, the vehicle information acquisition unit 15 acquires vehicle information indicating a change in vehicle speed accompanying the braking operation as the vehicle information. The vehicle information acquisition unit 15 outputs the vehicle information to the drinking level determination unit 16. When the driver has been drinking, the driving operation of the vehicle tends to be more unstable than when the driver has not been drinking. Specifically, when the driver has been drinking, the amount of change in steering angle per certain period of time tends to be larger. Furthermore, when the driver has been drinking, the amount of change in vehicle speed per certain period of time tends to be larger.

[0054] The drinking score calculation unit 16a acquires arousal level data from the arousal level estimation unit 12 and acquires vital data from the vital sign estimation unit 13. The drinking score calculation unit 16a acquires vehicle information from the vehicle information acquisition unit 15. If the driver's vehicle operation is a steering operation, the drinking score calculation unit 16a calculates the amount of change Δx in the steering angle as shown in the following equation (3). In the example of equation (3), the amount of change Δx in the steering angle is calculated by multiplying the steering angle x at sampling time i by ｉ and the steering angle x at the sampling time (i-n) n hours before the sampling time i. ｉ－ｎ The absolute value of the difference between is divided by n hours.

[0055]

[0056] If the driver's vehicle operation is either a brake operation or an accelerator operation, the drinking score calculation unit 16a calculates the vehicle speed y at the sampling time i as shown in the following equation (4):ｉ The vehicle speed variance Δy is calculated using the average vehicle speed y bar from the sampling time (i-n) n hours before the sampling time i to the sampling time i. In the text of the specification, due to electronic filing, the letter "y" cannot be preceded by a "-" sign, so it is written as "y bar".

[0057]

[0058] Here, the drinking score calculation unit 16a calculates the vehicle speed variance Δy using the average vehicle speed y. However, this is merely an example, and the drinking score calculation unit 16a may calculate the vehicle speed variance Δy using, for example, the median vehicle speed from sampling time (i-n) to sampling time i instead of the average vehicle speed y.

[0059] A learning model LM' is provided inside or outside the drinking score calculation unit 16a. The learning model LM' is realized, for example, by a neural network. During learning, the learning model LM' is provided with alertness data, vital sign data, and teacher data. During learning, the learning model LM' is also provided with at least one of a steering angle change amount Δx and a vehicle speed variance amount Δy. The teacher data indicates the drinking score Dsc when the driver's alertness level is AL, the driver's vital signs are VT, and the steering angle change amount is Δx. Alternatively, the teacher data indicates the drinking score Dsc when the driver's alertness level is AL, the driver's vital signs are VT, and the vehicle speed variance is Δy. When the learning model LM' is provided with the alertness data, vital sign data, steering angle change amount Δx, and teacher data, it learns the drinking score Dsc corresponding to the alertness data, vital sign data, and steering angle change amount Δx. Furthermore, when the learning model LM' is given the alertness data, vital data, the variance Δy of the vehicle speed, and the training data, it learns the drinking score Dsc corresponding to the alertness data, vital data, and the variance Δy of the vehicle speed.

[0060] The drinking score calculation unit 16a provides the learning model LM' with the wakefulness data, vital data, and the change in steering angle Δx. Alternatively, the drinking score calculation unit 16a provides the learning model LM' with the wakefulness data, vital data, and the variance Δx of the vehicle speed. During inference, when the learning model LM' is given the wakefulness data, vital data, and the change in steering angle Δx, it outputs a drinking score Dsc corresponding to the wakefulness data, vital data, and the change in steering angle Δx to the drinking score calculation unit 16a. During inference, when the learning model LM' is given the wakefulness data, vital data, and the variance Δy of the vehicle speed, it outputs a drinking score Dsc corresponding to the wakefulness data, vital data, and the variance Δy of the vehicle speed to the drinking score calculation unit 16a. The drinking score calculation unit 16a acquires the drinking score Dsc from the learning model LM' and outputs the drinking score Dsc to the drinking state determination processing unit 16b.

[0061] The alertness level AL indicated by the alertness data is likely to decrease as the driver drinks alcohol. The alertness level AL may decrease as the driver becomes more drowsy, even when the driver is not drinking alcohol. The heart rate included in the vital VT indicated by the vital data is likely to increase as the driver drinks alcohol. The heart rate included in the vital VT may decrease as the driver becomes more drowsy. The blood pressure included in the vital VT indicated by the vital data is likely to decrease as the driver drinks alcohol. The steering angle change amount Δx and the vehicle speed variance Δy are likely to increase as the driver drinks alcohol. The steering angle change amount Δx and the vehicle speed variance Δy are unlikely to increase as the driver becomes more drowsy. Therefore, when the driver drinks alcohol, there is a high probability that a decrease in alertness AL, an increase in heart rate, and an increase in the steering angle change amount Δx will occur simultaneously. When the driver drinks alcohol, there is a high probability that a decrease in alertness AL, a decrease in blood pressure, and an increase in the steering angle change amount Δx will occur simultaneously. Furthermore, when a driver drinks alcohol, there is a high probability that a decrease in alertness AL, an increase in heart rate, and an increase in vehicle speed variance Δy will occur simultaneously. When a driver drinks alcohol, there is a high probability that a decrease in alertness AL, a decrease in blood pressure, and an increase in vehicle speed variance Δy will occur simultaneously. On the other hand, when a driver has not drunk alcohol, even if the alertness AL decreases, there is a low probability that an increase in heart rate will occur, a low probability that a decrease in blood pressure will occur, an increase in the steering angle change Δx will occur, or an increase in vehicle speed variance Δy will occur. The alertness AL, heart rate, blood pressure, steering angle change Δx, and vehicle speed variance Δy are each hardly affected by, for example, sunlight shining on the vehicle or a decrease in brightness around the vehicle when the vehicle enters a tunnel.

[0062] Here, the drinking score calculation unit 16a calculates the drinking score Dsc using the learning model LM'. However, this is merely an example, and the drinking score calculation unit 16a may calculate the drinking score Dsc from the alertness data, vital sign data, and vehicle information by, for example, executing a random forest method or a logistic regression analysis.

[0063] The drinking level determination processing unit 16b acquires the drinking score Dsc from the drinking score calculation unit 16a. The drinking level determination processing unit 16b calculates the drinking score Dsc and the threshold value Th Ｄ Compare with the threshold value Th Ｄ may be stored in an internal memory of the drinking level determination processing unit 16b, or may be provided from outside the drinking level determination device 3. The drinking level determination processing unit 16b Ｄ If the drinking score Dsc is equal to or greater than the threshold value Th Ｄ If the alcohol intake level is less than the predetermined level, the drinking level determination processing unit 16b determines that the driver is not drunk. The drinking level determination processing unit 16b outputs the result of the drinking level determination to, for example, the DMS.

[0064] Here, the drinking level determination processing unit 16b calculates the drinking score Dsc and the threshold value Th Ｄ However, this is only an example, and two or more different thresholds Th Ｄ The drinking level determination processing unit 16b calculates the drinking score Dsc and two or more thresholds Th Ｄ In this case, the drinking level determination processing unit 16b can not only determine whether the driver is in a drinking state, but also determine the degree of drinking of the driver. For example, Ｄ１ , Th Ｄ２ (Th Ｄ１ <Th Ｄ２ ) is prepared, the drinking level determination processing unit 16b determines whether the drinking score Dsc is greater than or equal to the threshold value Th Ｄ２ If the drinking score Dsc is equal to or greater than the threshold value Th Ｄ１ is equal to or greater than the threshold value Th Ｄ２ If the drinking score Dsc is less than the threshold value Th Ｄ１ If the reading is less than this, it is determined that the person is not in a drunk state.

[0065] In the second embodiment described above, the drinking level determination device 3 shown in Fig. 5 is configured to include a vehicle information acquisition unit 15 that acquires, from the vehicle being driven by the driver, vehicle information indicating changes in the state of the vehicle as the driver operates the vehicle. The drinking level determination unit 16 of the drinking level determination device 3 shown in Fig. 5 determines the drinking level of the driver based on the alertness estimated by the alertness estimation unit 12, the vital signs estimated by the vital signs estimation unit 13, and the vehicle information acquired by the vehicle information acquisition unit 15. Therefore, the drinking level determination device 3 shown in Fig. 5 can determine whether or not the driver has been drinking more accurately than the drinking level determination device 3 shown in Fig. 1.

[0066] In the drinking level determination device 3 shown in FIG. 5 , the drinking score calculation unit 16a provides the steering angle change amount Δx to the learning model LM'. However, this is merely an example. If the steering angle change amount Δx is equal to or greater than the threshold Thx, the drinking score calculation unit 16a may provide, for example, "1" to the learning model LM' as the feature amount of the steering angle change amount Δx. If the steering angle change amount Δx is less than the threshold Thx, the drinking score calculation unit 16a may provide, for example, "0" to the learning model LM' as the feature amount of the steering angle change amount Δx. In this case, when the learning model LM' is provided with the alertness data, vital data, the feature amount of the steering angle change amount Δx, and teacher data during learning, it learns the drinking score Dsc corresponding to the alertness data, vital data, and the feature amount of the steering angle change amount Δx. Furthermore, the drinking score calculation unit 16a may assign, for example, "1" to the learning model LM' as the feature value of the vehicle speed variance Δy if the vehicle speed variance Δy is equal to or greater than the threshold Thy, and may assign, for example, "0" to the learning model LM' as the feature value of the vehicle speed variance Δy if the vehicle speed variance Δy is less than the threshold Thy. In this case, when the learning model LM' is given the alertness data, vital data, the feature value of the vehicle speed variance Δy, and teacher data during learning, it learns the drinking score Dsc corresponding to the alertness data, vital data, and the feature value of the vehicle speed variance Δy.

[0067] Third Embodiment In a third embodiment, a drinking level determination device 3 will be described that includes a drinking level determination processing unit 14c that normalizes the drinking score Dsc calculated by the drinking score calculation unit 14a.

[0068] FIG. 7 is a configuration diagram showing a drinking level determination device 3 according to embodiment 3. In FIG. 7, the same reference numerals as those in FIGS. 1 and 5 indicate the same or corresponding parts, and detailed description thereof will be omitted. FIG. 8 is a hardware configuration diagram showing the hardware of the drinking level determination device 3 according to embodiment 3. In FIG. 8, the same reference numerals as those in FIGS. 2 and 6 indicate the same or corresponding parts, and detailed description thereof will be omitted. The drinking level determination device 3 shown in FIG. 7 includes an image data acquisition unit 11, an alertness estimation unit 12, a vital sign estimation unit 13, and a drinking level determination unit 14.

[0069] The drinking level determination unit 14 is realized, for example, by the drinking level determination circuit 24 shown in FIG. 8 . The drinking level determination unit 14 includes a drinking score calculation unit 14a and a drinking level determination processing unit 14c. The drinking level determination processing unit 14c normalizes the drinking score calculated by the drinking score calculation unit 14a after a certain period of time has elapsed, based on the drinking score calculated by the drinking score calculation unit 14a before the driver starts driving the vehicle. The drinking level determination processing unit 14c compares the normalized drinking score with a threshold and determines whether the driver is in a drinking state based on the comparison result between the normalized drinking score and the threshold.

[0070] The drinking level determination device 3 shown in Fig. 7 includes a drinking level determination unit 14, and a drinking level determination processing unit 14c of the drinking level determination unit 14 normalizes the drinking score calculated by the drinking score calculation unit 14a. However, this is merely an example, and the drinking level determination processing unit 16b included in the drinking level determination unit 16 shown in Fig. 5 may also normalize the drinking score calculated by the drinking score calculation unit 16a, similar to the drinking level determination processing unit 14c. In this case, the drinking level determination processing unit 16b compares the normalized drinking score with a threshold and determines whether the driver is drunk based on the comparison result between the normalized drinking score and the threshold.

[0071] Next, the operation of the drinking level determination device 3 shown in Fig. 7 will be described. Except for the drinking level determination processing unit 14c, the drinking level determination device 3 shown in Fig. 1 is the same as that shown in Fig. 1. Therefore, only the operation of the drinking level determination processing unit 14c will be described here.

[0072] In the drinking level determination device 3 shown in Fig. 7, a sober driver is assumed to be in the vehicle, and the drinking level determination device 3 shown in Fig. 7 determines whether the driver has been drinking after getting in the vehicle. The drinking level determination device 3 shown in Fig. 7 normalizes the drinking score Dsc calculated by the drinking score calculation unit 16a to eliminate the influence of individual differences, even if the driver of the vehicle changes.

[0073] The drinking level determination processing unit 14c determines whether the driver has been drinking for a certain period T ０ Before the drinking score calculation unit 14a calculates the drinking score Dsc, the drinking score Dsc is calculated as Dsc ０ The drinking level determination processing unit 14c acquires the drinking score Dsc ０ is stored in the internal memory, for example.

[0074] The drinking level determination processing unit 14c determines whether the driver has been drinking for a certain period T ０ After the drinking score calculation unit 14a has elapsed, the drinking state determination processing unit 14c obtains the drinking score Dsc calculated by the drinking score calculation unit 14a. ０ The drinking score Dsc is normalized based on

[0075]

[0076] The drinking level determination processing unit 14c calculates the normalized drinking score Dsc′ and the threshold value Th Ｄ ' is compared with the threshold value Th Ｄ The drinking level determination processing unit 14c may store the normalized drinking score Dsc' in an internal memory of the drinking level determination processing unit 14c, or may be provided from outside the drinking level determination device 3. The drinking level determination processing unit 14c calculates the normalized drinking score Dsc' and the threshold value Th Ｄ 1. The drinking state determination processing unit 14c determines whether the driver is in a drunk state based on the comparison result with the drinking state determination processing unit 14b shown in FIG. Ｄ ' is prepared, and the normalized drinking score Dsc' and two or more thresholds Th Ｄ ' may be compared.

[0077] In the third embodiment described above, the drinking level determination device 3 shown in Fig. 7 is configured such that the drinking level determination processing unit 14c normalizes the drinking score calculated by the drinking score calculation unit 14a after a certain period of time has elapsed based on the drinking score calculated by the drinking score calculation unit 14a before the driver starts driving the vehicle, compares the normalized drinking score with a threshold, and determines whether the driver is drunk based on the comparison result between the normalized drinking score and the threshold. Therefore, like the drinking level determination device 3 shown in Fig. 1, the drinking level determination device 3 shown in Fig. 7 can reduce erroneous determinations of whether or not the driver has been drinking compared to the drunk driving prevention device disclosed in Patent Document 1. In addition, even if the driver of the vehicle changes, the influence of individual differences can be eliminated, thereby improving the accuracy of determining whether or not the driver has been drinking.

[0078] Fourth Embodiment In a fourth embodiment, a drinking level determination device 3 will be described that includes a drinking level determination processing unit 14e that normalizes the drinking score Dsc calculated by the drinking score calculation unit 14a based on the drinking score when the driver is not drinking.

[0079] Fig. 9 is a configuration diagram showing a drinking level determination device 3 according to embodiment 4. In Fig. 9, the same reference numerals as those in Figs. 1, 5, and 7 indicate the same or corresponding parts, and detailed description thereof will be omitted. Fig. 10 is a hardware configuration diagram showing the hardware of the drinking level determination device 3 according to embodiment 4. In Fig. 10, the same reference numerals as those in Figs. 2, 6, and 8 indicate the same or corresponding parts, and detailed description thereof will be omitted. The drinking level determination device 3 shown in Fig. 9 includes an image data acquisition unit 11, an alertness estimation unit 12, a vital sign estimation unit 13, and a drinking level determination unit 14.

[0080] The drinking level determination unit 14 is realized, for example, by a drinking level determination circuit 24 shown in Fig. 10. The drinking level determination unit 14 includes a drinking score calculation unit 14a, a drinking score storage unit 14d, and a drinking level determination processing unit 14e. The drinking score storage unit 14d stores image data showing a face image for each of a plurality of drivers, and the drinking score for each driver when they are not drinking.

[0081] The drinking level determination processing unit 14e acquires image data from the image data acquisition unit 11. The drinking level determination processing unit 14e compares a facial image represented by the acquired image data with facial images represented by multiple pieces of image data stored in the drinking level determination processing unit 14e. If there is an image among the facial images represented by the multiple pieces of image data stored in the drinking level determination processing unit 14e that matches the facial image represented by the acquired image data, the drinking level determination processing unit 14e acquires the drinking score of the driver corresponding to the matching image from the drinking level determination processing unit 14a. The drinking level determination processing unit 14e normalizes the drinking score calculated by the drinking score calculation unit 14a based on the acquired drinking score. The drinking level determination processing unit 14e compares the normalized drinking score with a threshold and determines whether the driver is drunk based on the comparison result between the normalized drinking score and the threshold.

[0082] The drinking level determination device 3 shown in Fig. 9 includes a drinking level determination unit 14, and a drinking level determination processing unit 14e of the drinking level determination unit 14 normalizes the drinking score calculated by the drinking score calculation unit 14a. However, this is merely an example, and the drinking level determination processing unit 16b included in the drinking level determination unit 16 shown in Fig. 5 may also normalize the drinking score calculated by the drinking score calculation unit 16a, similar to the drinking level determination processing unit 14e. In this case, the drinking level determination processing unit 16b compares the normalized drinking score with a threshold and determines whether the driver is drunk based on the comparison result between the normalized drinking score and the threshold.

[0083] Next, the operation of the drinking level determination device 3 shown in Fig. 9 will be described. Except for the drinking score storage unit 14d and the drinking level determination processing unit 14e, the drinking level determination device 3 shown in Fig. 1 is the same as that shown in Fig. 1. Therefore, only the operation of the drinking score storage unit 14d and the drinking level determination processing unit 14e will be described here.

[0084] The drinking score storage unit 14d stores the drinking score of the driver DR. １ , ..., DR Ｍ Image data G showing an image of the face of １ , ..., G ＭThe drinking score storage unit 14d stores the driver's DR when not drinking. ｍ Drinking score Dsc of (m = 1, ..., M) ｍ is stored.

[0085] The drinking level determination processing unit 14e acquires image data G from the image data acquisition unit 11. The drinking level determination processing unit 14e compares the face image represented by the image data G with the image data G stored in the drinking level determination processing unit 14e. １ , ..., G Ｍ The drinking level determination processing unit 14e compares the image data G １ , ..., G Ｍ If there is an image that matches the face image shown in the image data G among the face images shown in the image data G, the drinking level determination processing unit 14e calculates the drinking score Dsc of the driver corresponding to the matching image. ０ The matching image is not limited to an image in which the facial image exactly matches, but also includes an image in which the similarity of the facial image is equal to or greater than a threshold value. １ , ..., G Ｍ If there is no image that matches the image of the face indicated by the image data G, the drinking score Dsc ０ For example, M drinking scores Dsc １ ~Dsc Ｍ or the average of M drinking scores Dsc １ ~Dsc Ｍ Get the median value of .

[0086] The drinking level determination processing unit 14e calculates the drinking score Dsc as shown in Equation (5) or Equation (6). ０ The drinking level determination processing unit 14e normalizes the drinking score Dsc based on the normalized drinking score Dsc′ and the threshold value Th Ｄ The drinking level determination processing unit 14e compares the normalized drinking score Dsc' with the threshold value Th Ｄ1. The drinking state determination processing unit 14e determines whether the driver is drunk based on the comparison result with the drinking state determination processing unit 14b shown in FIG. Ｄ ' is prepared, and the normalized drinking score Dsc' and two or more thresholds Th Ｄ ' may be compared.

[0087] In the above-described fourth embodiment, the drinking level determination device 3 shown in Fig. 9 is configured so that the drinking level determination processing unit 14e normalizes the drinking score calculated by the drinking score calculation unit 14a based on the drinking score when the driver was not drinking, compares the normalized drinking score with a threshold, and determines whether the driver is drinking based on the comparison result between the normalized drinking score and the threshold. Therefore, like the drinking level determination device 3 shown in Fig. 1, the drinking level determination device 3 shown in Fig. 9 can reduce erroneous determinations of whether or not the driver has been drinking compared to the drunk driving prevention device disclosed in Patent Document 1, and can eliminate the influence of individual differences even when the driver of the vehicle changes, thereby improving the accuracy of determining whether or not the driver has been drinking.

[0088] Fifth Embodiment In a fifth embodiment, a drinking level determination system will be described in which an image data acquisition unit 11, an alertness level estimation unit 12, and a vital sign estimation unit 13 are each installed in a vehicle 4 driven by a driver, and a drinking level determination unit 14 is installed in a cloud server 5.

[0089] Fig. 11 is a configuration diagram showing a drinking level determination system according to embodiment 5. In Fig. 11, the same reference numerals as those in Fig. 1, Fig. 5, Fig. 7, and Fig. 9 indicate the same or corresponding parts, and detailed description thereof will be omitted.

[0090] 11 , a vehicle 4 includes an image data acquisition unit 11, an alertness level estimation unit 12, and a vital sign estimation unit 13. A cloud server 5 includes a drinking state determination unit 14. The cloud server 5 is connected to the vehicle 4 via a network 6. The cloud server 5 is a virtual server existing in a cloud environment on the network 6. The network 6 is realized by, for example, a telephone communication network or the Internet.

[0091] In the drinking level determination system shown in Fig. 11, the cloud server 5 includes a drinking level determination unit 14. However, this is merely an example, and the cloud server 5 may also include the drinking level determination unit 16 shown in Fig. 5. In this case, a vehicle information acquisition unit 15 is installed in the vehicle 4.

[0092] Next, the operation of the drinking level determination system shown in Fig. 11 will be described. The image data acquisition unit 11 acquires image data representing a face image from the imaging device 1. The image data acquisition unit 11 outputs the image data to each of the alertness estimation unit 12 and the vital sign estimation unit 13.

[0093] The alertness estimation unit 12 acquires image data from the image data acquisition unit 11. The alertness estimation unit 12 estimates the alertness AL of the driver based on the image data. The alertness estimation unit 12 outputs alertness data indicating the alertness AL of the driver to the drinking state determination unit 14 of the cloud server 5 via the network 6.

[0094] The vital sign estimation unit 13 acquires at least one of the image data acquired by the image data acquisition unit 11 and the sensing data output from the sensor 2. When the vital sign estimation unit 13 acquires image data from the image data acquisition unit 11, it estimates the driver's vital sign VT by analyzing the facial image represented by the image data. When the vital sign estimation unit 13 acquires sensing data from the sensor 2, it estimates the driver's vital sign VT based on the sensing data. The vital sign estimation unit 13 outputs vital sign data indicating the driver's vital sign VT to the drinking level determination unit 14 of the cloud server 5 via the network 6.

[0095] The drinking score calculation unit 14a of the drinking state determination unit 14 acquires the alertness data from the alertness estimation unit 12 via the network 6, and acquires the vital data from the vital sign estimation unit 13 via the network 6. The drinking score calculation unit 14a calculates a drinking score Dsc indicating the degree of drinking state of the driver from the alertness AL indicated by the alertness data and the vital sign VT indicated by the vital sign data. The drinking score calculation unit 14a outputs the drinking score Dsc to the drinking state determination processing unit 14b.

[0096] The drinking level determination processing unit 14b acquires the drinking score Dsc from the drinking score calculation unit 14a. The drinking level determination processing unit 14b calculates the drinking score Dsc and the threshold value Th Ｄ The drinking level determination processing unit 14b compares the drinking score Dsc with the threshold value Th Ｄ If the drinking score Dsc is equal to or greater than the threshold value Th Ｄ If the alcohol intake level is less than the predetermined level, the drinking level determination processing unit 14b determines that the driver is not drunk. The drinking level determination processing unit 14b outputs the result of the drinking level determination to, for example, the DMS via the network 6.

[0097] Here, the drinking level determination processing unit 14b calculates the drinking score Dsc and the threshold value Th Ｄ However, this is only an example, and two or more different thresholds Th Ｄ The drinking level determination processing unit 14b calculates the drinking score Dsc and two or more thresholds Th Ｄ In this case, the drinking level determination processing unit 14b can not only determine whether the driver is in a drinking state, but also determine the degree of drinking of the driver. For example, Ｄ１ , Th Ｄ２ (Th Ｄ１ <Th Ｄ２ ) is prepared, the drinking level determination processing unit 14b determines whether the drinking score Dsc is greater than or equal to the threshold value Th Ｄ２ If the drinking score Dsc is equal to or greater than the threshold value Th Ｄ１ is equal to or greater than the threshold value Th Ｄ２ If the drinking score Dsc is less than the threshold value Th Ｄ１ If the reading is less than this, it is determined that the person is not in a drunk state.

[0098] Here, the drinking level determination processing unit 14b calculates the drinking score Dsc and the threshold value Th Ｄ However, this is merely an example, and the drinking level determination processing unit 14b normalizes the drinking score Dsc and compares the normalized drinking score Dsc' with the threshold value Th.Ｄ ', and the normalized drinking score Dsc' is compared with the threshold Th Ｄ It may be possible to determine whether the driver is in a drunk state based on the comparison result with '. When normalizing the drinking score Dsc, the drinking state determination processing unit 14b collects the drinking scores Dsc of other drivers other than the driver in question. The drinking scores Dsc of other drivers are drinking scores when they are not in a drunk state, and the drinking state determination processing unit 14b collects the drinking scores Dsc of one or more other drivers as the drinking scores Dsc of other drivers. The drinking state determination processing unit 14b collects the drinking scores Dsc of one or more other drivers as the drinking scores Dsc of other drivers. ０ For example, the drinking state determination processing unit 14b calculates the average value of one or more collected drinking scores Dsc or the median value of one or more collected drinking scores Dsc as the drinking score Dsc as shown in Equation (5) or Equation (6). ０ The drinking score Dsc is normalized based on

[0099] In the above-described fifth embodiment, the drinking level determination system is configured such that the image data acquisition unit 11, the alertness estimation unit 12, and the vital sign estimation unit 13 are each installed in the vehicle 4 driven by the driver, and the drinking level determination unit 14 is installed in the cloud server 5. Therefore, similar to the drinking level determination device 3 shown in FIG. 1 , the drinking level determination system can reduce erroneous determinations of whether or not the driver has drunk alcohol more than the drunk driving prevention device disclosed in Patent Document 1, and can also simplify the configuration of the device installed in the vehicle 4 more than the drinking level determination device 3 shown in FIG. 1 .

[0100] 11 , the drinking level determination unit 14 is installed in the cloud server 5. However, this is merely an example, and the alertness estimation unit 12 and the drinking level determination unit 14 may each be installed in the cloud server 5.

[0101] In addition, the present disclosure allows for free combination of the respective embodiments, modification of any of the components of the respective embodiments, or omission of any of the components of the respective embodiments.

[0102] The disclosed technology can be applied to a drinking level determination device that determines the drinking level of a driver in the technical field of automobiles, for example, and has industrial applicability.

[0103] REFERENCE SIGNS LIST 1 Imaging device, 2 Sensor, 3 Drinking state determination device, 4 Vehicle, 5 Cloud server, 6 Network, 11 Image data acquisition unit, 12 Alertness estimation unit, 13 Vital sign estimation unit, 14 Drinking state determination unit, 14a Drinking score calculation unit, 14b, 14c Drinking state determination processing unit, 14d Drinking score storage unit, 14e Drinking state determination processing unit, 15 Vehicle information acquisition unit, 16 Drinking state determination unit, 16a Drinking score calculation unit, 16b Drinking state determination processing unit, 21 Image data acquisition circuit, 22 Alertness estimation circuit, 23 Vital sign estimation circuit, 24 Drinking state determination circuit, 25 Vehicle information acquisition circuit, 26 Drinking state determination circuit, 31 Memory, 32 Processor.

Claims

1. A drinking state determination device comprising: an image data acquisition unit that acquires image data showing an image of a driver's face from an imaging device that images the face of the driver; an alertness estimation unit that estimates the alertness of the driver based on the image data acquired by the image data acquisition unit; a vital sign estimation unit that estimates the vital signs of the driver based on either the image data acquired by the image data acquisition unit or sensing data from a sensor that senses the vital signs of the driver; and a drinking state determination unit that determines the drinking state of the driver based on the alertness estimated by the alertness estimation unit and the vital signs estimated by the vital sign estimation unit.

2. The drinking state determination device of claim 1, characterized in that the drinking state determination unit includes: a drinking score calculation unit that calculates a drinking score indicating the degree of drinking state of the driver from the alertness estimated by the alertness estimation unit and the vital signs estimated by the vital signs estimation unit; and a drinking state determination processing unit that compares the drinking score calculated by the drinking score calculation unit with a threshold value and determines whether or not the driver is in a drinking state based on the comparison result between the drinking score and the threshold value.

3. The drinking state determination device as described in claim 1, further comprising a vehicle information acquisition unit that acquires vehicle information from the vehicle being driven by the driver, the vehicle information indicating changes in the state of the vehicle associated with the driver's operation of the vehicle, and the drinking state determination unit determines the drinking state of the driver based on the alertness estimated by the alertness estimation unit, the vital signs estimated by the vital signs estimation unit, and the vehicle information acquired by the vehicle information acquisition unit.

4. The drinking state determination device of claim 3, characterized in that the drinking state determination unit includes: a drinking score calculation unit that calculates a drinking score indicating the degree of drinking state of the driver from the alertness estimated by the alertness estimation unit, the vital signs estimated by the vital signs estimation unit, and the vehicle information acquired by the vehicle information acquisition unit; and a drinking state determination processing unit that compares the drinking score calculated by the drinking score calculation unit with a threshold value and determines whether or not the driver is in a drinking state based on the comparison result between the drinking score and the threshold value.

5. The drinking state determination processing unit normalizes the drinking score calculated by the drinking score calculation unit after a certain period of time has elapsed based on the drinking score calculated by the drinking score calculation unit before the certain period of time has elapsed since the driver started driving the vehicle, compares the normalized drinking score with the threshold value, and determines whether the driver is in a drinking state based on the comparison result between the normalized drinking score and the threshold value. The drinking state determination device described in claim 2 or 4 is characterized in that:

6. The drinking level determination device of claim 2 or claim 4, characterized in that the drinking level determination processing unit normalizes the drinking score calculated by the drinking score calculation unit based on the drinking score when the driver is not drinking, compares the normalized drinking score with the threshold value, and determines whether the driver is drinking based on the comparison result between the normalized drinking score and the threshold value.

7. A drinking state determination system comprising: an image data acquisition unit that acquires image data showing an image of the driver's face from an imaging device that captures the face of the driver; an alertness estimation unit that estimates the driver's alertness based on the image data acquired by the image data acquisition unit; a vital sign estimation unit that estimates the driver's vital signs based on either the image data acquired by the image data acquisition unit or sensing data from a sensor that senses the driver's vital signs; and a drinking state determination unit that determines the driver's drinking state based on the alertness estimated by the alertness estimation unit and the vital signs estimated by the vital sign estimation unit, wherein each of the image data acquisition unit, the alertness estimation unit and the vital sign estimation unit are installed in a vehicle driven by the driver, and the drinking state determination unit is installed in a cloud server.

8. The drinking state determination system of claim 7, characterized in that the drinking state determination unit includes: a drinking score calculation unit that calculates a drinking score indicating the degree of drinking state of the driver from the alertness estimated by the alertness estimation unit and the vital signs estimated by the vital signs estimation unit; and a drinking state determination processing unit that compares the drinking score calculated by the drinking score calculation unit with a threshold value and determines whether or not the driver is in a drinking state based on the comparison result between the drinking score and the threshold value.

9. The drinking level determination system of claim 8, characterized in that the drinking level determination processing unit normalizes the drinking score calculated by the drinking score calculation unit based on the drinking score of a driver other than the driver when he or she is not drinking, compares the normalized drinking score with the threshold value, and determines whether or not the driver is drinking based on the comparison result between the normalized drinking score and the threshold value.

10. A drinking state determination method, in which an image data acquisition unit acquires image data showing an image of the driver's face from an imaging device that captures the face of the driver, an alertness estimation unit estimates the alertness of the driver based on the image data acquired by the image data acquisition unit, a vital sign estimation unit estimates the vital signs of the driver based on either the image data acquired by the image data acquisition unit or sensing data from a sensor that senses the vital signs of the driver, and a drinking state determination unit determines the drinking state of the driver based on the alertness estimated by the alertness estimation unit and the vital signs estimated by the vital sign estimation unit.

Images