Drunken state determination device, drunken state determination system, and drunken state determination method

Abstract

To enable reduction of erroneous determination concerning presence / absence of being drunken more than a drunk-driving prevention device disclosed in Patent Document 1.SOLUTION: There is constituted a drunken state determination device 3 comprising: an image data acquisition unit 11 for acquiring image data indicating an image of a face from an imaging device 1 for imaging the face of a driver; an awaking degree estimation unit 12 for estimating an awaking degree of the driver, based on the image data acquired by the image data acquisition unit 11; and a vital estimation unit 13 for estimating vital of the driver, based on either the image data acquired by the image data acquisition unit 11 or sensing data from a sensor 2 for sensing the vital of the driver. The drunken state determination device 3 further comprises a drunken state determination unit 14 for determining a drunken state of the driver based on the awaking degree estimated by the awaking degree estimation unit 12, and the vital estimated by the vital estimation unit 13.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a drinking level determination device, a drinking level determination system, and a drinking level determination method. [Background technology]

[0002] There is a drinking level determination device that determines the drinking level of a vehicle driver. As such a drinking level determination device, for example, Patent Document 1 discloses a drunk driving prevention device that includes a storage unit, an imaging unit, and a determination unit. The storage unit stores photographed data showing an image of the driver's face when the driver is sober. The photographed data shows an image of the driver's face taken when the driver is detected by an alcohol detector as not having consumed alcohol after getting into the vehicle and before starting the engine. The imaging unit captures an image of the face of a driver who has started driving a vehicle, and outputs imaging data showing an image of the driver's face to the determination unit. The determination unit compares the imaging data output from the imaging unit with the imaging data stored in the storage unit to determine whether or not the driver's facial color has changed. If there is a change in facial color, the determination unit determines that the driver has drunk alcohol, and if there is no change in facial color, the determination unit determines that the driver has not drunk alcohol. If the determination unit determines that the driver has drunk alcohol, the drunk driving prevention device, for example, performs a process of stopping the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2010-52635 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the drunk driving prevention device disclosed in Patent Document 1, the judgment unit judges whether the driver has drunk alcohol based on the change in facial color, and therefore there is a problem that the judgment unit may erroneously judge whether the driver has drunk alcohol. This is because the facial color of the driver may change due to factors other than drinking. Factors other than drinking that may change the facial color of the driver include, for example, sunlight shining into the vehicle or a decrease in brightness around the vehicle due to entering a tunnel. In such cases, the driver's drinking state may be erroneously judged.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a drinking state determination device that can reduce erroneous determination of whether or not a person has drunk alcohol more than the drunk driving prevention device disclosed in Patent Document 1. [Means for solving the problem]

[0006] The drinking state determination device according to the present disclosure includes an image data acquisition unit that acquires image data showing an image of the face from an imaging device that captures the face of the driver, an alertness estimation unit that estimates the alertness of the driver based on the image data acquired by the image data acquisition unit, and a vitals estimation unit that estimates the vitals of the driver based on either the image data acquired by the image data acquisition unit or sensing data from a sensor that senses the vitals of the driver. The drinking state determination device also includes a drinking state determination unit that determines the drinking state of the driver based on the alertness estimated by the alertness estimation unit and the vitals estimated by the vitals estimation unit. Effect of the Invention

[0007] According to the present disclosure, it is possible to reduce erroneous determination of whether or not a person has drunk alcohol more than the drunk driving prevention device disclosed in Patent Document 1. [Brief description of the drawings]

[0008] [Figure 1] 1 is a configuration diagram showing a drinking level determining device 3 according to a first embodiment. [Diagram 2]1 is a hardware configuration diagram showing hardware of a drinking level determining device 3 according to embodiment 1. FIG. [Diagram 3] FIG. 1 is a hardware configuration diagram of a computer in the case where the drinking level determining device 3 is realized by software, firmware, or the like. [Figure 4] 4 is a flowchart showing a drinking level determination method which is a processing procedure of the drinking level determination device 3. [Diagram 5] FIG. 11 is a configuration diagram showing a drinking level determining device 3 according to a second embodiment. [Figure 6] FIG. 11 is a hardware configuration diagram showing hardware of a drinking level determining device 3 according to a second embodiment. [Figure 7] FIG. 11 is a configuration diagram showing a drinking level determining device 3 according to a third embodiment. [Figure 8] FIG. 11 is a hardware configuration diagram showing hardware of a drinking level determining device 3 according to a third embodiment. [Figure 9] FIG. 11 is a configuration diagram showing a drinking level determining device 3 according to a fourth embodiment. [Figure 10] FIG. 11 is a hardware configuration diagram showing hardware of a drinking level determining device 3 according to embodiment 4. [Figure 11] FIG. 13 is a configuration diagram showing a drinking level determination system according to a fifth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] In order to describe the present disclosure in more detail, embodiments of the present disclosure will be described below with reference to the accompanying drawings.

[0010] Embodiment 1 FIG. 1 is a configuration diagram showing a drinking level determining device 3 according to the first embodiment. FIG. 2 is a hardware configuration diagram showing the hardware of the drinking level determining device 3 according to the first embodiment. The drinking level determining device 3 shown in FIG. 1 includes an image data acquiring section 11, an alertness level estimating section 12, a vital sign estimating section 13, and a drinking level determining section .

[0011] In FIG. 1, an imaging device 1 is installed, for example, on an instrument panel, an A-pillar, or a ceiling of a vehicle. The imaging device 1 captures an image of the driver's face, and outputs image data representing the image of the face to the drinking level determining device 3.

[0012] The sensor 2 is installed, for example, on a seat of the vehicle, an instrument panel of the vehicle, an A-pillar of the vehicle, or on the ceiling of the vehicle. The sensor 2 senses the driver's vital signs and outputs sensing data of the vital signs to the drinking level determining device 3. The driver's vital signs include, for example, the driver's heart rate or blood pressure.

[0013] The image data acquisition unit 11 is realized by, for example, an image data acquisition circuit 21 shown in FIG. The image data acquisition unit 11 acquires, from the imaging device 1, image data indicative of an image of a face. The image data acquisition unit 11 outputs the image data to each of the awakening level estimation unit 12 and the vital signs estimation unit 13.

[0014] The awakening level estimation unit 12 is realized by, for example, an awakening level estimation circuit 22 shown in FIG. The awakening level estimation unit 12 acquires image data from the image data acquisition unit 11 . The awakening level estimation unit 12 estimates the driver's awakening level based on the image data. The awakening level estimation unit 12 outputs awakening level data indicating the driver's awakening level to the drinking state determination unit 14.

[0015] The vital sign estimation unit 13 is realized by, for example, a vital sign estimation circuit 23 shown in FIG. The vital sign estimation unit 13 acquires at least one of the image data acquired by the image data acquisition unit 11 and the sensing data output from the sensor 2. When the vital sign estimation unit 13 acquires the image data from the image data acquisition unit 11, it estimates the vital signs of the driver based on the image data. Upon acquiring the sensing data from the sensor 2, the vital sign estimating unit 13 estimates the driver's vital signs based on the sensing data. When the vital sign estimating unit 13 acquires both the image data and the sensing data, it estimates the driver's vital signs based on the image data and the sensing data. The vital sign estimating unit 13 outputs vital sign data indicating the driver's vital signs to the drinking level determining unit 14.

[0016] The drinking level determination unit 14 is realized by, for example, a drinking level determination circuit 24 shown in FIG. The drinking level determination unit 14 includes a drinking score calculation unit 14a and a drinking level determination processing unit 14b. The drinking level determination unit 14 acquires the alertness data from the alertness estimation unit 12 and acquires the vital data from the vital sign estimation unit 13 . The drinking level determining unit 14 determines the drinking level of the driver based on the alertness indicated by the alertness data and the vital signs indicated by the vital sign data. The drinking level determination unit 14 outputs the result of the drinking level determination to, for example, a driver monitoring system (hereinafter referred to as "DMS") of the vehicle.

[0017] The drinking score calculation unit 14 a acquires the awakening level data from the awakening level estimation unit 12 and acquires the vital data from the vital sign estimation unit 13 . The drinking score calculation unit 14a calculates a drinking score indicating the degree of drinking of the driver from the alertness indicated by the alertness data and the vital signs indicated by the vital sign data. The drinking score calculation unit 14a outputs the drinking score to the drinking level determination processing unit 14b.

[0018] The drinking level determination processing unit 14b acquires the drinking score from the drinking score calculation unit 14a. The drinking level determination processing unit 14b compares the drinking score with a threshold value, and determines whether the driver is in a drinking state based on the comparison result between the drinking score and the threshold value.

[0019] In Fig. 1, it is assumed that each of the components of the drinking state determination device 3, that is, the image data acquisition unit 11, the alertness estimation unit 12, the vital sign estimation unit 13, and the drinking state determination unit 14, is realized by dedicated hardware as shown in Fig. 2. In other words, it is assumed that the drinking state determination device 3 is realized by an image data acquisition circuit 21, an alertness estimation circuit 22, a vital sign estimation circuit 23, and a drinking state determination circuit 24. Each of the image data acquisition circuit 21, the alertness estimation circuit 22, the vital signs estimation circuit 23 and the drinking state determination circuit 24 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a combination of these.

[0020] The components of the drinking state determination device 3 are not limited to those realized by dedicated hardware, and the drinking state determination device 3 may be realized by software, firmware, or a combination of software and firmware. The software or firmware is stored as a program in the memory of a computer. The computer means hardware that executes the program, and includes, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, a processor, or a DSP (Digital Signal Processor).

[0021] FIG. 3 is a hardware configuration diagram of a computer in the case where the drinking level determining device 3 is realized by software, firmware, or the like. When the drinking level determination device 3 is realized by software, firmware, or the like, a program for causing a computer to execute the respective processing procedures of the image data acquisition unit 11, the alertness estimation unit 12, the vital sign estimation unit 13, and the drinking level determination unit 14 is stored in a memory 31. Then, a processor 32 of the computer executes the program stored in the memory 31.

[0022] 2 shows an example in which each of the components of the drinking state determination device 3 is realized by dedicated hardware, while Fig. 3 shows an example in which the drinking state determination device 3 is realized by software, firmware, etc. However, this is merely an example, and some of the components in the drinking state determination device 3 may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, etc.

[0023] Next, the operation of the drinking level determining device 3 shown in FIG. 1 will be described. FIG. 4 is a flowchart showing a drinking level determination method which is a processing procedure of the drinking level determination device 3. The imaging device 1 captures an image of the driver's face, and outputs image data representing the image of the face to the drinking level determining device 3. The sensor 2 senses the driver's vital signs and outputs the sensed data of the vital signs to the drinking level determining device 3.

[0024] The image data acquisition unit 11 acquires image data indicating an image of a face from the imaging device 1 (step ST1 in FIG. 4). The image data acquisition unit 11 outputs the image data to each of the awakening level estimation unit 12 and the vital signs estimation unit 13.

[0025] The awakening level estimation unit 12 acquires image data from the image data acquisition unit 11 . The awakening level estimation unit 12 estimates the driver's awakening level AL based on the image data (step ST2 in FIG. 4). The awakening level estimation unit 12 outputs awakening level data indicating the driver's awakening level AL to the drinking state determination unit 14.

[0026] The wakefulness estimation process performed by the wakefulness estimation unit 12 will be described in detail below. The awakening level estimation unit 12 detects the eyes of the driver by analyzing the face image represented by the image data. The eye detection process itself is a known technique, and therefore a detailed description thereof will be omitted. The wakefulness estimation unit 12 measures the time Ta during which the driver's eyes are open and the time Tb during which the driver's eyes are closed during a certain period FI. The time Ta when the eyes are open is the time when the distance L between the upper eyelid and the lower eyelid exceeds the threshold value Th L The total time is equal to or greater than the time. The time Tb during which the eyes are closed is the time when the distance L between the upper eyelid and the lower eyelid exceeds a threshold value Th L is the total time that the Threshold Th L may be stored in an internal memory of the awakening level estimation unit 12, or may be provided from outside the drinking state determination device 3, for example. The distance L between the upper eyelid and the lower eyelid can be, for example, the distance of the longest line segment in the vertical direction of the face that is approximately perpendicular to the line segment connecting the inner corner and the outer corner of the eye. The fixed period FI is, for example, a period from the current time to a preset time ago.

[0027] Next, the awakening level estimation unit 12 calculates the ratio of the time Ta to the total time of the time Ta and the time Tb as the eye open rate R, as shown in the following formula (1).

[0028] TIFF2025043407000002.tif21166

[0029] The awakening level estimation unit 12 calculates the awakening level AL of the driver based on the eye opening rate R as shown in the following formula (2). AL = R × c (2) In equation (2), c is a constant.

[0030] Here, the awakening level estimation unit 12 calculates the awakening level AL assuming that the awakening level AL is proportional to the eye opening rate R. The awakening level AL may be any level as long as it increases with an increase in the eye opening rate R, and is not limited to being proportional to the eye opening rate R.

[0031] The vital sign estimation unit 13 acquires at least one of the image data acquired by the image data acquisition unit 11 and the sensing data output from the sensor 2. When the vital sign estimation unit 13 acquires the image data from the image data acquisition unit 11, it estimates the driver's vital signs VT by analyzing the face image shown in the image data (step ST3 in FIG. 4). The vital signs estimation process itself by analyzing the face image is a known technique, so a detailed description will be omitted. Upon acquiring the sensing data from the sensor 2, the vital sign estimating unit 13 estimates the driver's vital signs VT based on the sensing data.

[0032] When the vital sign estimation unit 13 acquires both image data and sensing data, it calculates the driver's vital sign VT, for example, as a weighted sum of the vital signs estimated from the image data and the vital signs estimated from the sensing data, or an average value of the vital signs estimated from the image data and the vital signs estimated from the sensing data. The vital sign estimating unit 13 outputs vital sign data indicating the driver's vital signs VT to the drinking level determining unit 14.

[0033] The drinking score calculation section 14 a of the drinking level determination section 14 acquires the awakening level data from the awakening level estimation section 12 and acquires the vital data from the vital sign estimation section 13 . The drinking score calculation unit 14a calculates a drinking score Dsc indicating the degree of drinking state of the driver from the awakening level AL indicated by the awakening level data and the vital sign VT indicated by the vital data (step ST4 in FIG. 4).

[0034] Hereinafter, the calculation process of the drinking score Dsc by the drinking score calculation unit 14a will be specifically described. The drinking score calculation unit 14a provides the wakefulness data and the vital data to the learning model LM. The learning model LM may be included in the drinking score calculation unit 14a, or may be provided outside the drinking score calculation unit 14a. The learning model LM is realized, for example, by a neural network. During learning, the learning model LM is given the alertness data, vital sign data, and teacher data. The teacher data indicates the drinking score Dsc when the driver's alertness is AL and the driver's vital signs are VT. When the learning model LM is given the alertness data, the vital data, and the teacher data, the learning model LM learns the drinking score Dsc corresponding to the alertness data and the vital data. During inference, when the awakening level data and the vital data are given, the learning model LM outputs the drinking score Dsc corresponding to the awakening level data and the vital data to the drinking score calculation unit 14a. The drinking score calculation unit 14a obtains the drinking score Dsc from the learning model LM, and outputs the drinking score Dsc to the drinking state determination processing unit 14b.

[0035] The alertness level AL indicated by the alertness data is likely to decrease as the driver drinks alcohol. The alertness level AL can also decrease as the driver becomes more sleepy, even when the driver is sober. The heart rate included in the vital VT as indicated by vital data is likely to increase as the driver drinks alcohol. The heart rate included in the vital VT may decrease as the driver becomes more drowsy. The blood pressure included in the vital VT indicated by the vital data is likely to decrease as the driver drinks alcohol. Therefore, when a driver drinks alcohol, there is a high probability that both a decrease in the level of alertness AL and an increase in heart rate will occur. In addition, when a driver drinks alcohol, there is a high probability that both a decrease in the level of alertness AL and a decrease in blood pressure will occur. On the other hand, when the driver is sober, even if the arousal level AL decreases, the probability of an increase in heart rate is low. Also, when the driver is sober, even if the arousal level AL decreases, the probability of a decrease in blood pressure is low. The alertness level AL, the heart rate, and the blood pressure are hardly affected by, for example, sunlight entering the vehicle or a decrease in brightness around the vehicle when the vehicle enters a tunnel.

[0036] Here, the drinking score calculation unit 14a calculates the drinking score Dsc by using the learning model LM. However, this is merely an example, and the drinking score calculation unit 14a may calculate the drinking score Dsc from the wakefulness data and the vital data by, for example, executing a random forest method or a logistic regression analysis.

[0037] The drinking level determination processing unit 14b obtains the drinking score Dsc from the drinking score calculation unit 14a. The drinking state determination processing unit 14b calculates the drinking score Dsc and the threshold value Th D Compare with the threshold value Th D may be stored in an internal memory of the drinking level determination processing unit 14b, or may be provided from outside the drinking level determination device 3, for example. The drinking level determination processing unit 14b is configured to determine whether the drinking score Dsc is a threshold value Th D If so (step ST5 in FIG. 4: YES), it is determined that the driver is in a drunk state (step ST6 in FIG. 4). The drinking level determination processing unit 14b is configured to determine whether the drinking score Dsc is a threshold value Th D If it is less than that (step ST5 in FIG. 4: NO), it is determined that the driver is not in a drunk state (step ST7 in FIG. 4). The drinking level determination processing unit 14b outputs the drinking level determination result to, for example, the DMS.

[0038] Here, the drinking level determination processing unit 14b calculates the drinking score Dsc and the threshold value Th D However, this is only one example, and two or more different thresholds ThD The drinking state determination processing unit 14b calculates the drinking score Dsc and two or more threshold values ​​Th D It is also possible to compare the above. In this case, the drinking level determination processing unit 14b can determine not only whether the driver is drunk or not, but also the degree of the driver's drinking. D1 ,Th D2 (Th D1 <Th D2 ), the drinking level determination processing unit 14b determines whether the drinking score Dsc is greater than or equal to the threshold value Th D2 If the drinking score Dsc is equal to or greater than the threshold value Th, the drinking state determination processing unit 14b determines that the driver is in an intoxicated state. D1 and the threshold value Th D2 If it is less than this, the driver is determined to be tipsy. The drinking level determination processing unit 14b is configured to determine whether the drinking score Dsc is a threshold value Th D1 If the amount is less than this, it is determined that the person is not drunk.

[0039] If the result of the drinking state determination indicates that the driver is drunk, the DMS performs, for example, stop control to stop the vehicle or display processing to display a message urging the driver to stop the vehicle. If the alcohol intake assessment result indicates that the driver is not drunk, the DMS allows the driver to continue driving the vehicle.

[0040] In the above-described first embodiment, the drinking state determination device 3 is configured to include an image data acquisition unit 11 that acquires image data showing an image of the face from an imaging device 1 that captures an image of the face of the driver, an alertness estimation unit 12 that estimates the alertness of the driver based on the image data acquired by the image data acquisition unit 11, and a vitals estimation unit 13 that estimates the vitals of the driver based on either the image data acquired by the image data acquisition unit 11 or sensing data from a sensor 2 that senses the vitals of the driver. The drinking state determination device 3 also includes a drinking state determination unit 14 that determines the drinking state of the driver based on the alertness estimated by the alertness estimation unit 12 and the vitals estimated by the vitals estimation unit 13. Therefore, the drinking state determination device 3 can reduce erroneous determination of the presence or absence of drinking compared to the drunk driving prevention device disclosed in Patent Document 1.

[0041] In the drinking state determination device 3 shown in FIG. 1, the wakefulness estimation unit 12 calculates the eye opening rate R, and calculates the wakefulness AL of the driver from the eye opening rate R. However, this is merely an example, and the wakefulness estimation unit 12 detects the number of times the driver yawns in a certain period FI, for example, by analyzing a face image shown in the image data. The yawn detection process itself is a known technique, and therefore a detailed description will be omitted. The wakefulness estimation unit 12 may calculate the wakefulness AL so that the wakefulness AL decreases as the number of yawns increases.

[0042] In the drinking state determination device 3 shown in FIG. 1, the drinking state determination unit 14 determines the drinking state of the driver based on the alertness estimated by the alertness estimation unit 12 and the vital signs estimated by the vital signs estimation unit 13. However, this is merely an example, and the drinking state determination unit 14 may determine the drinking state of the driver based on the alertness estimated by the alertness estimation unit 12, the vital signs estimated by the vital signs estimation unit 13, and a change in the face color of the driver. In addition, the drinking state determination unit 14 may determine the drinking state of the driver based on the alertness estimated by the alertness estimation unit 12, the vital signs estimated by the vital signs estimation unit 13, and the proportion of the driver's gaze fixed on the road ahead. If the driver is in a drunk state, the proportion of the driver's gaze fixed on the road ahead is likely to decrease.

[0043] Embodiment 2 In the second embodiment, a drinking level determining device 3 will be described that includes a vehicle information acquiring unit 15 that acquires vehicle information indicating the driving operation of the driver from the vehicle being driven by the driver.

[0044] Fig. 5 is a configuration diagram showing a drinking level determining device 3 according to embodiment 2. In Fig. 5, the same reference numerals as in Fig. 1 indicate the same or corresponding parts, and detailed description thereof will be omitted. Fig. 6 is a hardware configuration diagram showing the hardware of a drinking level determining device 3 according to embodiment 2. In Fig. 6, the same reference numerals as in Fig. 2 indicate the same or corresponding parts, and therefore detailed description thereof will be omitted. The drinking level determining device 3 shown in FIG. 5 includes an image data acquiring section 11, an alertness level estimating section 12, a vital sign estimating section 13, a vehicle information acquiring section 15, and a drinking level determining section 16.

[0045] The vehicle information acquisition unit 15 is realized by, for example, a vehicle information acquisition circuit 25 shown in FIG. The vehicle information acquisition unit 15 acquires vehicle information indicating changes in the state of the vehicle that accompany the vehicle operation by the driver from the vehicle that the driver is driving. Specifically, if the driver's vehicle operation is a steering operation, the vehicle information acquisition unit 15 acquires vehicle information indicating a change in steering angle accompanying the steering operation as the vehicle information. If the driver's vehicle operation is either a braking operation or an accelerator operation, the vehicle information acquisition unit 15 acquires vehicle information indicating a change in vehicle speed accompanying the braking operation as the vehicle information. The vehicle information acquisition unit 15 outputs the vehicle information to the drinking level determination unit 16 .

[0046] The drinking level determination unit 16 is realized by, for example, a drinking level determination circuit 26 shown in FIG. The drinking level determination unit 16 includes a drinking score calculation unit 16a and a drinking level determination processing unit 16b. The drinking level determination unit 16 acquires the alertness data from the alertness estimation unit 12 and the vital sign data from the vital sign estimation unit 13 . In addition, the drinking level determination unit 16 acquires vehicle information from the vehicle information acquisition unit 15. The drinking level determining unit 16 determines the drinking level of the driver based on the level of alertness indicated by the alertness data, the vital signs indicated by the vital data, and the vehicle information. The drinking level determination unit 16 outputs the result of the drinking level determination to, for example, the DMS of the vehicle.

[0047] The drinking score calculation unit 16 a acquires the awakening level data from the awakening level estimation unit 12 and the vital sign data from the vital sign estimation unit 13 . In addition, the drinking score calculation unit 16a acquires vehicle information from the vehicle information acquisition unit 15. The drinking score calculation unit 16a calculates a drinking score indicating the degree of drinking state of the driver from the alertness indicated by the alertness data, the vital signs indicated by the vital data, and the vehicle information. The drinking score calculation unit 16a outputs the drinking score to the drinking level determination processing unit 16b.

[0048] The drinking level determination processing unit 16b acquires the drinking score from the drinking score calculation unit 16a. The drinking level determination processing unit 16b compares the drinking score with a threshold value, and determines whether the driver is in a drinking state based on the comparison result between the drinking score and the threshold value.

[0049] In Fig. 5, it is assumed that each of the components of the drinking state determination device 3, that is, the image data acquisition unit 11, the alertness estimation unit 12, the vital sign estimation unit 13, the vehicle information acquisition unit 15, and the drinking state determination unit 16, is realized by dedicated hardware as shown in Fig. 6. In other words, it is assumed that the drinking state determination device 3 is realized by an image data acquisition circuit 21, an alertness estimation circuit 22, a vital sign estimation circuit 23, a vehicle information acquisition circuit 25, and a drinking state determination circuit 26. Each of the image data acquisition circuit 21, the alertness estimation circuit 22, the vital signs estimation circuit 23, the vehicle information acquisition circuit 25 and the drinking state determination circuit 26 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination of these.

[0050] The components of the drinking state determination device 3 are not limited to those realized by dedicated hardware, and the drinking state determination device 3 may be realized by software, firmware, or a combination of software and firmware. When the drinking level determination device 3 is realized by software, firmware, or the like, programs for causing a computer to execute the respective processing procedures of the image data acquisition unit 11, the alertness estimation unit 12, the vital signs estimation unit 13, the vehicle information acquisition unit 15, and the drinking level determination unit 16 are stored in a memory 31 shown in Fig. 3. Then, a processor 32 shown in Fig. 3 executes the programs stored in the memory 31.

[0051] 6 shows an example in which each of the components of the drinking state determination device 3 is realized by dedicated hardware, while Fig. 3 shows an example in which the drinking state determination device 3 is realized by software, firmware, etc. However, this is merely an example, and some of the components in the drinking state determination device 3 may be realized by dedicated hardware, and the remaining components may be realized by software, firmware, etc.

[0052] Next, the operation of the drinking level determination device 3 shown in Fig. 5 will be described. Other than the vehicle information acquisition unit 15 and the drinking level determination unit 16, the device is the same as the drinking level determination device 3 shown in Fig. 1. Therefore, only the operations of the vehicle information acquisition unit 15 and the drinking level determination unit 16 will be described here.

[0053] The vehicle information acquisition unit 15 acquires vehicle information indicating the driving operation of the vehicle by the driver from the vehicle being driven by the driver. If the driver's vehicle operation is a steering operation, the vehicle information acquisition unit 15 acquires vehicle information indicating a change in steering angle accompanying the steering operation as the vehicle information. If the driver's vehicle operation is either a brake operation or an accelerator operation, the vehicle information acquisition unit 15 acquires vehicle information indicating a change in vehicle speed accompanying the brake operation or the like as the vehicle information. The vehicle information acquisition unit 15 outputs the vehicle information to the drinking level determination unit 16 . When a driver has been drinking, the driver's operation of the vehicle tends to be more unstable than when the driver has not been drinking. Specifically, when a driver has been drinking, the amount of change in the steering angle per unit time tends to be large. Also, when a driver has been drinking, the amount of change in the vehicle speed per unit time tends to be large.

[0054] The drinking score calculation unit 16 a acquires the awakening level data from the awakening level estimation unit 12 and the vital sign data from the vital sign estimation unit 13 . The drinking score calculation unit 16a acquires vehicle information from the vehicle information acquisition unit 15. If the driver's vehicle operation is a steering operation, the drinking score calculation unit 16a calculates the amount of change in the steering angle Δx as shown in the following equation (3). In the example of equation (3), the change in steering angle Δx is the steering angle x at sampling time i i and the steering angle x at the sampling time (in) n hours before the sampling time i i-n The absolute value of the difference between and is divided by n hours.

[0055] TIFF2025043407000003.tif21166

[0056] If the driver's vehicle operation is either a brake operation or an accelerator operation, the drinking score calculation unit 16a calculates the vehicle speed y at the sampling time i as shown in the following equation (4). i The vehicle speed variance Δy is calculated using the average vehicle speed y bar from the sampling time (in) n hours before the sampling time i to the sampling time i. In the text of the specification, the letter "y" cannot be preceded by a "-" sign due to electronic filing, so it is written as "y bar".

[0057] TIFF2025043407000004.tif24166

[0058] Here, the drinking score calculation unit 16a calculates the vehicle speed variance Δy using the average vehicle speed y. However, this is merely an example, and the drinking score calculation unit 16a may calculate the vehicle speed variance Δy using, for example, the median of the vehicle speed from the sampling time (in) to the sampling time i instead of the average vehicle speed y.

[0059] A learning model LM' is provided inside the drinking score calculation unit 16a or outside the drinking score calculation unit 16a. The learning model LM' is realized by, for example, a neural network. The learning model LM' is provided with the awakening level data, the vital sign data, and the teacher data during learning. Also, at least one of the change amount Δx of the steering angle and the variance amount Δy of the vehicle speed is provided to the learning model LM' during learning. The teacher data indicates the drinking score Dsc when the driver's alertness is AL, the driver's vital signs are VT, and the change in steering angle is Δx. Alternatively, the teacher data indicates the drinking score Dsc when the driver's alertness is AL, the driver's vital signs are VT, and the variance of vehicle speed is Δy. When the awakening level data, vital data, the change amount Δx in the steering angle, and teacher data are given, the learning model LM' learns the drinking score Dsc corresponding to the awakening level data, vital data, and the change amount Δx in the steering angle. Furthermore, when the learning model LM' is given the alertness data, vital data, the variance Δy of the vehicle speed, and the teacher data, it learns the drinking score Dsc corresponding to the alertness data, vital data, and the variance Δy of the vehicle speed.

[0060] The drinking score calculation unit 16a provides the wakefulness data, the vital sign data, and the amount of change in steering angle Δx to the learning model LM′. Alternatively, the drinking score calculation unit 16a provides the awakening level data, the vital sign data, and the variance amount Δy of the vehicle speed to the learning model LM′. During inference, when the awakening level data, vital data, and the change in steering angle Δx are given, the learning model LM' outputs the drinking score Dsc corresponding to the awakening level data, vital data, and the change in steering angle Δx to the drinking score calculation unit 16a. During inference, when the awakening level data, vital data, and vehicle speed variance Δy are given, the learning model LM′ outputs the drinking score Dsc corresponding to the awakening level data, vital data, and vehicle speed variance Δy to the drinking score calculation unit 16a. The drinking score calculation unit 16a obtains the drinking score Dsc from the learning model LM', and outputs the drinking score Dsc to the drinking state determination processing unit 16b.

[0061] The alertness level AL indicated by the alertness data is likely to decrease as the driver drinks alcohol. The alertness level AL can also decrease as the driver becomes more sleepy, even when the driver is sober. The heart rate included in the vital VT as indicated by vital data is likely to increase as the driver drinks alcohol. The heart rate included in the vital VT may decrease as the driver becomes more drowsy. The blood pressure included in the vital VT indicated by the vital data is likely to decrease as the driver drinks alcohol. The steering angle change amount Δx and the vehicle speed variance amount Δy are likely to increase as the driver drinks more. The steering angle change amount Δx and the vehicle speed variance amount Δy are unlikely to increase as the driver becomes more drowsy. Therefore, when the driver drinks alcohol, there is a high probability that the decrease in the level of awakening AL, the increase in the heart rate, and the increase in the amount of change in the steering angle Δx will occur simultaneously. When the driver drinks alcohol, there is a high probability that the decrease in the level of awakening AL, the decrease in blood pressure, and the increase in the amount of change in the steering angle Δx will occur simultaneously. In addition, when the driver drinks alcohol, there is a high probability that a decrease in the level of alertness AL, an increase in the heart rate, and an increase in the variance amount Δy of the vehicle speed will occur simultaneously. When the driver drinks alcohol, there is a high probability that a decrease in the level of alertness AL, a decrease in blood pressure, and an increase in the variance amount Δy of the vehicle speed will occur simultaneously. On the other hand, when the driver has not consumed alcohol, even if the alertness level AL decreases, there is a low probability of an increase in heart rate, a decrease in blood pressure, an increase in the amount of change in steering angle Δx, or an increase in the amount of variance in vehicle speed Δy. The level of alertness AL, heart rate, blood pressure, change in steering angle Δx, and variance of vehicle speed Δy are hardly affected by, for example, sunlight entering the vehicle or a decrease in brightness around the vehicle when the vehicle enters a tunnel.

[0062] Here, the drinking score calculation unit 16a calculates the drinking score Dsc by using the learning model LM'. However, this is merely an example, and the drinking score calculation unit 16a may calculate the drinking score Dsc from the wakefulness data, the vital data, and the vehicle information by, for example, executing a random forest method or a logistic regression analysis.

[0063] The drinking level determination processing unit 16b obtains the drinking score Dsc from the drinking score calculation unit 16a. The drinking state determination processing unit 16b calculates the drinking score Dsc and the threshold value Th D Compare with the threshold value Th D may be stored in an internal memory of the drinking level determination processing unit 16b, or may be provided from outside the drinking level determination device 3, for example. The drinking level determination processing unit 16b is configured to determine whether the drinking score Dsc is a threshold value Th D If so, it is determined that the driver is drunk. The drinking level determination processing unit 16b is configured to determine whether the drinking score Dsc is a threshold value Th D If the alcohol level is less than this, it is determined that the driver is not drunk. The drinking level determination processing unit 16b outputs the drinking level determination result to, for example, the DMS.

[0064] Here, the drinking level determination processing unit 16b calculates the drinking score Dsc and the threshold value Th D However, this is only one example, and two or more different thresholds Th D The drinking state determination processing unit 16b calculates the drinking score Dsc and two or more threshold values ​​Th D It is also possible to compare the above. In this case, the drinking level determination processing unit 16b can determine not only whether the driver is drunk or not, but also the degree of the driver's drinking. D1 ,Th D2 (Th D1 <Th D2 ), the drinking level determination processing unit 16b determines whether the drinking score Dsc is greater than or equal to the threshold value Th D2If the drinking score Dsc is equal to or greater than the threshold value Th, the drinking state determination processing unit 16b determines that the driver is in an intoxicated state. D1 and the threshold value Th D2 If it is less than this, the driver is determined to be tipsy. The drinking level determination processing unit 16b is configured to determine whether the drinking score Dsc is a threshold value Th D1 If the amount is less than this, it is determined that the person is not drunk.

[0065] In the above-described second embodiment, the drinking level determination device 3 shown in Fig. 5 is configured to include a vehicle information acquisition unit 15 that acquires vehicle information indicating changes in the state of the vehicle associated with the operation of the vehicle by the driver from the vehicle driven by the driver. Also, the drinking level determination unit 16 of the drinking level determination device 3 shown in Fig. 5 determines the drinking level of the driver based on the alertness estimated by the alertness estimation unit 12, the vital signs estimated by the vital signs estimation unit 13, and the vehicle information acquired by the vehicle information acquisition unit 15. Therefore, the drinking level determination device 3 shown in Fig. 5 can determine the presence or absence of drinking with higher accuracy than the drinking level determination device 3 shown in Fig. 1.

[0066] 5, the drinking score calculation unit 16a provides the change amount Δx of the steering angle to the learning model LM'. However, this is merely an example, and the drinking score calculation unit 16a may provide, for example, "1" to the learning model LM' as the feature amount of the change amount Δx of the steering angle if the change amount Δx of the steering angle is equal to or greater than the threshold Thx, and may provide, for example, "0" to the learning model LM' as the feature amount of the change amount Δx of the steering angle if the change amount Δx of the steering angle is less than the threshold Thx. In this case, when the learning model LM' is given alertness data, vital data, feature values ​​of the change in steering angle Δx, and teacher data, it learns the drinking score Dsc corresponding to the alertness data, vital data, and feature values ​​of the change in steering angle Δx. In addition, the drinking score calculation unit 16a may be configured to give, for example, “1” to the learning model LM' as the feature of the vehicle speed variance Δy if the vehicle speed variance Δy is greater than or equal to the threshold Thy, and to give, for example, “0” to the learning model LM' as the feature of the vehicle speed variance Δy if the vehicle speed variance Δy is less than the threshold Thy. In this case, when the learning model LM' is given alertness data, vital data, feature values ​​of the vehicle speed variance Δy, and teacher data, it learns the drinking score Dsc corresponding to the alertness data, vital data, and feature values ​​of the vehicle speed variance Δy.

[0067] Embodiment 3 In the third embodiment, a drinking level determination device 3 including a drinking level determination processing unit 14c that normalizes the drinking score Dsc calculated by the drinking score calculation unit 14a will be described.

[0068] Fig. 7 is a configuration diagram showing a drinking level determining device 3 according to embodiment 3. In Fig. 7, the same reference numerals as in Fig. 1 and Fig. 5 indicate the same or corresponding parts, and detailed description thereof will be omitted. Fig. 8 is a hardware configuration diagram showing the hardware of a drinking level determining device 3 according to embodiment 3. In Fig. 8, the same reference numerals as in Fig. 2 and Fig. 6 indicate the same or corresponding parts, and therefore detailed description thereof will be omitted. The drinking level determining device 3 shown in FIG. 7 includes an image data acquiring section 11, an alertness level estimating section 12, a vital sign estimating section 13, and a drinking level determining section .

[0069] The drinking level determination unit 14 is realized by, for example, a drinking level determination circuit 24 shown in FIG. The drinking level determination unit 14 includes a drinking score calculation unit 14a and a drinking level determination processing unit 14c. The drinking state judgment processing unit 14c normalizes the drinking score calculated by the drinking score calculation unit 14a after a certain period of time has elapsed based on the drinking score calculated by the drinking score calculation unit 14a before the certain period of time has elapsed since the driver started driving the vehicle. The drinking level determination processing unit 14c compares the normalized drinking score with a threshold value, and determines whether the driver is in a drinking state based on the comparison result between the normalized drinking score and the threshold value.

[0070] The drinking state determination device 3 shown in Fig. 7 includes a drinking state determination unit 14, and a drinking state determination processing unit 14c of the drinking state determination unit 14 normalizes the drinking score calculated by the drinking score calculation unit 14a. However, this is merely an example, and a drinking state determination processing unit 16b included in the drinking state determination unit 16 shown in Fig. 5 may normalize the drinking score calculated by the drinking score calculation unit 16a, similar to the drinking state determination processing unit 14c. In this case, the drinking state determination processing unit 16b compares the normalized drinking score with a threshold value, and determines whether or not the driver is drunk based on the comparison result between the normalized drinking score and the threshold value.

[0071] Next, the operation of the drinking level determination device 3 shown in Fig. 7 will be described. Except for the drinking level determination processing unit 14c, the drinking level determination device 3 is the same as that shown in Fig. 1. Therefore, only the operation of the drinking level determination processing unit 14c will be described here.

[0072] In the drinking level determination device 3 shown in FIG. 7, it is assumed that a driver who has not drunk alcohol gets into a vehicle, and the drinking level determination device 3 shown in FIG. 7 determines whether the driver has drunk alcohol after getting into the vehicle. The drinking level determining device 3 shown in FIG. 7 normalizes the drinking score Dsc calculated by the drinking score calculation unit 16a in order to eliminate the influence of individual differences even when the driver of the vehicle changes.

[0073] The drinking state determination processing unit 14c detects the drinking state of the driver for a certain period T 0 Before the drinking score calculation unit 14a calculates the drinking score Dsc as Dsc 0 Obtain as. The drinking state determination processing unit 14c calculates the drinking score Dsc 0 is stored in, for example, an internal memory.

[0074] The drinking state determination processing unit 14c detects the drinking state of the driver for a certain period T 0 After the drinking score calculation unit 14a has elapsed, the drinking score Dsc calculated by the drinking score calculation unit 14a is obtained. The drinking level determination processing unit 14c calculates the drinking score Dsc as shown in the following formula (5) or the following formula (6): 0 Based on this, the drinking score Dsc is normalized.

[0075] TIFF2025043407000005.tif32166

[0076] The drinking state determination processing unit 14c calculates the normalized drinking score Dsc′ and the threshold value Th D ' is compared with the threshold value Th D The drinking level determination time ' may be stored in an internal memory of the drinking level determination processing unit 14c, for example, or may be provided from outside the drinking level determination device 3. The drinking state determination processing unit 14c calculates the normalized drinking score Dsc′ and the threshold value Th D Based on the result of the comparison with ', it is determined whether the driver is drunk or not. The drinking level determination processing unit 14c, like the drinking level determination processing unit 14b shown in FIG. D Prepare the normalized drinking score Dsc and two or more thresholds Th D ' may be compared.

[0077] In the above-described third embodiment, the drinking state determination processing unit 14c normalizes the drinking score calculated by the drinking score calculation unit 14a after a certain period of time has elapsed based on the drinking score calculated by the drinking score calculation unit 14a before a certain period of time has elapsed since the driver started driving the vehicle, compares the normalized drinking score with a threshold, and determines whether the driver is drunk or not based on the comparison result between the normalized drinking score and the threshold. Thus, the drinking state determination device 3 shown in FIG. 7 is configured to reduce erroneous determination of whether or not the driver has drunk alcohol compared to the drunk driving prevention device disclosed in Patent Document 1, similar to the drinking state determination device 3 shown in FIG. 1, and can eliminate the influence of individual differences even if the driver of the vehicle changes, thereby improving the accuracy of determining whether or not the driver has drunk alcohol.

[0078] Embodiment 4 In the fourth embodiment, a drinking level determination device 3 is described that includes a drinking level determination processing unit 14e that normalizes the drinking score Dsc calculated by the drinking score calculation unit 14a based on the drinking score when the driver was not drinking.

[0079] Fig. 9 is a configuration diagram showing a drinking level determining device 3 according to embodiment 4. In Fig. 9, the same reference numerals as those in Fig. 1, Fig. 5 and Fig. 7 indicate the same or corresponding parts, and therefore detailed description thereof will be omitted. Fig. 10 is a hardware configuration diagram showing the hardware of a drinking level determining device 3 according to embodiment 4. In Fig. 10, the same reference numerals as those in Fig. 2, Fig. 6, and Fig. 8 indicate the same or corresponding parts, and therefore detailed description thereof will be omitted. The drinking level determining device 3 shown in FIG. 9 includes an image data acquiring section 11, an alertness level estimating section 12, a vital sign estimating section 13, and a drinking level determining section .

[0080] The drinking level determination unit 14 is realized by, for example, a drinking level determination circuit 24 shown in FIG. The drinking level determination unit 14 includes a drinking score calculation unit 14a, a drinking score storage unit 14d, and a drinking level determination processing unit 14e. The drinking score storage unit 14d stores image data showing a face image for each of a plurality of drivers, and the drinking score for each driver when not drinking.

[0081] The drinking level determination processing unit 14 e acquires image data from the image data acquisition unit 11 . The drinking level determination processing unit 14e compares the face image indicated by the acquired image data with face images indicated by a plurality of image data stored in the drinking level determination processing unit 14e. If there is an image among the facial images represented by the multiple image data stored in the drinking state determination processing unit 14e that matches the facial image represented by the acquired image data, the drinking state determination processing unit 14e obtains the drinking score of the driver corresponding to the matching image from the drinking state determination processing unit 14e. The drinking level determination processing unit 14e normalizes the drinking score calculated by the drinking score calculation unit 14a based on the acquired drinking score. The drinking level determination processing unit 14e compares the normalized drinking score with a threshold value, and determines whether or not the driver is in a drinking state based on the comparison result between the normalized drinking score and the threshold value.

[0082] The drinking state determination device 3 shown in Fig. 9 includes a drinking state determination unit 14, and a drinking state determination processing unit 14e of the drinking state determination unit 14 normalizes the drinking score calculated by the drinking score calculation unit 14a. However, this is merely an example, and a drinking state determination processing unit 16b included in the drinking state determination unit 16 shown in Fig. 5 may normalize the drinking score calculated by the drinking score calculation unit 16a, similar to the drinking state determination processing unit 14e. In this case, the drinking state determination processing unit 16b compares the normalized drinking score with a threshold value, and determines whether or not the driver is drunk based on the comparison result between the normalized drinking score and the threshold value.

[0083] Next, the operation of the drinking level determination device 3 shown in Fig. 9 will be described. Other than the drinking score storage unit 14d and the drinking level determination processing unit 14e, the drinking level determination device 3 shown in Fig. 1 is the same as that of the drinking level determination device 3 shown in Fig. 1. Therefore, only the operation of the drinking score storage unit 14d and the drinking level determination processing unit 14e will be described here.

[0084] The drinking score storage unit 14d stores the driver DR 1 ,···,DR M Image data G showing an image of a face of 1 ,···,G M is stored. M is an integer equal to or greater than 2. In addition, the drinking score storage unit 14d stores the driver's DR when he / she is not drinking. m Drinking score Dsc for (m=1,...,M) m is stored.

[0085] The drinking level determination processing unit 14e acquires image data G from the image data acquisition unit 11. The drinking level determination processing unit 14e compares the face image represented by the image data G with the image data G stored in the drinking level determination processing unit 14e. 1 ,···,G M The facial image shown is compared with the facial image shown by the The drinking level determination processing unit 14e uses the image data G 1 ,···,G M If there is an image that matches the face image shown in the image data G among the face images shown in the image data G, the drinking level determination processing unit 14e outputs the drinking score Dsc of the driver corresponding to the matching image. 0 The matching images are not limited to images in which the facial image exactly matches, but also include images in which the similarity of the facial image is equal to or greater than a threshold value. The drinking level determination processing unit 14e uses the image data G 1 ,···,G M If there is no face image that matches the face image shown in the image data G, the drinking score Dsc 0 For example, M drinking scores Dsc 1 ~Dsc M or M drinking scores Dsc 1~Dsc M Get the median value of .

[0086] The drinking level determination processing unit 14e calculates the drinking score Dsc as shown in formula (5) or formula (6). 0 Based on this, the drinking score Dsc is normalized. The drinking state determination processing unit 14e calculates the normalized drinking score Dsc′ and the threshold value Th D ' and compare. The drinking state determination processing unit 14e calculates the normalized drinking score Dsc′ and the threshold value Th D Based on the result of the comparison with ', it is determined whether the driver is drunk or not. The drinking level determination processing unit 14e, like the drinking level determination processing unit 14b shown in FIG. D Prepare the normalized drinking score Dsc and two or more thresholds Th D ' may be compared.

[0087] In the above-described fourth embodiment, the drinking level determination device 3 shown in Fig. 9 is configured so that the drinking level determination processing unit 14e normalizes the drinking score calculated by the drinking score calculation unit 14a based on the drinking score when the driver is not drinking, compares the normalized drinking score with a threshold, and determines whether the driver is drinking based on the comparison result between the normalized drinking score and the threshold. Therefore, the drinking level determination device 3 shown in Fig. 9 can reduce erroneous determination of whether or not the driver has drunk alcohol, as with the drinking level determination device 3 shown in Fig. 1, compared to the drunk driving prevention device disclosed in Patent Document 1, and can eliminate the influence of individual differences and increase the accuracy of determination of whether or not the driver of the vehicle has drunk alcohol, even if the driver of the vehicle changes.

[0088] Embodiment 5. In embodiment 5, a drinking state determination system is described in which an image data acquisition unit 11, an alertness level estimation unit 12, and a vital sign estimation unit 13 are each installed in a vehicle 4 driven by a driver, and a drinking state determination unit 14 is installed in a cloud server 5.

[0089] Fig. 11 is a configuration diagram showing a drinking level determination system according to embodiment 5. In Fig. 11, the same reference numerals as those in Fig. 1, Fig. 5, Fig. 7 and Fig. 9 indicate the same or corresponding parts, and therefore detailed description thereof will be omitted.

[0090] In FIG. 11, the vehicle 4 includes an image data acquisition unit 11, an awakening level estimation unit 12, and a vital sign estimation unit 13. The cloud server 5 includes a drinking level determination unit 14. The cloud server 5 is connected to the vehicle 4 via a network 6 . Cloud server 5 is a virtual server that exists in a cloud environment on network 6 . The network 6 is realized, for example, by a telephone communication network or the Internet.

[0091] In the drinking level determination system shown in Fig. 11, the cloud server 5 includes a drinking level determination unit 14. However, this is merely an example, and the cloud server 5 may also include the drinking level determination unit 16 shown in Fig. 5. In this case, a vehicle information acquisition unit 15 is installed in the vehicle 4.

[0092] Next, the operation of the drinking level determining system shown in FIG. 11 will be described. The image data acquisition unit 11 acquires, from the imaging device 1, image data indicative of an image of a face. The image data acquisition unit 11 outputs the image data to each of the awakening level estimation unit 12 and the vital signs estimation unit 13.

[0093] The awakening level estimation unit 12 acquires image data from the image data acquisition unit 11 . The awakening level estimation unit 12 estimates the driver's awakening level AL based on the image data. The awakening level estimation unit 12 outputs the awakening level data indicating the driver's awakening level AL to the drinking level determination unit 14 of the cloud server 5 via the network 6.

[0094] The vital sign estimation unit 13 acquires at least one of the image data acquired by the image data acquisition unit 11 and the sensing data output from the sensor 2. When the vital sign estimation unit 13 acquires image data from the image data acquisition unit 11, it estimates the driver's vital signs VT by analyzing the face image represented by the image data. Upon acquiring the sensing data from the sensor 2, the vital sign estimating unit 13 estimates the driver's vital signs VT based on the sensing data. The vital sign estimation unit 13 outputs vital sign data indicating the driver's vital signs VT to the drinking level determination unit 14 of the cloud server 5 via the network 6.

[0095] The drinking score calculation section 14 a of the drinking state determination section 14 acquires the awakening level data from the awakening level estimation section 12 via the network 6 , and acquires the vital data from the vital sign estimation section 13 via the network 6 . The drinking score calculation unit 14a calculates a drinking score Dsc indicating the degree of drinking state of the driver from the awakening level AL indicated by the awakening level data and the vital sign VT indicated by the vital data. The drinking score calculation unit 14a outputs the drinking score Dsc to the drinking state determination processing unit 14b.

[0096] The drinking level determination processing unit 14b obtains the drinking score Dsc from the drinking score calculation unit 14a. The drinking state determination processing unit 14b calculates the drinking score Dsc and the threshold value Th D Compare with. The drinking level determination processing unit 14b is configured to determine whether the drinking score Dsc is a threshold value Th D If so, it is determined that the driver is drunk. The drinking level determination processing unit 14b is configured to determine whether the drinking score Dsc is a threshold value Th D If the alcohol level is less than this, it is determined that the driver is not drunk. The drinking level determination processing unit 14b outputs the drinking level determination result via the network 6 to, for example, the DMS.

[0097] Here, the drinking level determination processing unit 14b calculates the drinking score Dsc and the threshold value Th D However, this is only one example, and two or more different thresholds Th D The drinking state determination processing unit 14b calculates the drinking score Dsc and two or more threshold values ​​Th D It is also possible to compare the above. In this case, the drinking level determination processing unit 14b can determine not only whether the driver is drunk or not, but also the degree of the driver's drinking. D1 ,Th D2 (Th D1 <Th D2 ), the drinking level determination processing unit 14b determines whether the drinking score Dsc is greater than or equal to the threshold value Th D2 If the drinking score Dsc is equal to or greater than the threshold value Th, the drinking state determination processing unit 14b determines that the driver is in an intoxicated state. D1 and the threshold value Th D2 If it is less than this, the driver is determined to be tipsy. The drinking level determination processing unit 14b is configured to determine whether the drinking score Dsc is a threshold value Th D1 If the amount is less than this, it is determined that the person is not drunk.

[0098] Here, the drinking level determination processing unit 14b calculates the drinking score Dsc and the threshold value Th D However, this is merely an example, and the drinking level determination processing unit 14b normalizes the drinking score Dsc and compares the normalized drinking score Dsc' with the threshold value Th. D 'Compare the normalized drinking score Dsc' with the threshold value Th D Based on the result of the comparison with "Is it okay if the driver is drunk?", it may be determined whether the driver is drunk or not. When normalizing the drinking score Dsc, the drinking state judgment processing unit 14b collects the drinking scores Dsc of other drivers other than the driver in question. The drinking scores Dsc of other drivers are drinking scores when the driver is not in a drinking state, and the drinking state judgment processing unit 14b collects the drinking scores Dsc of one or more other drivers as the drinking scores Dsc of the other drivers. The drinking state determination processing unit 14b calculates the drinking score Dsc0 As the drinking score, for example, the average value of one or more collected drinking scores Dsc or the median value of one or more collected drinking scores Dsc is calculated. The drinking level determination processing unit 14b calculates the drinking score Dsc as shown in formula (5) or formula (6). 0 Based on this, the drinking score Dsc is normalized.

[0099] In the above-described fifth embodiment, the drinking level determination system is configured such that the image data acquisition unit 11, the alertness estimation unit 12, and the vital sign estimation unit 13 are each installed in the vehicle 4 driven by the driver, and the drinking level determination unit 14 is installed in the cloud server 5. Therefore, like the drinking level determination device 3 shown in FIG. 1, the drinking level determination system can reduce erroneous determination of whether or not the driver has drunk alcohol more than the drunk driving prevention device disclosed in Patent Document 1, and can simplify the configuration of the device installed in the vehicle 4 more than the drinking level determination device 3 shown in FIG. 1.

[0100] 11, the drinking level determination unit 14 is installed in the cloud server 5. However, this is merely an example, and each of the awakening level estimation unit 12 and the drinking level determination unit 14 may be installed in the cloud server 5.

[0101] In addition, the present disclosure allows free combination of the respective embodiments, modification of any of the components of each embodiment, or omission of any of the components of each embodiment. [Explanation of symbols]

[0102] 1 imaging device, 2 sensor, 3 drinking state determination device, 4 vehicle, 5 cloud server, 6 network, 11 image data acquisition unit, 12 alertness estimation unit, 13 vital sign estimation unit, 14 drinking state determination unit, 14a drinking score calculation unit, 14b, 14c drinking state determination processing unit, 14d drinking score storage unit, 14e drinking state determination processing unit, 15 vehicle information acquisition unit, 16 drinking state determination unit, 16a drinking score calculation unit, 16b drinking state determination processing unit, 21 image data acquisition circuit, 22 alertness estimation circuit, 23 vital sign estimation circuit, 24 drinking state determination circuit, 25 vehicle information acquisition circuit, 26 drinking state determination circuit, 31 memory, 32 processor.

Claims

1. an image data acquisition unit that acquires image data representing an image of a driver's face from an imaging device that captures an image of the face; an alertness estimation unit that estimates an alertness of the driver based on the image data acquired by the image data acquisition unit; a vital sign estimation unit that estimates the vital signs of the driver based on either image data acquired by the image data acquisition unit or sensing data from a sensor that senses the vital signs of the driver; a drinking state determination unit that determines a drinking state of the driver based on the level of alertness estimated by the alertness estimation unit and the vital signs estimated by the vital signs estimation unit; A drinking state determination device comprising:

2. The drinking state determination unit is a drinking score calculation unit that calculates a drinking score indicating a degree of drinking of the driver from the wakefulness estimated by the wakefulness estimation unit and the vital signs estimated by the vital signs estimation unit; The drinking state determination device according to claim 1, further comprising a drinking state determination processing unit that compares the drinking score calculated by the drinking score calculation unit with a threshold value and determines whether the driver is in a drinking state based on the comparison result between the drinking score and the threshold value.

3. a vehicle information acquisition unit that acquires vehicle information indicating a change in a state of the vehicle associated with an operation of the vehicle by the driver from the vehicle being driven by the driver; The drinking state determination unit is The drinking state determination device according to claim 1, characterized in that the drinking state of the driver is determined based on the level of alertness estimated by the alertness estimation unit, the vital signs estimated by the vital signs estimation unit, and the vehicle information acquired by the vehicle information acquisition unit.

4. The drinking state determination unit is a drinking score calculation unit that calculates a drinking score indicating a degree of drinking of the driver from the alertness estimated by the alertness estimation unit, the vital signs estimated by the vital signs estimation unit, and the vehicle information acquired by the vehicle information acquisition unit; The drinking state determination device according to claim 3, further comprising a drinking state determination processing unit that compares the drinking score calculated by the drinking score calculation unit with a threshold value and determines whether the driver is in a drinking state based on the comparison result between the drinking score and the threshold value.

5. The drinking state determination processing unit, The drinking state determination device according to claim 2 or claim 4, characterized in that the drinking score calculated by the drinking score calculation unit after a certain period of time has elapsed is normalized based on the drinking score calculated by the drinking score calculation unit before the certain period of time has elapsed since the driver started driving the vehicle, the normalized drinking score is compared with the threshold value, and whether or not the driver is in a drinking state is determined based on the comparison result between the normalized drinking score and the threshold value.

6. The drinking state determination processing unit, The drinking state determination device according to claim 2 or claim 4, characterized in that the drinking score calculated by the drinking score calculation unit is normalized based on the drinking score when the driver is not drinking, the normalized drinking score is compared with the threshold value, and whether or not the driver is drinking is determined based on the comparison result between the normalized drinking score and the threshold value.

7. an image data acquisition unit that acquires image data representing an image of a driver's face from an imaging device that captures an image of the face; an alertness estimation unit that estimates an alertness of the driver based on the image data acquired by the image data acquisition unit; a vital sign estimation unit that estimates the vital signs of the driver based on either image data acquired by the image data acquisition unit or sensing data from a sensor that senses the vital signs of the driver; a drinking state determination unit that determines a drinking state of the driver based on the level of alertness estimated by the alertness estimation unit and the vital signs estimated by the vital signs estimation unit, the image data acquisition unit, the awakening level estimation unit, and the vital sign estimation unit are each installed in a vehicle driven by the driver, A drinking level determination system, characterized in that the drinking level determination unit is installed on a cloud server.

8. The drinking state determination unit is a drinking score calculation unit that calculates a drinking score indicating a degree of drinking of the driver from the wakefulness estimated by the wakefulness estimation unit and the vital signs estimated by the vital signs estimation unit; The drinking state determination system according to claim 7, further comprising a drinking state determination processing unit that compares the drinking score calculated by the drinking score calculation unit with a threshold value and determines whether the driver is in a drinking state based on the comparison result between the drinking score and the threshold value.

9. The drinking state determination processing unit, The drinking state determination system of claim 8, characterized in that the drinking score calculated by the drinking score calculation unit is normalized based on the drinking score of a driver other than the driver when he or she is not drinking, the normalized drinking score is compared with the threshold value, and whether or not the driver is drinking is determined based on the comparison result between the normalized drinking score and the threshold value.

10. The image data acquisition unit acquires image data representing an image of the driver's face from an imaging device that captures an image of the driver's face, an alertness estimation unit that estimates an alertness level of the driver based on the image data acquired by the image data acquisition unit; a vital sign estimation unit estimating the vital signs of the driver based on either the image data acquired by the image data acquisition unit or sensing data from a sensor that senses the vital signs of the driver; A drinking state determination unit determines the drinking state of the driver based on the level of alertness estimated by the level of alertness estimation unit and the vital signs estimated by the vital signs estimation unit. Method for determining drinking status.

Images