Occupant condition detection device and occupant condition improvement system

Abstract

To provide an occupant condition detection device capable of reducing false detections.SOLUTION: An occupant condition detection device 1 comprises: a plurality of condition determination units 6 that detect mutually-different occupant conditions and levels of occupant conditions using occupant condition information; a propriety order recording unit 7 that records an order in which the plurality of condition determination units 6 operate; and a propriety order control unit 71 that determines the order in which the plurality of condition determination units 6 operate using the recorded order. When the predetermined condition determination unit 6 specified by the propriety order control unit 71 determines that the occupant condition information corresponds to an occupant condition that it can determine, the predetermined condition determination unit 6 determines the level of the occupant condition. When the predetermined condition determination unit 6 determines that the occupant condition information does not correspond to an occupant condition that it can determine, the next condition determination unit 6 in the order determines whether or not the occupant condition information corresponds to an occupant condition that it can determine.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present disclosure relates to an occupant state detection device that detects levels of various occupant states such as drowsiness or fatigue of an occupant riding in a vehicle, and an occupant state improvement system that improves the occupant state.

Background Art

[0002] The drowsiness detection device described in Patent Document 1 detects the level of the drowsiness state of an occupant based on the temperature difference between the temperature of the arteriovenous anastomosis part among the facial parts of the occupant and the temperature of a reference part other than the arteriovenous anastomosis part among the facial parts of the occupant. Specifically, when the temperature difference tends to increase with the passage of time, the drowsiness detection device determines that the level of the drowsiness state of the occupant is rising, and when the temperature difference tends to decrease with the passage of time, the drowsiness detection device determines that the level of the drowsiness state of the occupant is falling.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Incidentally, generally in an occupant state detection device, a state determination unit that determines the level of an occupant state such as drowsiness or fatigue of the occupant is set so as to be able to discriminate between the target state and the rest state regardless of whether its algorithm uses a rule-based or machine learning method. Therefore, for a drowsiness determination unit that detects the level of the occupant's drowsiness or a fatigue determination unit that detects the level of the occupant's fatigue, for example, when a signal representing an unlearned emotion different from drowsiness or fatigue, such as anger or irritation, is input, false detection may occur. For example, in the drowsiness detection device described in Patent Document 1, when an emotion such as anger or irritation enters the detection of the occupant's drowsiness, there is a concern that the level of the occupant's drowsiness may be falsely detected due to a temperature change caused by a state different from drowsiness. As a result, based on the false detection, there is a possibility that the occupant state improvement system may perform unnecessary or incorrect actuations on the occupant.

[0005] In view of the above points, an object of the present disclosure is to provide an occupant state detection device and an occupant state improvement system capable of reducing false detection.

Means for Solving the Problems

[0006] According to one aspect of the present disclosure, an occupant state detection device (1) detects an occupant state based on information acquired from an occupant information detection unit (4) that detects information of an occupant riding in a vehicle. An occupant information acquisition unit (5) that acquires occupant state information as information regarding the state of the occupant from the occupant information detection unit. A plurality of state determination units (6) that detect the occupant state and the level of the occupant state using the occupant state information, the plurality of state determination units being configured to be able to execute detection of different occupant states and levels of the occupant state. An execution / no-execution order recording unit (7) that records the order in which the plurality of state determination units operate. An execution / no-execution order control unit (71) that determines the order in which the plurality of state determination units operate using the order read from the execution / no-execution order recording unit. When the predetermined state determination unit designated by the yes / no sequence control unit determines that the occupant state information corresponds to an occupant state that the unit itself can determine, the predetermined state determination unit determines the level of the occupant state. When the predetermined state determination unit designated by the yes / no sequence control unit determines that the occupant state information does not correspond to an occupant state that the unit itself can determine, the next sequential state determination unit designated by the yes / no sequence control unit is configured to determine whether the occupant state information corresponds to an occupant state that the unit itself can determine.

[0007] According to this, before determining the level of the occupant state, the state determination unit performs a yes / no determination as to whether the occupant state information corresponds to an occupant state that the unit itself can determine. Therefore, when the state determination unit determines the level of the occupant state, it is possible to suppress the mixing of occupant state information different from the occupant state that the unit itself can determine. Therefore, this occupant state detection device can reduce the erroneous detection of the occupant state and the level of the occupant state. In addition, when the state determination unit determines that the occupant state information does not correspond to an occupant state that the unit itself can determine, the state determination unit does not determine the level of the occupant state and advances the process to the yes / no determination by the next sequential state determination unit. Therefore, in this occupant state detection device, unnecessary level determination execution in a plurality of state determination units is prevented, and the calculation load can be reduced.

[0008] According to another aspect of the present disclosure, an occupant state improvement system (2) includes the occupant state detection device (1) according to one aspect of the present disclosure described above, and an actuator (3) that is driven according to the occupant state and the level of the occupant state detected by the occupant state detection device and executes an operation to promote the improvement of the occupant's state. According to this, the occupant state improvement system according to another aspect of the present disclosure can also achieve the same effects as the occupant state detection device according to one aspect of the present disclosure. In addition, in this occupant state improvement system, the actuator is driven according to the occupant state and the level of the occupant state correctly detected by the state determination unit, so that the occupant state can be improved appropriately and effectively.

[0009] Note that the reference signs in parentheses attached to each component etc. indicate an example of the correspondence relationship between the component etc. and the specific components etc. described in the embodiments described later.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Mode for Carrying Out the Invention

[0011] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the following embodiments, parts that are identical or equivalent to each other are denoted by the same reference numerals, and the description thereof will be omitted.

[0012] (First Embodiment) The first embodiment will be described with reference to FIGS. 1 to 6. The occupant state detection device 1 according to the first embodiment is a device capable of detecting the levels of various occupant states such as drowsiness or fatigue of an occupant riding in a vehicle. Further, the occupant state improvement system 2 according to the first embodiment is a system for improving the occupant state detected by the occupant state detection device 1, and in addition to the occupant state detection device 1, includes an action 3 that executes an operation for promoting the improvement of the occupant state. In the present disclosure, the occupant includes the driver of the vehicle.

[0013] In the first embodiment, the occupant state detection device 1 and the occupant state improvement system 2 will be described as being mounted on a vehicle. However, the occupant state detection device 1 and the occupant state improvement system 2 are not limited thereto, and may be incorporated into a device that can be carried by an occupant, such as a smartphone, for example, or a part or all of them may be incorporated on a server that can communicate with the vehicle, and the server may be installed in an external facility of the vehicle.

[0014] First, the configurations of the occupant state detection device 1 and the occupant state improvement system 2 will be described. As shown in FIG. 1, the vehicle is equipped with an occupant information detection unit 4 that detects occupant information. The occupant information detection unit 4 includes, for example, an in-vehicle camera 41, a biological sensor 42, an operation signal output unit 43, and the like. The in-vehicle camera 41 is an imaging device that captures the occupant. The biological sensor 42 is a sensor that measures biological information such as the occupant's heartbeat by contact or non-contact means. The operation signal output unit 43 is a sensor that outputs a signal corresponding to the operation amount when the occupant operates the vehicle, such as a steering angle sensor. The signal output by the occupant information detection unit 4 is transmitted to the occupant state detection device 1.

[0015] The occupant state detection device 1 detects the occupant state based on the information acquired from the occupant information detection unit 4. The occupant state detection device 1 is mainly composed of a microcomputer having a processor that performs control processing and arithmetic processing, and a memory that stores programs, data, and the like. The processor is composed of a CPU or an MPU. The memory includes various non-transitory tangible storage media such as a ROM, a RAM, and a non-volatile rewritable memory. The occupant state detection device 1 functions as an occupant information acquisition unit 5, a plurality of state determination units 6, an affirmative / negative order recording unit 7, an affirmative / negative order control unit 71, and an action control unit 8 when the processor executes the program stored in the memory. Therefore, it can be said that the occupant state detection device 1 of the first embodiment includes an occupant information acquisition unit 5, a plurality of state determination units 6, an affirmative / negative order recording unit 7, an affirmative / negative order control unit 71, and an action control unit 8.

[0016] The occupant information acquisition unit 5 acquires occupant state information as information regarding the occupant state from the above-described occupant information detection unit 4. The occupant information acquisition unit 5 includes an image acquisition unit 51, a biological signal acquisition unit 52, an operation signal acquisition unit 53, and the like. The image acquisition unit 51 acquires, as an example of the occupant state information, the occupant's face image, face temperature, body movement, and the like based on the signal output from the in-vehicle camera 41. The biological signal acquisition unit 52 acquires, as an example of the occupant state information, the occupant's electrocardiogram waveform and the like based on the signal output from the biological sensor 42. The operation signal acquisition unit 53 acquires, as an example of the occupant state information, the amount of change in the steering angle and the like based on the signal output from the operation signal output unit 43.

[0017] The plurality of state determination units 6 detect the occupant state and its level using the occupant state information acquired by the occupant information acquisition unit 5. Each of the plurality of state determination units 6 is configured to be able to execute the detection of different occupant states and their levels. In FIG. 1, as an example of the plurality of state determination units 6, a drowsiness determination unit 6a that determines the drowsiness of the occupant and its level, and an irritation determination unit 6b that determines the irritation feeling of the occupant and its level are illustrated. Note that the plurality of state determination units 6 are not limited thereto, and may include, for example, a fatigue determination unit that determines the fatigue feeling of the occupant and its level, a discomfort determination unit that determines the discomfort feeling of the occupant and its level, and the like.

[0018] Each of the plurality of state determination units 6 includes a state applicability determination unit 61, a state level determination unit 62, and a state confidence determination unit 63. The state applicability determination unit 61 determines whether the occupant state information corresponds to an occupant state that it can determine. The state level determination unit 62 determines the level of the occupant state that it can determine. The state confidence determination unit 63 determines the confidence level of the state level based on the level of the occupant state determined by the state level determination unit 62.

[0019] As shown in FIG. 1, the drowsiness determination unit 6a includes a drowsiness applicability determination unit 61a as the state applicability determination unit 61, a drowsiness level determination unit 62a as the state level determination unit 62, and a drowsiness confidence determination unit 63a as the state confidence determination unit 63. The irritation determination unit 6b includes an irritation applicability determination unit 61b as the state applicability determination unit 61, an irritation level determination unit 62b as the state level determination unit 62, and an irritation confidence determination unit 63b as the state confidence determination unit 63.

[0020] The plurality of state determination units 6 can be configured with those used in general anomaly detection algorithms. In the present embodiment, the plurality of state determination units 6 are configured by, for example, autoencoders. The autoencoders are learning with a learning dataset including the target occupant state and the resting state. For example, the drowsiness determination unit 6a configured by an autoencoder learns with a learning dataset of only the drowsy and resting states, and other states such as irritation and fatigue are not included in the learning dataset. The state appropriateness determination unit 61 of the state determination unit 6 performs an appropriateness determination using a threshold on whether the value obtained by quantifying the above-mentioned occupant state information as an input signal is within the range of the learning dataset.

[0021] Figures 2 and 3 are explanatory diagrams created by the inventors based on Russell's affect circumplex model. The area surrounded by the square in Figure 2 shows the learning dataset for drowsiness appropriateness determination, and the scale within the area surrounded by the square indicates the level of drowsiness. The area surrounded by the square in Figure 3 shows the learning dataset for irritation appropriateness determination, and the scale within the area surrounded by the square indicates the level of irritation. The occupant state information is quantified as an input signal and corresponds to one of the points (not shown) within the circles in Figures 2 and 3. For example, information such as the very small opening degree of the occupant's eyes in a face image or large variation in the steering angle corresponds to a position close to the circumference (i.e., the outer circumference) within the learning dataset for drowsiness appropriateness determination shown in Figure 2. Also, for example, information such as the appearance of wrinkles between the eyebrows in a face image corresponds to a position close to the circumference within the learning dataset for irritation appropriateness determination shown in Figure 3. Note that the level of the state is smaller the closer it is to the center of the circle (i.e., the resting state), and higher the closer it is to the circumference. In this way, each state determination unit 6 can perform an appropriateness determination on whether the occupant state information corresponds to the learning dataset it has learned, and can perform a level determination of the occupant state within the learning dataset.

[0022] As shown in FIG. 1, the acceptance / rejection order recording unit 7 records the order in which the plurality of state determination units 6 operate. The order is set such that the acceptance / rejection determination is performed in descending order of the risk level during vehicle operation. For example, since drowsiness is considered to have a higher risk level than irritation during vehicle operation, the order is set such that the acceptance / rejection determination is first performed by the drowsiness determination unit 6a and then by the irritation determination unit 6b. The acceptance / rejection order control unit 71 determines the order in which the plurality of state determination units 6 operate using the order read from the acceptance / rejection order recording unit 7, and activates the state determination units 6 in that order one by one.

[0023] The actuation control unit 8 controls the drive of the actuations 3 mounted on the vehicle according to the confidence level of the occupant state level determined by the state confidence level determination unit 63. Specifically, the actuation control unit 8 of the first embodiment includes a wakefulness control unit 8a and an irritation mitigation control unit 8b. The wakefulness control unit 8a controls the drive of the wakefulness output unit 3a mounted on the vehicle according to the drowsiness level confidence determined by the drowsiness confidence determination unit 63a. The irritation mitigation control unit 8b controls the drive of the irritation mitigation output unit 3b mounted on the vehicle according to the irritation level confidence determined by the irritation confidence determination unit 63b.

[0024] The wakefulness output unit 3a and the irritation mitigation output unit 3b are examples of the plurality of actuations 3 mounted on the vehicle. The wakefulness output unit 3a is, for example, a speaker, buzzer, display that outputs voice, alarm, and image to prompt the occupant's attention respectively, or an air conditioner that blows cold air to wake up the occupant. The irritation mitigation output unit 3b is, for example, a speaker that plays music to relieve the occupant's tension and relax, or a massager.

[0025] Next, the occupant state determination process executed by the occupant state detection device 1 will be described with reference to the flowchart of FIG. 4. This process is repeatedly executed at a predetermined control cycle (for example, 1 minute) when the start switch of the vehicle is turned on. In the following description and drawings, the steps are simply denoted as "S".

[0026] In S10, the pass / fail order recording unit 7 operates. In S20, the pass / fail order control unit 71 reads from the pass / fail order recording unit 7 the order in which the plurality of state determination units 6 operate, and operates the state determination units 6 in that order. In the present embodiment, it is assumed that the order in which the plurality of state determination units 6 operate is recorded in the pass / fail order recording unit 7 in the order of the drowsiness determination unit 6a first and then the irritation determination unit 6b. Therefore, the pass / fail order control unit 71 first operates the drowsiness determination unit 6a.

[0027] In S30, the drowsiness determination unit 6a first designated by the pass / fail order control unit 71 determines, by the drowsiness pass / fail determination unit 61a, whether the passenger state information acquired by the passenger information acquisition unit 5 corresponds to a drowsy state that it can determine. If, in S40, the drowsiness pass / fail determination unit 61a determines that it corresponds to a drowsy state, the process proceeds to S50.

[0028] In S50, the drowsiness level determination unit 62a of the drowsiness determination unit 6a determines the drowsiness level of the passenger using the passenger state information. For example, as shown in the graphs of FIGS. 5 and 6, the drowsiness level determination unit 62a calculates the estimated probability for each level regarding the drowsiness of the passenger using the passenger state information. On the horizontal axis of the graphs of FIGS. 5 and 6, level 0 indicates a non-drowsy state (i.e., a resting state), level 1 indicates a slightly drowsy state, and level 2 indicates a very drowsy state (i.e., a state of resisting drowsiness). In the example shown in the graph of FIG. 5, the estimated probability of level 0 is 0.3, the estimated probability of level 1 is 0.3, and the estimated probability of level 2 is 0.4. Also, in the example shown in the graph of FIG. 6, the estimated probability of level 0 is 0.1, the estimated probability of level 1 is 0.1, and the estimated probability of level 2 is 0.8.

[0029] Subsequently, in S60, the drowsiness confidence determination unit 63a calculates the confidence level based on the estimated probabilities of each level of the passenger's drowsiness calculated by the drowsiness level determination unit 62a. For example, the drowsiness confidence determination unit 63a calculates the confidence level as the odds ratio with respect to the other levels of the highest level. In the example shown in the graph of FIG. 5, the confidence level (i.e., the odds ratio) of level 2 is approximately 0.67. In the example shown in the graph of FIG. 6, the confidence level (i.e., the odds ratio) of level 2 is 4.

[0030] Next, at S70, the wakefulness control unit 8a controls the driving of a speaker, a buzzer, a display, an air conditioner, etc., which are the wakefulness output units 3a mounted on the vehicle, according to the confidence level of the drowsiness level, that is, according to the odds ratio. At S80, the wakefulness output unit 3a is driven according to the driving instruction from the wakefulness control unit 8a to wake up the occupant. For example, when the confidence level of the drowsiness level is relatively small, that is, when the odds ratio is relatively small, the wakefulness control unit 8a drives the speaker as the wakefulness output unit 3a to ask a question such as "Are you sleepy?". On the other hand, when the confidence level of the drowsiness level is relatively large, that is, when the odds ratio is relatively large, the wakefulness control unit 8a drives the buzzer or the speaker as the wakefulness output unit 3a to issue an alarm.

[0031] On the contrary, when the drowsiness determination unit 61a determines at S40 that it does not correspond to the drowsy state, the process proceeds to S90.

[0032] At S90, the irritation determination unit 6b designated as the next order of the drowsiness determination unit 6a by the presence / absence order control unit 71 determines, by the irritation presence / absence determination unit 61b, whether the occupant state information acquired by the occupant information acquisition unit 5 corresponds to an irritation state that it can determine. When the irritation presence / absence determination unit 61b determines at S100 that it corresponds to the irritation state, the process proceeds to S110.

[0033] At S110, the irritation level determination unit 62b of the irritation determination unit 6b calculates the estimated probability for each level regarding the irritation of the occupant using the occupant state information. Subsequently, at S120, the irritation confidence determination unit 63b calculates the confidence level based on the estimated probabilities of each level of the irritation of the occupant calculated by the irritation level determination unit 62b. For example, the confidence level is calculated by the odds ratio with respect to the other levels of the highest level.

[0034] Next, in S130, the irritation alleviation control unit 8b controls the driving of a speaker, a massage device, etc., which are the irritation alleviation output units 3b mounted on the vehicle, according to the confidence level of the irritation level, that is, according to the odds ratio. In S140, the irritation alleviation output unit 3b is driven according to the driving instruction from the irritation alleviation control unit 8b to alleviate the irritation of the occupant.

[0035] On the other hand, when the irritation presence / absence determination unit 61b determines in S100 that it does not correspond to the irritated state, the process proceeds to S150.

[0036] In S150, the state determination unit 6 designated in the next order of the irritation determination unit 6 by the presence / absence order control unit 71 determines whether the occupant state information acquired by the occupant information acquisition unit 5 corresponds to a state that the state determination unit 6 can determine, by the state presence / absence determination unit 61 included in the state determination unit 6. Note that the processes executed after S150 are substantially the same as those described in S30 to S140 above, so the description is omitted.

[0037] The occupant state detection device 1 and the occupant state improvement system 2 of the first embodiment described above have the following operational effects.

[0038] (1) In the occupant state detection device 1 of the first embodiment, when a predetermined state determination unit 6 (for example, the drowsiness determination unit 6a) designated by the presence / absence order control unit 71 determines that the occupant state information corresponds to an occupant state that the state determination unit 6 can determine, the predetermined state determination unit 6 determines the level of the occupant state. On the other hand, when the predetermined state determination unit 6 determines that the occupant state information does not correspond to an occupant state that the state determination unit 6 can determine, the next state determination unit 6 in order (for example, the irritation determination unit 6b) determines whether the occupant state information corresponds to an occupant state that the state determination unit 6 can determine.

[0039] According to this, before determining the level of the occupant condition, the state determination unit 6 determines whether the occupant condition information corresponds to an occupant condition that can be determined by the state determination unit 6. Therefore, when the state determination unit 6 determines the level of the occupant condition, it is possible to prevent occupant condition information that differs from the occupant condition that can be determined by the state determination unit 6 from being mixed in. Therefore, this occupant condition detection device 1 can reduce erroneous detection of the occupant condition and the occupant condition level. Furthermore, if the occupant state information does not correspond to an occupant state that can be determined by the state determination unit 6 itself, the state determination unit 6 does not determine the level of the occupant state, and proceeds to the next state determination unit 6 to determine whether the occupant state is applicable. Therefore, this occupant state detection device 1 can reduce the calculation load by preventing the execution of unnecessary level determinations in multiple state determination units 6.

[0040] (2) In the occupant state detection device 1 of the first embodiment, each of the multiple state determination units 6 has a state applicability determination unit 61 and a state level determination unit 62. The state applicability determination unit 61 determines whether the occupant state information corresponds to an occupant state that it can determine. The state level determination unit 62 determines the level of the occupant state that it can determine. Therefore, when the state applicability determination unit 61 (e.g., the drowsiness applicability determination unit 61a) of a predetermined state determination unit 6 (e.g., the drowsiness applicability determination unit 61a) determines that the occupant state information corresponds to an occupant state that it can determine, the state level determination unit 62 (e.g., the drowsiness level determination unit 62a) of the predetermined state determination unit 6 determines the level of the occupant state. On the other hand, if the state applicability determination unit 61 (e.g., drowsiness applicability determination unit 61a) of a predetermined state determination unit 6 (e.g., drowsiness applicability determination unit 6a) determines that the occupant state information does not correspond to an occupant state that it can determine, the state applicability determination unit 61 (e.g., irritation applicability determination unit 61b) of the next state determination unit 6 (e.g., irritation applicability determination unit 6b) determines whether the occupant state information corresponds to an occupant state that it can determine.

[0041] (3) The actuation control unit 8 provided in the occupant state detection device 1 of the first embodiment controls the drive of the actuation 3 installed in the vehicle depending on the degree of certainty of the level of the occupant state determined by the state certainty determination unit 63. According to this, the occupant state detection device 1 can drive the actuation 3 according to the occupant state correctly detected by the state determination unit 6 and the confidence level of the level of the occupant state, and can appropriately and effectively improve the occupant state.

[0042] (4) The occupant state improvement system 2 of the first embodiment includes the occupant state detection device 1 and the actuation 3. The actuation 3 is driven according to the occupant state detected by the occupant state detection device 1 and the level of the occupant state, and executes an operation to promote the improvement of the occupant's state. According to this, since the actuation 3 of this occupant state improvement system 2 is driven according to the occupant state correctly detected by the state determination unit 6 and the level of the occupant state, the occupant state can be appropriately and effectively improved.

[0043] (Second Embodiment) The second embodiment will be described. The second embodiment is a modification of a part of the configuration and control process of the occupant state detection device 1 with respect to the first embodiment, and since the other parts are the same as those of the first embodiment, only the parts different from the first embodiment will be described.

[0044] In addition, in FIGS. 7 and 8 referred to in the description of the second embodiment, specific states are not described for the plurality of state determination units 6, but the plurality of state determination units 6 are, for example, a drowsiness determination unit 6a, an irritation determination unit 6b, a fatigue determination unit, a discomfort determination unit, etc., in the same manner as in the first embodiment.

[0045] As shown in FIG. 7, the occupant state detection device 1 of the second embodiment includes, in addition to the occupant information acquisition unit 5, the plurality of state determination units 6, the acceptance / rejection order recording unit 7, the acceptance / rejection order control unit 71, and the state improvement control unit 81 as the actuation control unit 8 described in the first embodiment, a state recording unit 9. The state recording unit 9 records the occupant state determined to be applicable by the state applicability determination unit 61 of the state determination unit 6 and the level of the occupant state determined by the state level determination unit 62.

[0046] In the yes / no order recording unit 7, similar to the first embodiment, the order in which a plurality of state determination units 6 operate is recorded. The yes / no order control unit 71 reads out the order recorded in the yes / no order recording unit 7, and reads out the occupant state and its level recorded by the state recording unit 9 in the previous occupant state determination process. Then, when the level of the occupant state recorded by the state recording unit 9 in the previous occupant state determination process is higher than a predetermined threshold value, the state determination unit 6 capable of executing the determination of that occupant state is made to operate earlier than the other state determination units 6, that is, the order read from the yes / no order recording unit 7 is changed so as to operate first. The predetermined threshold value is set, for example, to be more than half of all levels (for example, three or more levels when the level is a total of five levels).

[0047] Next, the occupant state determination process executed by the occupant state detection device 1 of the second embodiment will be described with reference to the flowchart of FIG. 8. This process is repeatedly executed at a predetermined control cycle (for example, one minute) when the start switch of the vehicle is turned on.

[0048] In S200, the yes / no order recording unit 7 operates. In S210, the yes / no order control unit 71 reads out the order in which a plurality of state determination units 6 operate from the yes / no order recording unit 7, and further reads out the occupant state and its level recorded by the state recording unit 9 from the state recording unit 9 in the previous occupant state determination process as described in S310. Then, when the level of the occupant state recorded by the state recording unit 9 in the previous occupant state determination process is higher than a predetermined threshold value, the state determination unit 6 capable of executing the determination of that occupant state is made to operate earlier than the other state determination units 6, and the order read from the yes / no order recording unit 7 is changed. Specifically, in that case, the yes / no order control unit 71 sets the state determination unit 6 capable of executing the determination of the occupant state recorded by the state recording unit 9 in the previous occupant state determination process at the beginning of the order table. On the other hand, when the level of the occupant state recorded by the state recording unit 9 in the previous occupant state determination process is lower than the predetermined threshold value, the yes / no order control unit 71 does not change the order read from the yes / no order recording unit 7 and maintains the order table as it is.

[0049] Next, the state determination unit 6 set at the very beginning of the order table is designated in S220. In S230, the state compliance determination unit 61 of the designated state determination unit 6 determines whether the passenger state information acquired by the passenger information acquisition unit 5 corresponds to a state that it can determine. If, in S240, the state compliance determination unit 61 determines that it corresponds to a state that it can determine, the process proceeds to S250.

[0050] In S250, the state level determination unit 62 of the state determination unit 6 determines the state level of the passenger using the passenger state information. When the process of S250 is executed, in S310, the state recording unit 9 records the passenger state determined to be applicable by the state compliance determination unit 61 in S240 and the level of the passenger state determined by the state level determination unit 62 in S250.

[0051] In S260 following S250, the state confidence determination unit 63 calculates the confidence level based on the estimated probabilities of the respective levels of the passenger state calculated by the state level determination unit 62. For example, the confidence level is calculated by the odds ratio with respect to the other levels of the highest level.

[0052] Next, in S270, when the odds ratio is equal to or greater than a predetermined threshold, the state improvement control unit 81 controls the drive of the state improvement output unit 31 (i.e., actuation 3) mounted on the vehicle according to the odds ratio. In S280, the state improvement output unit 31 is driven by a drive instruction from the state improvement control unit 81.

[0053] On the other hand, if, in S240 above, the state compliance determination unit 61 determines that it does not correspond to a state that it can determine, the process proceeds to S290 to determine whether it is the last state in the order table. In the determination of S290, if it is determined to be the last state, it proceeds to S300 to determine a resting state, proceeds to S310, records it in the state recording unit 9, and exits the control loop. On the other hand, if, in the determination of S290, it is determined that it is not the last state, in order to proceed to the next state determination, the counter of the order table is incremented in S320. Thereafter, the process proceeds to S220. In S220, the state determination unit 6 set in the next order in the order table is specified. Thereafter, the same processing as that of S200 to S320 described above is executed.

[0054] The occupant state detection device 1 and the occupant state improvement system 2 of the second embodiment described above have the following operational effects.

[0055] (1) The occupant state detection device 1 of the second embodiment includes a state recording unit 9. The state recording unit 9 records the occupant state determined to be applicable by the state determination unit 6. Then, the yes / no order control unit 71 changes the order read from the yes / no order recording unit 7 so that the state determination unit 6 capable of executing the determination of the occupant state recorded by the state recording unit 9 in the previous occupant state determination process operates earlier than other state determination units 6. According to this, by changing the order read from the yes / no order recording unit 7 in consideration of the result of the yes / no determination of the occupant state executed last time, the possibility that the yes / no determination of the state determination unit 6 that makes the yes / no determination first among the plurality of state determination units 6 results in a corresponding result is increased. Therefore, this occupant state detection device 1 can reduce the number of yes / no determinations by the plurality of state determination units 6, and thus can reduce the calculation load.

[0056] (2) In the second embodiment, the state recording unit 9 records the occupant state determined to be applicable by the state determination unit 6 and its level. Then, when the level of the occupant state recorded by the state recording unit 9 is higher than a predetermined threshold, the yes / no order control unit 71 changes the order read from the yes / no order recording unit 7 so that the state determination unit 6 capable of executing the determination of the occupant state recorded by the state recording unit 9 in the previous occupant state determination process operates earlier than other state determination units 6. According to this, by changing the order read from the yes / no order recording unit 7 in consideration of the result of the yes / no determination of the occupant state executed last time and also the result of its level, the possibility that the yes / no determination of the state determination unit 6 that makes the yes / no determination first among the plurality of state determination units 6 results in a corresponding result is increased. Therefore, this occupant state detection device 1 can reduce the number of yes / no determinations by the plurality of state determination units 6, and thus can reduce the calculation load.

[0057] (Third Embodiment) The third embodiment will be described. The third embodiment is also a modification of part of the configuration and control process of the occupant state detection device 1 with respect to the first embodiment and the like. Since the other parts are the same as those of the first embodiment and the like, only the parts different from the first embodiment and the like will be described.

[0058] As shown in FIG. 9, the occupant state detection device 1 of the third embodiment has a corresponding confidence determination unit 64 in addition to the state compliance determination unit 61, the state level determination unit 62, and the state confidence determination unit 63 described in the first embodiment for each of a plurality of occupant state determination units 6. The corresponding confidence determination unit 64 calculates the probability of whether the occupant state information corresponds to an occupant state that it can determine.

[0059] In FIG. 9, as a plurality of state determination units 6, a drowsiness determination unit 6a and a fatigue determination unit 6c are illustrated. The drowsiness determination unit 6a determines the drowsiness of the occupant and its level, and the fatigue determination unit 6c determines the fatigue of the occupant and its level. The drowsiness determination unit 6a has a drowsiness corresponding confidence determination unit 64a as the corresponding confidence determination unit 64. The fatigue determination unit 6c has a fatigue corresponding confidence determination unit 64c as the corresponding confidence determination unit 64. Note that the plurality of state determination units 6 are not limited thereto, and may include, for example, an irritation determination unit 6b that detects the irritation of the occupant and its level, a discomfort determination unit that detects the discomfort of the occupant and its level, and the like. In that case, the irritation determination unit 6b has an irritation corresponding confidence determination unit as the corresponding confidence determination unit 64, and the discomfort determination unit has a discomfort corresponding confidence determination unit as the corresponding confidence determination unit 64.

[0060] Next, the occupant state determination process executed by the occupant state detection device 1 of the third embodiment will be described with reference to the flowchart of FIG. 10. This process is repeatedly executed at a predetermined control cycle (for example, 1 minute) when the vehicle start switch is turned on.

[0061] In S310, the acceptance / rejection order recording unit 7 operates. In S320, the acceptance / rejection order control unit 71 reads out from the acceptance / rejection order recording unit 7 the order in which a plurality of state determination units 6 operate, and operates the state determination units 6 in that order. In the third embodiment, it is assumed that the order in which the plurality of state determination units 6 operate is recorded in the acceptance / rejection order recording unit 7 in the order of the drowsiness determination unit 6a first and then the fatigue determination unit 6c. Therefore, the acceptance / rejection order control unit 71 first operates the drowsiness determination unit 6a.

[0062] In S330, the drowsiness determination unit 6a first designated by the acceptance / rejection order control unit 71 determines, by the drowsiness acceptance / rejection determination unit 61a, whether the passenger state information acquired by the passenger information acquisition unit 5 corresponds to a drowsiness state that it can determine. Subsequently, in S340, the drowsiness correspondence confidence determination unit 64a calculates the probability that the passenger state information corresponds to a drowsiness state that it can determine.

[0063] Specifically, as shown in the graphs of FIGS. 11 to 13, the drowsiness correspondence confidence determination unit 64a calculates the estimated probability that the passenger state information corresponds to the learning data set of the drowsiness state and the estimated probability that the passenger state information does not correspond to the learning data set of the drowsiness state, and calculates the odds ratio based on these estimated probabilities. Hereinafter, the estimated probability that the passenger state information corresponds to the learning data set of the drowsiness state is referred to as the "estimated probability of corresponding to drowsiness data", and the estimated probability that the passenger state information does not correspond to the learning data set of the drowsiness state is referred to as the "estimated probability of not corresponding to drowsiness data". Also, the odds ratio of the "estimated probability of corresponding to drowsiness data" is referred to as the "corresponding odds ratio". The corresponding odds ratio corresponds to the drowsiness correspondence confidence.

[0064] In the example shown in the graph of FIG. 11, since the estimated probability of corresponding to drowsiness data is 0.7 and the estimated probability of not corresponding to drowsiness data is 0.3, the corresponding odds ratio is approximately 2.3. In the example shown in the graph of FIG. 12, since the estimated probability of corresponding to drowsiness data is 0.5 and the estimated probability of not corresponding to drowsiness data is 0.5, the corresponding odds ratio is 1. In the example shown in the graph of FIG. 13, since the estimated probability of corresponding to drowsiness data is 0.3 and the estimated probability of not corresponding to drowsiness data is 0.7, the corresponding odds ratio is approximately 0.43.

[0065] Next, at S350, it is determined whether or not the corresponding odds ratio is greater than the first threshold value. If the corresponding odds ratio is greater than the first threshold value (for example, 2), the process proceeds to S360.

[0066] At S360, the drowsiness level determination unit 62a of the drowsiness determination unit 6a determines the drowsiness level of the occupant using the occupant state information. Subsequently, at S370, the drowsiness confidence determination unit 63a calculates the confidence level based on the estimated probabilities of each level of the occupant's drowsiness calculated by the drowsiness level determination unit 62a. For example, the confidence level is calculated using the odds ratio of the highest level with respect to the other levels. Next, at S380, when the odds ratio is equal to or greater than a predetermined threshold value, the wake-up control unit 8a controls the driving of the wake-up output unit 3a (that is, the actuation 3) mounted on the vehicle according to the odds ratio. At S390, the wake-up output unit 3a is driven according to the drive instruction from the wake-up control unit 8a.

[0067] On the other hand, if the corresponding odds ratio is less than or equal to the first threshold value at S350, the process proceeds to S400. At S400, it is determined whether or not the corresponding odds ratio is greater than the second threshold value (for example, 0.5). The second threshold value is set to a value lower than the first threshold value. If the corresponding odds ratio is less than or equal to the second threshold value, the process proceeds to S410. On the other hand, if the corresponding odds ratio is greater than the second threshold value, the process proceeds to both S410 and S360 described above. That is, when the corresponding odds ratio is between the first threshold value and the second threshold value at S400, the processes from S360 to S390 and the processes after S410 are executed in parallel.

[0068] At S410, the fatigue determination unit 6c designated as the next order by the yes / no order control unit 71 determines, by the fatigue yes / no determination unit 61c, whether the occupant state information acquired by the occupant information acquisition unit 5 corresponds to a fatigue state that it can determine. Subsequently, at S420, the fatigue correspondence confidence determination unit 64c calculates the probability of whether the occupant state information corresponds to a fatigue state that it can determine. Specifically, the fatigue correspondence confidence determination unit 64c calculates the estimated probability that the occupant state information corresponds to the learning data set of the fatigue state and the estimated probability that the occupant state information does not correspond to the learning data set of the fatigue state, and calculates the corresponding odds ratio based on these estimated probabilities. The corresponding odds ratio corresponds to the fatigue correspondence confidence.

[0069] Next, at S420, it is determined whether the corresponding odds ratio is greater than the first threshold. If the corresponding odds ratio is greater than the first threshold (for example, 2), the process proceeds to S440.

[0070] At S440, the fatigue level determination unit of the fatigue determination unit 6c determines the fatigue level of the occupant using the occupant state information. Subsequently, at S450, the fatigue confidence determination unit calculates the confidence based on the estimated probability of each level of the occupant's fatigue calculated by the fatigue level determination unit. For example, the confidence is calculated by the odds ratio of the other levels to the highest level. Next, at S460, when the odds ratio is equal to or greater than a predetermined threshold, the fatigue mitigation control unit 8c controls the drive of the fatigue mitigation output unit 3c (that is, the actuation 3) mounted on the vehicle according to the odds ratio. At S470, the fatigue mitigation output unit 3c is driven by the drive instruction from the fatigue mitigation control unit 8c.

[0071] On the other hand, when the corresponding odds ratio is equal to or less than the first threshold in S430, the process proceeds to S480. In S480, it is determined whether the corresponding odds ratio is greater than the second threshold. When the corresponding odds ratio is equal to or less than the second threshold, the process proceeds to S490. On the other hand, when the corresponding odds ratio is greater than the second threshold, the process proceeds to both S490 and S440 described above. That is, when the corresponding odds ratio is between the first threshold and the second threshold in S480, the processes from S440 to S470 and the processes after S490 are executed in parallel. Since the processes after S490 are substantially the same as those described in S330 to S480 above, the description thereof is omitted.

[0072] The occupant state detection device 1 and the occupant state improvement system 2 of the third embodiment described above have the following operational effects.

[0073] (1) Each of the plurality of state determination units 6 included in the occupant state detection device 1 of the third embodiment has a corresponding confidence determination unit 64. The corresponding confidence determination unit 64 calculates the probability that the occupant state information corresponds to an occupant state that it can determine. When the corresponding probability calculated by the corresponding confidence determination unit 64 of a predetermined state determination unit 6 designated by the yes / no order control unit 71 is higher than a predetermined threshold, the state level determination unit 62 of the predetermined state determination unit 6 determines the level of the occupant state. On the other hand, when the corresponding probability calculated by the corresponding confidence determination unit 64 of the predetermined state determination unit 6 is lower than the predetermined threshold, the state yes / no determination unit 61 of the next state determination unit 6 determines whether the occupant state information corresponds to an occupant state that it can determine. According to this, before the plurality of state determination units 6 determine the level of the occupant state, in addition to determining whether or not the occupant state information corresponds to an occupant state that the unit itself can determine, the calculation of the corresponding probability is performed. Then, when the corresponding probability is higher than a predetermined threshold value, the state determination unit 6 determines the level of the occupant state. When the corresponding probability is lower than the predetermined threshold value, the state determination unit 6 does not determine the level of the occupant state and proceeds with the process to the determination of whether or not it corresponds by the next state determination unit 6 in order. Therefore, when the state determination unit 6 determines the level of the occupant state, it is possible to prevent the mixing of occupant state information different from the occupant state that the unit itself can determine. Therefore, this occupant state detection device 1 can reduce the misdetection of the occupant state and its level. Also, when the corresponding probability is lower than the predetermined threshold value, the state determination unit 6 does not determine the level of the occupant state and proceeds with the process to the determination of whether or not it corresponds by the next state determination unit 6 in order. Therefore, in this occupant state detection device 1, the execution of unnecessary level determination in the plurality of state determination units 6 is prevented, so that the calculation load can be reduced.

[0074] (2) In the third embodiment, when the corresponding probability calculated by the state correspondence determination unit 61 of the predetermined state determination unit 6 designated by the correspondence order control unit 71 is higher than a predetermined first threshold value, the state level determination unit 62 of the predetermined state determination unit 6 determines the level of the occupant state. On the other hand, when the corresponding probability calculated by the state correspondence determination unit 61 of the predetermined state determination unit 6 is lower than a predetermined second threshold value, the state correspondence determination unit 61 of the next state determination unit 6 in order determines whether or not the occupant state information corresponds to an occupant state that the unit itself can determine. Further, when the corresponding probability calculated by the state correspondence determination unit 61 of the predetermined state determination unit 6 is between the first threshold value and the second threshold value, the state level determination unit 62 of the predetermined state determination unit 6 determines the level of the occupant state, and the state correspondence determination unit 61 of the next state determination unit 6 in order determines whether or not the occupant state information corresponds to an occupant state that the unit itself can determine. According to this, when the probability of correspondence of the state determination unit 6 is between the first threshold value and the second threshold value, that is, when it is unclear whether or not it corresponds to the occupant state that can be determined by itself, it determines the level of the occupant state by itself, and the next state determination unit 6 in the order also determines the determination of presence or absence and the level of the occupant state. Therefore, this occupant state detection device 1 can reduce the erroneous detection of the occupant state and its level by proceeding with the determination of the level of the occupant state in parallel by a plurality of state determination units 6 even when the determination of presence or absence is unclear in a predetermined state determination unit 6.

[0075] (Other embodiments) The present disclosure is not limited to the above-described embodiments, and can be appropriately changed within the scope described in the claims. In addition, each of the above embodiments and a part thereof are not unrelated to each other, and can be appropriately combined except when the combination is clearly impossible. Further, in each of the above embodiments, the elements constituting the embodiment are not necessarily essential except when it is clearly specified as essential or when it is considered to be clearly essential in principle. Further, in each of the above embodiments, when numerical values such as the number, numerical value, amount, range, etc. of the components of the embodiment are mentioned, they are not limited to the specific number except when it is clearly specified as essential or when it is clearly limited to a specific number in principle. Further, in each of the above embodiments, when referring to the shape, positional relationship, etc. of the components, etc., they are not limited to the shape, positional relationship, etc. except when it is clearly specified or when it is clearly limited to a specific shape, positional relationship, etc. in principle.

[0076] The control unit and its method described in the present disclosure may be implemented by a dedicated computer provided by configuring a processor and a memory programmed to execute one or more functions embodied by a computer program. Alternatively, the control unit and its method described in the present disclosure may be implemented by a dedicated computer provided by configuring a processor with one or more dedicated hardware logic circuits. Or, the control unit and its method described in the present disclosure may be implemented by one or more dedicated computers configured by a combination of a processor and a memory programmed to execute one or more functions and a processor configured by one or more hardware logic circuits. Also, the computer program may be stored in a computer-readable non-transitory tangible recording medium as instructions to be executed by a computer. The above memory is a non-transitory physical storage medium.

Explanation of Signs

[0077] 1 Occupant state detection device 4 Occupant information detection unit 5 Occupant information acquisition unit 6 State determination unit 7 Approval / disapproval order recording unit 71 Approval / disapproval order control unit

Claims

1. In an occupant state detection device (1) that detects an occupant state based on information acquired from an occupant information detection unit (4) that detects information on an occupant riding in a vehicle, an occupant information acquisition unit (5) that acquires occupant state information as information on the state of the occupant from the occupant information detection unit; a plurality of state determination units (6) that detect an occupant state and a level of the occupant state using the occupant state information, the plurality of state determination units being configured to be capable of respectively detecting different occupant states and levels of the occupant state; an existence / nonexistence order recording unit (7) that records the order in which the plurality of state determination units operate; an existence / nonexistence order control unit (71) that determines the order in which the plurality of state determination units operate using the order read from the existence / nonexistence order recording unit, and when a predetermined state determination unit designated by the existence / nonexistence order control unit determines that the occupant state information corresponds to an occupant state that the predetermined state determination unit can determine, the predetermined state determination unit determines the level of the occupant state; when a predetermined state determination unit designated by the existence / nonexistence order control unit determines that the occupant state information does not correspond to an occupant state that the predetermined state determination unit can determine, the next state determination unit designated by the existence / nonexistence order control unit determines whether or not the occupant state information corresponds to an occupant state that the next state determination unit can determine. An occupant state detection device configured as described above.

2. Each of the plurality of state determination units includes a state existence / nonexistence determination unit (61) that determines whether or not the occupant state information corresponds to an occupant state that the state determination unit can determine, and a state level determination unit (62) that determines the level of the occupant state that the state determination unit can determine. When the state existence / nonexistence determination unit included in a predetermined state determination unit designated by the existence / nonexistence order control unit determines that the occupant state information corresponds to an occupant state that the predetermined state determination unit can determine, the state level determination unit included in the predetermined state determination unit determines the level of the occupant state. When the state existence / nonexistence determination unit included in a predetermined state determination unit designated by the existence / nonexistence order control unit determines that the occupant state information does not correspond to an occupant state that the predetermined state determination unit can determine, the state existence / nonexistence determination unit included in the next state determination unit designated by the existence / nonexistence order control unit determines whether or not the occupant state information corresponds to an occupant state that the next state determination unit can determine. The occupant state detection device according to Claim 1, configured as described above.

3. The apparatus further includes a state recording unit (9) that records the occupant state determined to be applicable by the state determination unit. The presence / absence order control unit changes the order read from the presence / absence order recording unit so that the state determination unit capable of executing the determination of the occupant state recorded by the state recording unit in the previous occupant state determination process operates before the other state determination units. The occupant state detection device according to claim 1. **Claim 4** The state recording unit records the occupant state determined to be applicable by the state determination unit and the level of the occupant state determined to be applicable by the state determination unit. When the level of the occupant state recorded by the state recording unit in the previous occupant state determination process is higher than a predetermined threshold, the presence / absence order control unit changes the order read from the presence / absence order recording unit so that the state determination unit capable of executing the determination of that occupant state operates before the other state determination units. The occupant state detection device according to claim 3. **Claim 5** Each of the plurality of state determination units includes a state applicability determination unit that determines whether the occupant state information applies to the occupant state that the unit can determine, a corresponding confidence determination unit (64) that calculates the probability of whether the occupant state information applies to the occupant state that the unit can determine, and a state level determination unit that determines the level of the occupant state that the unit can determine. When the corresponding probability calculated by the corresponding confidence determination unit of a predetermined state determination unit designated by the presence / absence order control unit is higher than a predetermined threshold, the predetermined state determination unit determines the level of the occupant state. When the corresponding probability calculated by the corresponding confidence determination unit of a predetermined state determination unit designated by the presence / absence order control unit is lower than a predetermined threshold, the state determination unit in the next order designated by the presence / absence order control unit is configured to determine whether the occupant state information applies to the occupant state that the unit can determine. The occupant state detection device according to claim 1. **Claim 6** Each of the plurality of state determination units includes a state applicability determination unit that determines whether the occupant state information applies to the occupant state that the unit can determine, a corresponding confidence determination unit that calculates the probability of whether the occupant state information applies to the occupant state that the unit can determine, and a state level determination unit that determines the level of the occupant state that the unit can determine. When the probability calculated by the corresponding confidence determination unit of the predetermined state determination unit specified by the yes / no order control unit is higher than a predetermined first threshold value, the predetermined state determination unit determines the level of the occupant state. When the probability calculated by the corresponding confidence determination unit of the predetermined state determination unit specified by the yes / no order control unit is lower than a predetermined second threshold value set lower than the first threshold value, the state determination unit in the next order specified by the yes / no order control unit determines whether the occupant state information corresponds to an occupant state that the state determination unit can determine. The occupant state detection device according to claim 1, wherein when the probability calculated by the corresponding confidence determination unit of the predetermined state determination unit specified by the yes / no order control unit is between the first threshold value and the second threshold value, the state level determination unit of the predetermined state determination unit determines the level of the occupant state, and the state determination unit in the next order specified by the yes / no order control unit is configured to determine whether the occupant state information corresponds to an occupant state that the state determination unit can determine.

7. Each of the plurality of state determination units includes a state yes / no determination unit (61) that determines whether the occupant state information corresponds to an occupant state that the state determination unit can determine, a state level determination unit (62) that determines the level of the occupant state that the state determination unit can determine, and a state confidence determination unit (63) that determines the confidence level of the state level based on the level of the occupant state determined by the state level determination unit. The occupant state detection device according to claim 1.

8. The vehicle is equipped with an actuator (3) that performs an operation to promote the improvement of the occupant's state. The occupant state detection device according to claim 7, further comprising an actuator control unit (8) that controls the driving of the actuator according to the confidence level of the level of the occupant state determined by the state confidence determination unit.

9. The plurality of state determination units include at least one of a drowsiness determination unit (6a) that determines the drowsiness of the occupant and its level, an irritation determination unit (6b) that determines the irritation of the occupant and its level, and a fatigue determination unit (6c) that determines the fatigue of the occupant and its level. The occupant state detection device according to any one of claims 1 to 8.

10. The occupant state detection device (1) according to any one of claims 1 to 8, and An occupancy state improvement system comprising: an actuation (3) that is driven according to the occupancy state detected by the occupancy state detection device and the level of the occupancy state, and that executes an operation to promote improvement of the occupant's state.

Images