Crew member monitoring device

Abstract

To provide a crew member monitoring device capable of accurately detecting an abnormal state of a vehicle crew member.SOLUTION: A crew member monitoring device 1 includes an acquisition unit 11 for acquiring the eye opening distance of a crew member aboard a vehicle and a setting unit 12 for setting an eye opening reference value for determining the abnormal state of the crew member on the basis of the eye opening distance. The setting unit 12 sets a prescribed setting value for the initial value of the eye opening reference value, calculates an update value of the eye opening reference value on the basis of the eye opening distance acquired while the vehicle is travelling, and changes the eye opening reference value from the initial value to the update value when a prescribed condition is satisfied.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an occupant monitoring device. [Background technology]

[0002] Patent document 1 discloses an occupant monitoring device that, during an initialization process to determine the normal eye size of a vehicle driver, determines 100% eye opening degree based on a moving average of the eye opening distance, and then measures the driver's eye opening degree based on an image captured of the driver, using 100% eye opening degree as a standard. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-047086 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the configuration described in Patent Document 1, when the initialization process is performed using information including the driver's blinking and squinting, a value smaller than the driver's actual eye size is calculated as 100% eye opening degree, resulting in an error in the calculated 100% eye opening degree.

[0005] In a vehicle, when determining an abnormal state of a driver such as drowsiness based on the degree of eye opening of the driver, for example, a 20% degree of eye opening may be set based on a 100% degree of eye opening, and if the measured degree of eye opening is 20% or less, it may be determined that the driver has closed eyes. In this case, if there is an error in the 100% degree of eye opening, the 20% degree of eye opening set based on that degree will also have an error. As a result, closed eyes may not be detected, and the accuracy of detecting abnormal states of the driver such as drowsiness or microsleep may decrease.

[0006] An object of the present invention is to provide an occupant monitoring device capable of detecting an abnormal state of a vehicle occupant with high accuracy. [Means for solving the problem]

[0007] The occupant monitoring device of the present invention comprises an acquisition unit that acquires the eye-opening distance of an occupant riding in a vehicle, and a setting unit that sets an eye-opening reference value for determining an abnormal state of the occupant based on the eye-opening distance, and is characterized in that the setting unit sets a predetermined setting value as an initial value of the eye-opening reference value, calculates an update value of the eye-opening reference value based on the eye-opening distance acquired while the vehicle is traveling, and changes the eye-opening reference value from the initial value to the update value if a predetermined condition is satisfied. Effect of the Invention

[0008] The occupant monitoring device according to the present invention can accurately detect an abnormal state of a vehicle occupant. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing a schematic diagram of a vehicle equipped with an occupant monitoring device according to an embodiment. [Diagram 2] FIG. 2 is a block diagram showing a functional configuration of the occupant monitoring device. [Diagram 3] FIG. 3 is a diagram for explaining the eye opening distance at 100% eye opening degree. [Figure 4] FIG. 4 is a diagram showing the relationship between the degree of eye opening and the open / closed state of the eyes. [Diagram 5] FIG. 5 is a diagram showing a method for setting the eye opening distance for 100% eye opening. [Figure 6] FIG. 6 is a flow chart showing a flow for setting the eye opening distance for 100% eye opening. [Figure 7] FIG. 7 is a flowchart showing the calculation process of the update value. [Figure 8] FIG. 8 is a diagram showing the eye opening distance within a set time. [Figure 9] FIG. 9 is a diagram showing the distribution of eye opening distances calculated from the data shown in FIG. [Figure 10] FIG. 10 is a diagram for explaining a method of calculating an update value in the modified example. [Figure 11] FIG. 11 is a flow chart showing a calculation flow of the update value in the modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an occupant monitoring device according to an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to the embodiment. In addition, the components in the following embodiment include those that can be easily imagined by a person skilled in the art or those that are substantially the same.

[0011] An occupant monitoring device according to an embodiment will be described with reference to Figs. 1 to 9. Fig. 1 is a diagram showing a model of a vehicle equipped with an occupant monitoring device according to an embodiment. Fig. 2 is a block diagram showing a functional configuration of the occupant monitoring device. Fig. 3 is a diagram for explaining the eye opening distance at 100% eye opening. Fig. 4 is a diagram showing the relationship between the eye opening degree and the open / closed state of the eyes. Fig. 5 is a diagram showing a method for setting the eye opening distance at 100% eye opening. Fig. 6 is a flow chart showing a setting flow of the eye opening distance at 100% eye opening. Fig. 7 is a flow chart showing a calculation process of an update value. Fig. 8 is a diagram showing the eye opening distance within a set time. Fig. 9 is a diagram showing the distribution of the eye opening distance calculated from the data shown in Fig. 8.

[0012] As shown in FIG. 1, the occupant monitoring device 1 is provided in a vehicle 3 equipped with a camera 2. The occupant monitoring device 1 is a device that detects the blinking of an occupant riding in the vehicle 3. The camera 2 captures an image of the face of a driver 4 of the vehicle 3. The vehicle 3 is an automobile, and is driven by the driver 4. The camera 2 is disposed in front of the driver 4 in the cabin of the vehicle 3, and captures an image of the eyes 5 of the driver 4. The camera 2 can continuously capture an image of the face of the driver 4 at a predetermined frame rate. For example, the camera 2 can capture a video at 30 fbs.

[0013] 2, the occupant monitoring device 1 includes an acquisition unit 11 and a setting unit 12. The occupant monitoring device 1 is a computer having an arithmetic circuit, a memory, and an input / output interface. The occupant monitoring device 1 is configured as, for example, an electronic control unit (ECU).

[0014] The acquisition unit 11 acquires the eye opening distance of the driver 4 based on the image acquired from the camera 2. The eye opening distance is an index showing the open / closed state of the eyes 5, and is the distance between the upper eyelid and the lower eyelid. For example, the distance between the points where the upper eyelid and the lower eyelid are farthest apart is the eye opening distance. The acquisition unit 11 calculates the eye opening distance of the driver 4 in each image, for example, by image recognition. In other words, the acquisition unit 11 calculates the eye opening distance for each of the images acquired continuously from the camera 2. When the eyes 5 of the driver 4 are completely closed, the eye opening distance is 0.

[0015] The acquisition unit 11 may calculate the eye opening distance for each of the left eye and the right eye of the driver 4. The eye opening distance may be an average value of the eye opening distance for the left eye and the eye opening distance for the right eye of the driver 4. The acquisition unit 11 may calculate only the eye opening distance for one of the eyes 5 of the driver 4.

[0016] The setting unit 12 sets the eye opening distance of the state where the eye opening degree of the driver 4 is 100% based on the eye opening distance acquired from the driver 4. The eye opening distance of 100% eye opening degree is an eye opening reference value for determining an abnormal state such as drowsiness or microsleep of an occupant of the vehicle 3, in this case the driver 4. As shown in FIG. 3, the setting unit 12 can set the eye opening distance of 100% eye opening degree based on the eye opening distance acquired by the acquisition unit 11. As shown in FIG. 4, the eye opening degree is an index indicating the open / closed state of the eyes 5, and is determined according to the eye opening distance. The state where the eye opening degree is 100% is an awakening state. The state where the eye opening degree is 25% is a state such as laughing or narrowing the eyes. The state where the eye opening degree is 0% is a state where the eyes 5 are completely closed. The occupant monitoring device 1 is configured to calculate the degree of eye opening of the driver 4 using the above-mentioned eye opening distance of 100% eye opening as the eye opening reference value, and to determine abnormal states of the driver 4, such as drowsiness or microsleep, using various known determination methods based on the degree of eye opening of the driver 4.

[0017] When setting the eye opening distance of 100% eye opening, the setting unit 12 first sets it to an initial value, and then, if a more appropriate value than the initial value can be set, sets a value other than the initial value to the eye opening distance of 100% eye opening. That is, in the occupant monitoring device 1, the eye opening distance of 100% eye opening is first set to the initial value by the setting unit 12. In this state, the eye opening degree of the driver 4 is measured. The setting unit 12 analyzes the eye opening distance obtained from the driver 4. If it is determined as a result of the analysis that a more appropriate eye opening distance of 100% eye opening can be set, the setting unit 12 changes the eye opening distance of 100% eye opening from the initial value to another value.

[0018] A method for setting the eye opening distance for 100% eye opening by the occupant monitoring device 1 will be described with reference to Fig. 5. The method shown in Fig. 5 is performed by the occupant monitoring device 1.

[0019] In step S1, an initial value is set to the eye opening distance for 100% eye opening. The initial value is a predetermined value. For example, the initial value is set to 12 mm. After step S1 is executed, the process proceeds to step S2.

[0020] In step S2, the condition for calculating the update value of the eye opening distance for 100% eye opening is met. The setting unit 12 determines whether the update value calculation condition is met. If the result of step S2 is affirmative, the setting unit 12 determines that it is time to calculate the update value. The calculation condition for the update value is set to the vehicle 3 being traveling at a predetermined vehicle speed or faster. The setting unit 12 starts the update process for the eye opening distance for 100% eye opening, with the calculation condition for the update value as the first condition. Once step S2 has been executed, the process proceeds to step S3.

[0021] In step S3, an update value of the eye opening distance for 100% eye opening is calculated based on the eye opening distance acquired from the driver 4 under the first condition. The setting unit 12 calculates an update value of the eye opening distance for 100% eye opening using the eye opening distance acquired from the driver 4 while the vehicle 3 is traveling at or above a predetermined vehicle speed. Once step S3 has been executed, the process proceeds to step S4.

[0022] In step S4, the condition for updating the eye opening distance to 100% eye opening is met. The setting unit 12 determines whether the update condition for the eye opening distance to 100% eye opening is met. If the result of the determination in step S4 is affirmative, the setting unit 12 determines that it is time to update the eye opening distance to 100% eye opening. Once step S4 has been executed, the process proceeds to step S5.

[0023] In step S5, the eye opening distance of 100% eye opening degree is updated. The setting unit 12 sets the updated value calculated in step S3 to the eye opening distance of 100% eye opening degree. When step S5 is executed, the update process of the eye opening distance of 100% eye opening degree is completed.

[0024] The setting process by the passenger monitoring device 1 will be described with reference to Fig. 6. The control shown in Fig. 6 is executed by the passenger monitoring device 1.

[0025] In step S11, the ignition of the vehicle 3 is turned on. The occupant monitoring device 1 detects that the power of the vehicle 3 has been turned on. If the vehicle 3 is a vehicle equipped with an engine, the occupant monitoring device 1 detects that the engine has been started. After step S11 is executed, the process proceeds to step S12.

[0026] In step S12, an initial value is set for the eye opening distance of 100% eye opening. The setting unit 12 sets a preset value as the initial value for the eye opening distance of 100% eye opening. This setting value is set to, for example, 12 mm. After step S12 is executed, the process proceeds to step S13.

[0027] In step S13, it is determined whether the vehicle speed of the vehicle 3 is 20 km / h or more. The setting unit 12 determines whether the vehicle 3 is traveling at 20 km / h or more. For example, a signal from a vehicle speed sensor provided in the vehicle 3 is input to the occupant monitoring device 1. Alternatively, a signal from a vehicle ECU mounted in the vehicle 3 is input to the occupant monitoring device 1. The occupant monitoring device 1 acquires the vehicle speed of the vehicle 3 based on these input signals. If the determination in step S13 is negative, the determination process in step S13 is repeated. If the determination in step S13 is positive, the process proceeds to step S14.

[0028] In step S14, the stored value is reset to 0. The setting unit 12 resets the stored value stored in the memory of the occupant monitoring device 1 to 0. After step S14 is executed, the process proceeds to step S15.

[0029] In step S15, an updated value of the eye opening distance for 100% eye opening is calculated. The setting unit 12 calculates the eye opening distance for 100% eye opening based on the eye opening distance acquired by the acquisition unit 11. In step S15, a subroutine shown in Fig. 7 is executed. This subroutine will be described later with reference to Fig. 7. After step S15 is executed, the process proceeds to step S16.

[0030] In step S16, it is determined whether the updated value is greater than the initial value. The setting unit 12 compares the initial value set in step S12 with the updated value calculated in step S15, and determines whether the updated value is greater than the initial value. If the determination in step S16 is affirmative, the process proceeds to step S17. If the determination in step S16 is negative, the process proceeds to step S18.

[0031] In step S17, the update value is set to the eye opening distance of 100% eye opening. The setting unit 12 sets the update value calculated in step S15 to the eye opening distance of 100% eye opening. If the eye opening distance of 100% eye opening is set to the initial value, the setting unit 12 changes the eye opening distance of 100% eye opening from the initial value to the update value. After step S17 is executed, the process returns to step S15.

[0032] In step S18, it is determined whether the updated value is greater than the stored value. The setting unit 12 determines whether the updated value calculated in step S15 is greater than the stored value. If the determination in step S18 is affirmative, the process proceeds to step S19. If the determination in step S18 is negative, the process proceeds to step S20.

[0033] In step S19, the updated value is stored in the memory as a saved value. The setting unit 12 stores the updated value calculated in step S15 as a saved value in the memory of the occupant monitoring device 1. After step S19 is executed, the process proceeds to step S20.

[0034] In step S20, a predetermined waiting time is performed. This predetermined time is set to, for example, five minutes. The occupant monitoring device 1 waits for five minutes. After step S20 is executed, the process proceeds to step S21.

[0035] In step S21, it is determined whether or not the calculation of the update value has been performed three or more times. The setting unit 12 determines whether or not the calculation process in step S15 has been performed three or more times. If the determination in step S21 is negative, the process returns to step S15. If the determination in step S21 is positive, the process proceeds to step S22.

[0036] In step S22, the stored value is set to the eye opening distance of 100% eye opening. The setting unit 12 sets the stored value stored in step S19 to the eye opening distance of 100% eye opening. If the eye opening distance of 100% eye opening is set to the initial value, the setting unit 12 changes the eye opening distance of 100% eye opening from the initial value to the stored value. When step S22 is executed, this routine ends.

[0037] Also, when the process returns to step S15 due to a negative determination in step S21, the update value is recalculated in step S15. Then, in step S16, the recalculated update value is compared with the initial value. If the recalculated update value is equal to or smaller than the initial value, the process proceeds from step S16 to step S18.

[0038] Here, step S18 in the previous routine will be explained. When the determination process of step S18 is performed for the first time, the stored value set in step S14 is compared with the updated value calculated in step S15. The stored value set in step S14 is 0. Therefore, the first step S18 is always determined to be positive. That is, in the first routine, the process always proceeds from step S18 to step S19, and step S19 is executed. As a result, the stored value becomes the updated value at the time of returning from step S21 to step S15.

[0039] On the other hand, in step S18 for the second time, the update value calculated again in step S15 of the current routine is compared with the stored value stored in step S19 of the previous routine. The setting unit 12 determines whether or not the update value calculated again in step S15 is greater than the stored value stored in the memory in step S19. In short, in step S18 of the second or subsequent routines, the update value calculated in step S15 of the current routine is compared with the stored value stored in step S19 of the previous or previous routine.

[0040] If the updated value calculated in the current routine is greater than the stored value stored in the previous or previous routine, an affirmative determination is made in step S18. In this case, the process proceeds from step S18 to step S19, and the updated value calculated in the current routine is stored as the stored value. In other words, the stored value is updated.

[0041] On the other hand, if the updated value calculated in the current routine is equal to or less than the stored value stored in the previous routine, a negative determination is made in step S18. In this case, the process proceeds from step S18 to step S20, and the stored value is not updated.

[0042] 6, multiple loop processes occur depending on the result of the determination in step S21. In the determination process in step S21, the condition for exiting the loop is that the update value has been calculated three or more times, so the stored value set in step S22 is the appropriate value.

[0043] The update value calculation process will be described with reference to Fig. 7. As shown in Fig. 7, in the update value calculation process, the face and eyes 5 of the driver 4 are detected in step S31. The acquisition unit 11 detects the face and eyes 5 of the driver 4 based on an image acquired from the camera 2. After step S31 is executed, the process proceeds to step S32.

[0044] In step S32, the eye opening distance of the eyes 5 of the driver 4 is acquired. The acquisition unit 11 calculates the eye opening distance of the eyes 5 based on the image of the eyes 5 acquired from the camera 2. At this time, the eye opening distance is acquired from the driver 4 in a state in which a positive determination is made in step S13 shown in Fig. 6. In other words, the acquisition unit 11 acquires the eye opening distance of the driver 4 under the first condition. After step S32 is executed, the process proceeds to step S33.

[0045] In step S33, the distribution of the eye-opening distance within the set time is calculated. The set time is a predetermined time, and is set to, for example, 3 minutes. The setting unit 12 creates a distribution of the eye-opening distance using the eye-opening distance of the driver 4 acquired when the vehicle speed satisfies the calculation condition of the update value, that is, 20 km / h or more. The data for which this distribution is created is the eye-opening distance within the set time. For example, the setting unit 12 creates a distribution of the eye-opening distance as shown in FIG. 9 based on the eye-opening distance within the set time as shown in FIG. 8.

[0046] As shown in FIG. 8, the distribution of eye open distances is calculated based on the eye open distances acquired while the vehicle 3 is traveling at 20 km / h or faster during the set time of 3 minutes. The horizontal axis of FIG. 8 represents the number of frames, and the vertical axis of FIG. 8 represents the eye open distance. If the camera 2 is capable of shooting video at 30 fbs, the set time of 3 minutes will consist of 5,400 frames. The acquisition unit 11 calculates the eye open distance for each frame within the set time, and the setting unit 12 creates a distribution using the eye open distance for each frame within the set time. Note that FIG. 8 only shows a portion of the data for the set time.

[0047] As shown in FIG. 9, the setting unit 12 creates a distribution of the eye opening distance within the set time. The setting unit 12 creates the distribution of the eye opening distance by a distribution every 1 mm. For example, when the eye opening distance is 0 mm or more and less than 1.5 mm, the eye opening distance is included in the distribution of "1 mm" in the distribution shown in FIG. 9. The frequency of the eye opening distance shown as "1 mm" in the distribution shown in FIG. 9 represents the frequency of the eye opening distance of 0 mm or more and less than 1.5 mm. In the distribution shown in FIG. 9, the value of the eye opening distance near 1 mm is represented as "1 mm". In short, in the distribution shown in FIG. 9, if the eye opening distance is in the class of "0 mm or more and less than 1.5 mm", the class value is represented as "1 mm", and the number of data (frequency) included in this class is represented as the frequency of "1 mm". Also, if the eye opening distance is in the class of "1.5 mm or more and less than 2.5 mm", the class value is represented as "2 mm", and the frequency of this class is represented as the frequency of "2 mm". The same applies to the subsequent eye opening distances. For example, if the eye opening distance is in the class of "6.5 mm or more and less than 7.5 mm", the class value is expressed as "7 mm", and the frequency of this class is expressed as a frequency of "7 mm". If the eye opening distance is in the class of "9.5 mm or more and less than 10.5 mm", the class value is expressed as "100 mm", and the frequency of this class is expressed as a frequency of "10 mm". In this way, the setting unit 12 calculates the distribution of the eye opening distance. After step S33 is executed, the process proceeds to step S34.

[0048] In step S34, the average value of the eye opening distance in the most frequent class is calculated based on the distribution of the eye opening distance. The setting unit 12 identifies the most frequent class value (mode) using the distribution calculated in step S33, and calculates the average value of the eye opening distance based on the data included in the class of the mode. The most frequent class value is synonymous with the class value with the highest frequency. In the example shown in FIG. 9, the most frequent class value (mode) is "10 mm". The setting unit 12 calculates the average value of the eye opening distance in the mode class of "9.5 mm or more and less than 10.5 mm" based on the distribution of the eye opening distance calculated in step S33. In other words, when the mode is "10 mm", the setting unit 12 calculates the average value of the eye opening distance "9.5 mm or more and less than 10.5 mm" within the set time. After step S34 is executed, the process proceeds to step S35.

[0049] In step S35, the average value calculated in step S34 is set as the updated value of the eye opening distance for 100% eye opening. The setting unit 12 sets the eye opening distance calculated in step S34 as the average value of the eye opening distances in the mode class as the updated value of the eye opening distance for 100% eye opening. If the setting unit 12 has calculated the average value of the eye opening distances in the mode class of "9.5 mm or more and less than 10.5 mm" in step S34, the setting unit 12 sets the average value as the updated value of the eye opening distance for 100% eye opening in step S35. After step S35 is executed, the process proceeds to step S16 shown in FIG. 6.

[0050] As described above, according to the occupant monitoring device 1 of the embodiment, the eye opening distance of 100% eye opening degree can be set based on the eye opening distance acquired from the driver 4 while the vehicle 3 is traveling. This allows the occupant monitoring device 1 to set the eye opening distance of 100% eye opening degree, which is the eye opening reference value for determining an abnormal state such as drowsiness or microsleep of the driver 4, to an appropriate value. As a result, the occupant monitoring device 1 can reliably detect, for example, the driver 4's closed eyes (typically, a state in which the eyes are opened by 20% or less) using the eye opening distance of 100% eye opening degree as the eye opening reference value, and can accurately detect an abnormal state of the occupant such as drowsiness or microsleep.

[0051] Furthermore, in the occupant monitoring device 1 of the embodiment, the eye opening distance acquired while the vehicle 3 is traveling is used to calculate the distribution of the eye opening distance within a set time. Then, the average value of the eye opening distance in the most frequent class in the distribution can be set as the update value of the eye opening distance at 100% eye opening. This makes it possible to calculate a more appropriate update value for the eye opening distance at 100% eye opening. As a result, the eye opening distance at 100% eye opening is set to an appropriate eye opening reference value, making it possible to accurately detect an abnormal state of an occupant of the vehicle 3.

[0052] Although the configuration for setting the eye opening distance of 100% eye opening degree of the driver 4 based on the eye opening distance of the driver 4 has been described, the target is not limited to the driver 4, but can be any passenger in the vehicle 3. For example, if two cameras 2 are installed in the driver's seat and the passenger seat, the above setting process can capture an image of the face of the passenger in the passenger seat and set the eye opening distance of 100% eye opening degree for the passenger in the passenger seat.

[0053] The set time is not limited to 3 minutes and may be set to any appropriate time, as long as it is set to a time that allows for obtaining the number of data points required to create a distribution of eye opening distances.

[0054] In addition, the calculation condition of the update value is not limited to the vehicle speed being 20km / h or more. The calculation timing of the update value may be a situation in which the driver 4 is highly likely to be in an awake state. For example, the vehicle speed is not limited to 20km / h, and may be 18km / h or the like.

[0055] In addition, the initial value of the eye opening distance of 100% eye opening degree is not limited to a preset value. The setting unit 12 can calculate a set value as the initial value. The initial value may be a value calculated by the setting unit 12. For example, the setting unit 12 calculates the maximum value of the eye opening distance within the initial setting time based on the eye opening distance within the initial setting time. The initial setting time is a predetermined time, and is set to, for example, 3 minutes. Then, the setting unit 12 sets the maximum value of the calculated eye opening distance within the initial setting time to the initial value of the eye opening distance of 100% eye opening degree. According to this occupant monitoring device 1, the initial value of the eye opening distance of 100% eye opening degree can be set to a value calculated using the eye opening distance acquired from the driver 4. This makes it possible to set the initial value according to the opening and closing operation of the eyelids of the driver 4. Note that this initial setting time is not limited to 3 minutes, and may be set to an appropriate time.

[0056] A modified example of the embodiment will be described with reference to Fig. 10 and Fig. 11. In the occupant monitoring device 1 according to the modified example of the embodiment, the method of calculating the eye opening distance for 100% eye opening degree is different from that of the above embodiment. Fig. 10 is a diagram for explaining the calculation method of the update value in the modified example. Fig. 11 is a flow chart showing the calculation flow of the update value in the modified example. Note that steps S41 and S42 shown in Fig. 11 are the same processes as steps S31 and S32 shown in Fig. 9, and therefore description thereof will be omitted here.

[0057] As shown in FIG. 10, the setting unit 12 sets a maximum value d of the eye opening distance per unit time based on the eye opening distance within a set time among the eye opening distances acquired while the vehicle 3 is traveling. max The set time is a predetermined time, and is set to, for example, 3 minutes. The number of frames within the set time is 5400. The unit time is a predetermined time, and is set to, for example, 3 seconds. The number of frames within the unit time is 90. The setting unit 12 calculates the maximum eye opening distance within the first unit time d as the maximum eye opening distance per unit time. max1 , the maximum eye opening distance within the second unit time d max2 , the maximum eye opening distance within the third unit time d max3 , the maximum eye opening distance within the fourth unit time d max4, the maximum eye opening distance within the fifth unit time d max5 The setting unit 12 calculates the maximum eye opening distance per unit time d max Then, the setting unit 12 sets the calculated average value as an update value for the eye opening distance of 100% eye opening degree.

[0058] As shown in FIG. 11, in the calculation process of the update value in the modified example, when step S41 is executed, the process proceeds to step S42, and when step S42 is executed, the process proceeds to step S43.

[0059] In step S43, the eye-opening distance is analyzed for each unit time. The setting unit 12 analyzes the eye-opening distance acquired in step S42 while the vehicle 3 is traveling for each predetermined time within the set time. After step S43 is executed, the process proceeds to step S44.

[0060] In step S44, the maximum eye opening distance d max The setting unit 12 calculates a maximum value d of the eye opening distance per unit time within the set time based on the eye opening distance within the set time. max After step S44 is executed, the process proceeds to step S45.

[0061] In step S45, it is determined whether or not the set time has elapsed. If the determination in step S45 is affirmative, the process proceeds to step S46. If the determination in step S45 is negative, the process proceeds to step S47.

[0062] In step S46, the maximum eye opening distance d max is stored in the memory as a stored value. The setting unit 12 stores the maximum eye opening distance d max is stored in the memory as a saved value. After step S46 is executed, the process returns to step S45.

[0063] In step S47, the saving process is ended. After step S47 is executed, the process proceeds to step S48.

[0064] In step S48, the maximum eye opening distance per unit time within the set time d max The setting unit 12 calculates the maximum value d of the eye opening distance per unit time calculated based on the eye opening distance within the set time. max After step S48 is executed, the process proceeds to step S49.

[0065] In step S49, the average value calculated in step S48 is set as the update value of the eye opening distance for 100% eye opening. The setting unit 12 sets the average value calculated in step S48 as the update value of the eye opening distance for 100% eye opening. After step S49 is executed, the process proceeds to step S16 shown in FIG.

[0066] According to the modified passenger monitoring device 1, the eye opening distance of 100% eye opening degree can be set to an appropriate value. As a result, the eye opening distance of 100% eye opening degree is set to an appropriate eye opening reference value, so that an abnormal state of the passenger, such as drowsiness or microsleep, can be detected with high accuracy.

[0067] The maximum eye opening distance d max The unit time used in calculating the maximum eye opening distance d is not limited to 3 seconds and may be set to an appropriate value. max The set period used when calculating the average value is not limited to 3 minutes, and may be set to an appropriate value.

[0068] In addition, in the process of step S33 shown in Fig. 7, a distribution of eye opening distances represented by classes and frequencies may be created without calculating class values. That is, the distribution shown in Fig. 9 does not necessarily have to be created. For example, in step S33, the setting unit 12 may create a frequency distribution table (a frequency distribution table not including class values). In this case, the setting unit 12 calculates the frequency (degree) of each of the following classes based on the eye opening distance within the set time: "0 mm or more and less than 1.5 mm", "1.5 mm or more and less than 2.5 mm", "2.5 mm or more and less than 3.5 mm", "3.5 mm or more and less than 4.5 mm", "4.5 mm or more and less than 5.5 mm", "5.5 mm or more and less than 6.5 mm", "6.5 mm or more and less than 7.5 mm", "7.5 mm or more and less than 8.5 mm", "8.5 mm or more and less than 9.5 mm", "9.5 mm or more and less than 10.5 mm", "10.5 mm or more and less than 11.5 mm", and "11.5 mm or more and less than 12.5 mm".

[0069] In addition, the process of step S34 shown in FIG. 7 may specify the most frequent class without specifying the mode, and calculate the average value of the eye-opening distance in that class. In other words, the specified target is not limited to the class value, and may be a class. If a frequency distribution table not including class values ​​is created in step S33, the most frequent class is specified based on the frequency distribution table in step S34, and the average value of the eye-opening distance in that class is calculated. In this way, the setting unit 12 can calculate the average value of the eye-opening distance in the most frequent class without specifying the mode. In other words, the calculation method for calculating the average value of the eye-opening distance in the most frequent class includes a calculation method for specifying the most frequent class value (mode) and calculating the average value of the eye-opening distance in the mode class, and a calculation method for specifying the most frequent class without setting a class value and specifying the most frequent class value and calculating the average value of the eye-opening distance in that class. In step S32, the setting unit 12 calculates the average value of the eye-opening distance using one of these two calculation methods.

[0070] Furthermore, the contents disclosed in the above-described embodiments and modifications can be implemented in appropriate combinations. [Explanation of symbols]

[0071] 1: Occupant monitoring device, 2: Camera, 3: Vehicle, 4: Driver, 5: Eye 11: Acquisition section, 12: Setting section

Claims

1. An acquisition unit that acquires an eye opening distance of a passenger in a vehicle; a setting unit that sets an eye opening reference value for determining an abnormal state of the occupant based on the eye opening distance; Equipped with The setting unit is A predetermined set value is set as an initial value of the eye opening reference value, calculating an update value of the eye opening reference value based on the eye opening distance acquired while the vehicle is traveling; When a predetermined condition is satisfied, the eye opening reference value is changed from the initial value to the updated value. Occupant monitoring device.

2. The setting unit is Calculating a distribution of the eye opening distances based on the eye opening distances within a set time among the eye opening distances acquired while the vehicle is traveling; The average value of the eye opening distance in the most frequent class in the distribution is set as the updated value.

2. An occupant monitoring device according to claim 1.

3. The setting unit is calculating a maximum value of the eye opening distance for each predetermined time based on the eye opening distance within a set time among the eye opening distances acquired while the vehicle is traveling; Calculating an average value of the maximum eye opening distance for each predetermined time period; The calculated average value is set as the updated value.

2. An occupant monitoring device according to claim 1.

4. The setting unit calculates the setting value based on the eye opening distance acquired within a preset initial setting time. An occupant monitoring device according to any one of claims 1 to 3.

Images