Drowsiness determination system and drowsiness determination method

Abstract

To improve accuracy in determining drowsiness of a crewman.SOLUTION: A drowsiness determination system 100 includes a first acquisition section 11, a second acquisition section 12, a third acquisition section 13, and a determination section 14. The first acquisition section 11 acquires vibration information of a vibration of a vehicle. The second acquisition section 12 acquires head information indicating a fluctuation of the head of a crewman. The third acquisition section 13 acquires seating height information indicating a seating height of the crewman. When the vibration information indicates generation of the vibration of the vehicle, the determination section 14 determines whether the crewman has drowsiness.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a drowsiness determination system and a drowsiness determination method for determining drowsiness of an occupant. [Background technology]

[0002] Patent Document 1 discloses a drowsiness determination device. This drowsiness determination device includes a determination unit that determines whether or not an occupant is drowsy based on vehicle information indicating the amount of change in the vehicle attitude relative to the road surface and head information indicating the amount of change in the occupant's head relative to a predetermined position. The determination unit determines that the occupant is drowsy if, when the vehicle attitude changes, the occupant's head changes in the same direction as the direction of change in the vehicle attitude and the delay time between the time when the vehicle attitude changes and the time when the occupant's head changes is longer than a predetermined time. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6516187 Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides a drowsiness determination system and the like that can improve the accuracy of determining drowsiness of an occupant. [Means for solving the problem]

[0005] A drowsiness determination system according to one aspect of the present disclosure includes a first acquisition unit that acquires vibration information indicating vehicle vibration, which is an amount of variation in vehicle attitude relative to a road surface, a second acquisition unit that acquires head information indicating a variation in the head of an occupant, a third acquisition unit that acquires sitting height information indicating a sitting height of the occupant, and a determination unit that determines whether the occupant is drowsy based on magnitudes of the head information, the sitting height information, and the vibration information. In determining whether the occupant is drowsy, the determination unit determines a magnitude relationship of peak values ​​from the vibration information and determines a threshold value from the magnitude relationship between the sitting height information and the peak value, determines that vibration has occurred in the vehicle when the vibration information exceeds a predetermined value, and determines the drowsiness of the occupant by comparing the head information with the threshold value when the head of the occupant has varied in the same direction as the variation direction of the vehicle attitude, and in determining the magnitude relationship, determines that the peak value is small when the peak value is equal to or less than a first upper limit value and determines that the peak value is equal to or less than a second upper limit value. Value The third upper limit is larger than the Value The peak value is determined to be large when the the second upper limit value is greater than the first upper limit value, and the third upper limit value is greater than the second upper limit value; The first upper limit value is greater than the difference between the second upper limit value and the third upper limit value.

[0006] A drowsiness determination method according to one aspect of the present disclosure includes a first acquisition step of acquiring vibration information indicating vehicle vibration, which is a variation in vehicle attitude relative to a road surface, a second acquisition step of acquiring head information indicating a variation in the head of an occupant, a third acquisition step of acquiring sitting height information indicating a sitting height of the occupant, and a determination step of determining whether the occupant is drowsy based on magnitudes of the head information, the sitting height information, and the vibration information, wherein the determination step determines a magnitude relationship of peak values ​​from the vibration information, determines a threshold value from the magnitude relationship between the sitting height information and the peak value, determines that vibration has occurred in the vehicle when the vibration information exceeds a predetermined value, and determines the drowsiness of the occupant by comparing the head information with the threshold value when the head of the occupant has varied in the same direction as the variation direction of the vehicle attitude, and wherein the determination of the magnitude relationship includes: When the peak value is equal to or less than a first upper limit value, the peak value is determined to be small, and when the peak value is equal to or less than a second upper limit value, Value The third upper limit is larger than the Value The peak value is determined to be large when the the second upper limit value is greater than the first upper limit value, and the third upper limit value is greater than the second upper limit value; The first upper limit value is greater than the difference between the second upper limit value and the third upper limit value. Effect of the Invention

[0007] The drowsiness determination system and the like of the present disclosure can improve the accuracy of determining drowsiness of an occupant. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing an overview of a drowsiness determination system according to the first embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of the state of the vehicle and the head of the occupant when the amount of change in the attitude of the vehicle reaches a peak. [Diagram 3] FIG. 3 is a diagram showing an example of the state of the vehicle and the occupant's head at the time when the amount of movement of the occupant's head reaches a peak. [Figure 4] FIG. 4 is a waveform diagram showing an example of a change over time in the amount of change in the attitude of the vehicle and the amount of change in the head of the occupant. [Diagram 5] FIG. 5 is a scatter plot showing the correlation between the drowsiness level of the occupant, the delay time, and the seated height of the occupant. [Figure 6] FIG. 6 is a flowchart showing an operation example of the drowsiness determination system according to the first embodiment. [Figure 7] FIG. 7 is a block diagram showing an outline of a drowsiness determination system according to the second embodiment. [Figure 8] FIG. 8 is a flowchart showing an operation example of the drowsiness determination system in the second embodiment. [Figure 9] Figure 9 is a scatter plot showing the correlation between the occupant's drowsiness level based on the peak value of the change in the vehicle's posture, the delay time, and the occupant's sitting height in the drowsiness determination system of embodiment 3, where (a) is when the peak value is small, (b) is when the peak value is medium, and (c) is when the peak value is large. [Figure 10]FIG. 10 is a flowchart showing an operation example of the drowsiness determination system according to the third embodiment. [Figure 11] FIG. 11 is a correlation diagram between the peak value of the amount of change in the attitude of the vehicle and the delay time in the drowsiness determination system according to the third embodiment. [Figure 12] FIG. 12 is a flowchart showing an operation example of the drowsiness determination system in the fourth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] A drowsiness determination system according to an embodiment of the present disclosure includes a first acquisition unit, a second acquisition unit, a third acquisition unit, and a determination unit. The first acquisition unit acquires vibration information indicating vibration of a vehicle. The second acquisition unit acquires head information indicating a movement of an occupant's head. The third acquisition unit acquires sitting height information indicating a sitting height of the occupant. When the vibration information indicates the occurrence of vibration of the vehicle, the determination unit determines whether the occupant is drowsy based on the head information and the sitting height information.

[0010] According to this, taking into consideration that the head information depends on the seated height of the occupant, whether or not the occupant is sleepy is judged by referring to not only the head information but also the seated height information. Therefore, compared to the case where whether or not the occupant is sleepy is judged by referring only to the head information, it is possible to reduce the influence of individual differences in the occupant's reaction to the vibration of the vehicle, and it is possible to improve the accuracy of judging the occupant's sleepiness.

[0011] In a drowsiness determination system according to another aspect of the present disclosure, the third acquisition unit acquires the sitting height information by estimating the sitting height based on a captured image of the head.

[0012] This has the advantage that the seated height information can be used without the occupant having to input the seated height information in advance, reducing the effort required by the occupant and making it easier to improve convenience.

[0013] In the drowsiness determination system according to another aspect of the present disclosure, the determination unit determines whether the occupant is drowsy by comparing the head information with a threshold value. The determination unit determines the threshold value from among a plurality of candidate threshold values ​​based on the sitting height information.

[0014] This has the advantage that the drowsiness of an occupant can be determined by a simple method of comparing head information with an appropriate threshold value according to sitting height information, making it easier to design the system.

[0015] In the drowsiness determination system according to another aspect of the present disclosure, the determination unit has a determination model that receives the vibration information and the head information and outputs a determination result as to whether the occupant is drowsy. The determination unit determines the determination model from a plurality of candidate determination models based on the sitting height information.

[0016] This has the advantage that it is possible to expect further improvement in the accuracy of determining occupant drowsiness compared to methods that determine occupant drowsiness by comparing vibration information with a predetermined value and comparing head information with a threshold value.

[0017] In a drowsiness determination system according to another aspect of the present disclosure, the head information includes at least one of the delay time until the head moves during a predetermined period from the time when vibration of the vehicle occurs, the maximum value of the amplitude of the head movement relative to a predetermined position, the average value of the amplitude of the head movement relative to the predetermined position, and the variance of the head position relative to the predetermined position.

[0018] According to this, since the drowsiness of the occupant can be determined using one or more parameters that can vary depending on the drowsiness of the occupant, there is an advantage that it is easy to improve the accuracy of determining the drowsiness of the occupant.

[0019] In a drowsiness determination system according to another aspect of the present disclosure, the determination unit resets the counting of the predetermined period when the predetermined period has elapsed or when a magnitude of vibration of the vehicle exceeds a predetermined value.

[0020] This has the advantage that it becomes easier to eliminate the movement of the occupant's head that occurs regardless of the vibration of the vehicle, and it becomes easier to further improve the accuracy of determining the occupant's drowsiness.

[0021] In a drowsiness determination system according to another aspect of the present disclosure, the determination unit does not determine whether the occupant is drowsy until the magnitude of the vibration of the vehicle exceeds the predetermined value after resetting the timing of the specified period.

[0022] This has the advantage that it becomes easier to eliminate the movement of the occupant's head that occurs regardless of the vibration of the vehicle, and it becomes easier to further improve the accuracy of determining the occupant's drowsiness.

[0023] In a drowsiness determination system according to another aspect of the present disclosure, the determination unit decelerates the vehicle when a period of time during which no vibration occurs in the vehicle continues for a predetermined period of time or longer.

[0024] This has the advantage that it is possible to judge the drowsiness of the occupant by forcibly generating vehicle vibration even when a long period of time continues in which no vehicle vibration occurs, for example, due to a continuous flat road surface, etc. In particular, since the occupant's drowsiness is likely to increase during a period in which no vehicle vibration occurs, there is also the advantage that by judging the occupant's drowsiness during such a period, it becomes easier to take measures such as encouraging the occupant to wake up.

[0025] In a drowsiness determination system according to another aspect of the present disclosure, the determination unit further determines whether the occupant is drowsy based on a magnitude of the vibration information.

[0026] This has the advantage that the error in the delay time based on the magnitude of the vehicle vibration information can be corrected, thereby enabling even more accurate drowsiness determination.

[0027] In a drowsiness determination system according to another aspect of the present disclosure, when the determination unit determines whether the occupant is drowsy, the range in which the vibration information is determined to be small is wider than the range in which the vibration information is determined to be large.

[0028] According to this, the delay time between the actual vehicle vibration and the head movement does not vary much when the vehicle vibration is small, but increases sharply as the vehicle vibration increases. Therefore, since the magnitude of the vibration information can be determined taking into account such nonlinear behavior, it has the advantage of enabling even more accurate drowsiness determination.

[0029] In a drowsiness determination system according to another aspect of the present disclosure, the determination unit further calculates a representative value of the eyelid position of the occupant within a specified period of time from the head information based on an image of the head, and determines whether the occupant is drowsy by taking into account the representative value of the eyelid position.

[0030] According to this, the representative value of the eyelid position is also used in the drowsiness judgment, which has the advantage that drowsiness judgment can be made with higher accuracy.

[0031] In a drowsiness determination system according to another aspect of the present disclosure, the determination unit determines head drowsiness based on the head information and the sitting height information, and eyelid drowsiness based on a representative value of the eyelid position, and weights the head drowsiness and the eyelid drowsiness to determine the drowsiness of the occupant.

[0032] According to this, for example, by weighting eyelid drowsiness, which represents direct drowsiness, higher than head drowsiness, the effect of eyelid drowsiness on overall drowsiness can be increased, which has the advantage of further increasing the accuracy of drowsiness judgment.

[0033] In a drowsiness determination system according to another aspect of the present disclosure, the determination unit determines whether the occupant is wearing glasses based on the head information, and if the occupant is wearing glasses, the type of glasses, and determines the occupant's drowsiness by weighting the head drowsiness and eyelid drowsiness according to the determination result.

[0034] This has the advantage that the accuracy of drowsiness determination is improved because errors in eyelid drowsiness caused by whether or not glasses are worn, and, if glasses are worn, the type of glasses, can be compensated for by head drowsiness.

[0035] In a drowsiness determination system according to another aspect of the present disclosure, the determination unit increases the weighting of the eyelid drowsiness relative to the head drowsiness if the occupant is not wearing sunglasses, and decreases the weighting of the eyelid drowsiness relative to the head drowsiness if the occupant is wearing sunglasses.

[0036] This has the advantage that the error in eyelid drowsiness caused by sunglasses can be compensated for by head drowsiness, thereby improving the accuracy of drowsiness determination.

[0037] A drowsiness determination method according to one aspect of the present disclosure includes a first acquisition step, a second acquisition step, a third acquisition step, and a determination step. In the first acquisition step, vibration information indicating vibration of a vehicle is acquired. In the second acquisition step, head information indicating a movement of an occupant's head is acquired. In the third acquisition step, sitting height information indicating a sitting height of the occupant is acquired. In the determination step, when the vibration information indicates the occurrence of vibration of the vehicle, it is determined whether the occupant is drowsy based on the head information and the sitting height information.

[0038] According to this, taking into consideration that the head information depends on the seated height of the occupant, whether or not the occupant is sleepy is judged by referring to not only the head information but also the seated height information. Therefore, compared to the case where whether or not the occupant is sleepy is judged by referring only to the head information, it is possible to reduce the influence of individual differences in the occupant's reaction to the vibration of the vehicle, and it is possible to improve the accuracy of judging the occupant's sleepiness.

[0039] In the drowsiness determination method according to one aspect of the present disclosure, the determining step further determines whether the occupant is drowsy based on a magnitude of the vibration information.

[0040] This has the advantage that the error in the delay time based on the magnitude of the vehicle vibration information can be corrected, thereby enabling even more accurate drowsiness determination.

[0041] In one embodiment of the drowsiness determination method of the present disclosure, the determination step further includes calculating a representative value of the eyelid position of the occupant within a specified period from the head information based on an image of the head, and determining whether the occupant is drowsy by taking into account the representative value of the eyelid position.

[0042] According to this, the representative value of the eyelid position is also used in the drowsiness judgment, which has the advantage that drowsiness judgment can be made with higher accuracy.

[0043] Hereinafter, the first to fourth embodiments will be specifically described with reference to the drawings.

[0044] Each of the embodiments described below is either a comprehensive or specific example. The numerical values, shapes, materials, components, the installation positions and connection forms of the components, steps, and the order of steps shown in each of the embodiments below are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in each of the embodiments below, components that are not described in the independent claims are described as optional components.

[0045] In addition, each drawing is a schematic diagram and is not necessarily a precise illustration. In addition, in each drawing, the same components are denoted by the same reference numerals.

[0046] (Embodiment 1) <Configuration> Fig. 1 is a block diagram showing an overview of a drowsiness determination system 100 in the first embodiment. The drowsiness determination system 100 in the first embodiment is a system used in a moving object such as a vehicle 2 (see Fig. 2) for assisting driving of the vehicle 2. In the first embodiment, the drowsiness determination system 100 is realized by an in-vehicle device, but may be realized by an external device brought in from outside the vehicle 2. In the first embodiment, an occupant 3 (see Fig. 2) targeted by the drowsiness determination system 100 is a driver of the vehicle seated in the driver's seat, but may be an occupant other than the driver seated in the passenger seat or a rear seat.

[0047] 1, the drowsiness determination system 100 includes a first acquisition unit 11, a second acquisition unit 12, a third acquisition unit 13, a determination unit 14, and a notification unit 15. Note that the drowsiness determination system 100 only needs to include at least the first acquisition unit 11, the second acquisition unit 12, the third acquisition unit 13, and the determination unit 14, and does not necessarily need to include the notification unit 15.

[0048] The first acquisition unit 11 acquires vibration information indicating vibration of the vehicle 2. The first acquisition unit 11 is an entity that executes a first acquisition step ST1 in the drowsiness determination method. The first acquisition unit 11 acquires the detection result output by the vibration detection unit 21, thereby acquiring the vibration information.

[0049] The vibration detection unit 21 is, for example, a gyro sensor mounted on the vehicle 2, and detects the vibration of the vehicle 2, in other words, the amount of change in the attitude of the vehicle 2 with respect to the road surface. The amount of change in the attitude of the vehicle 2 is represented by the magnitude of the pitch angle, roll angle, or yaw angle of the vehicle 2 with respect to the road surface. Note that the amount of change in the attitude of the vehicle 2 may be represented, for example, by the distance between one end of the vehicle 2 in the longitudinal direction and the road surface.

[0050] The vibration detection unit 21 is not limited to a gyro sensor, and may be, for example, an acceleration sensor that detects the acceleration of the vehicle 2.

[0051] The second acquisition unit 12 acquires head information indicating a movement of the head 31 of the occupant 3. The second acquisition unit 12 is an entity that executes the second acquisition step ST2 in the drowsiness determination method. The second acquisition unit 12 calculates the amount of movement of the head 31 of the occupant 3 relative to a predetermined position in the vehicle 2, for example, by performing appropriate image processing on the captured image, and acquires the head information based on the calculated amount of movement of the head 31 of the occupant 3. The captured image is an image including the head 31 of the occupant 3, and is captured by, for example, the camera 22 mounted on the vehicle 2. The predetermined position is, for example, the position of the headrest of the seat 4 in which the occupant 3 is seated, the position of a side door, etc.

[0052] Specifically, the second acquisition unit 12 extracts feature points of the head 31 of the occupant 3 by performing appropriate image processing on the captured image, and calculates the coordinates of the feature points. The feature points are, for example, the top of the head 31 of the occupant 3. Of course, the feature points may be other than the top of the head, such as the eyes of the occupant 3. Then, the second acquisition unit 12 calculates the amount of change in the extracted feature points over time as the amount of fluctuation of the head 31 of the occupant 3.

[0053] In the first embodiment, the head information is the delay time Td (see FIG. 4). The delay time Td is a time corresponding to the difference between the time when the amount of change in the attitude of the vehicle 2 (i.e., the vibration of the vehicle 2) reaches a peak and the time when the amount of change in the head 31 of the occupant 3 reaches a peak, during a predetermined period T1 (see FIG. 4) from the time when the change in the attitude of the vehicle 2 (i.e., the vibration of the vehicle 2) occurs. Note that the delay time Td is not limited to this, and may be, for example, a time difference based on the time when each of the changes in the attitude of the vehicle 2 and the head 31 of the occupant 3 occurs.

[0054] The delay time Td will be described in detail below with reference to Figs. 2 to 4. Fig. 2 is a diagram showing an example of the state of the vehicle 2 and the head 31 of the occupant 3 at the time when the amount of change in the attitude of the vehicle 2 reaches a peak. Fig. 3 is a diagram showing an example of the state of the vehicle 2 and the head 31 of the occupant 3 at the time when the amount of change in the head 31 of the occupant 3 reaches a peak. Note that in both Figs. 2 and 3, the changes in the vehicle 2 and the head 31 of the occupant 3 are emphasized so that they can be easily visually grasped. Fig. 4 is a waveform diagram showing an example of the change over time in the amount of change in the attitude of the vehicle 2 and the amount of change in the head 31 of the occupant 3.

[0055] As shown in Fig. 2, while the vehicle 2 is traveling toward the right side of the drawing, the attitude of the vehicle 2 changes in the pitch direction relative to the road surface due to acceleration or deceleration of the vehicle 2, a change in the road surface condition, or the like. In this case, as shown in Fig. 3, the head 31 of the occupant 3 tilts toward the traveling direction of the vehicle 2 (here, the right side of the drawing) so as to follow the change in the attitude of the vehicle 2. Specifically, the head 31 of the occupant 3 (here, the top of the head as a characteristic point) moves from an initial position P2 a distance Ho away from a predetermined position P1 (here, the position of the headrest) to a peak position P3 where the amount of movement from an initial position P2 reaches a peak value Hp.

[0056] In the example shown in Fig. 4, the posture of the vehicle 2 starts to change at time t0, then the amount of change reaches a peak value Vp at time t1, and returns to the original state after time t3. Meanwhile, the head 31 of the occupant 3 starts to change at time t3, which is delayed from time t1 when the posture of the vehicle 2 starts to change, then the amount of change reaches a peak value Hp at time t4, and returns to the original state after time t5. Therefore, in the example shown in Fig. 4, the delay time Td is a time corresponding to the difference between time t1 when the amount of change in the posture of the vehicle 2 reaches the peak value Vp and time t4 when the amount of change in the head 31 of the occupant 3 reaches the peak value Hp in the predetermined period T1 from time t1 when the change in the posture of the vehicle 2 (i.e., the vibration of the vehicle 2) occurs.

[0057] Here, if the occupant 3 is not drowsy, the occupant 3 makes a reflexive movement in response to the change in the attitude of the vehicle 2 in order to maintain his / her own attitude. In other words, if the occupant 3 is not drowsy, the occupant 3 immediately follows the behavior of the head 31 to the change in the attitude of the vehicle 2, so the delay in the change in the head 31 of the occupant 3 in response to the change in the attitude of the vehicle 2 is short. On the other hand, if the occupant 3 is drowsy, the function of the occupant 3 to maintain his / her attitude decreases as the consciousness of the occupant 3 decreases. Therefore, the occupant 3 cannot immediately follow the behavior of the head 31 to the change in the attitude of the vehicle 2, so the delay in the change in the head 31 of the occupant 3 in response to the change in the attitude of the vehicle 2 becomes long. In this way, the delay time Td is a parameter that can change depending on the drowsiness of the occupant 3, and the greater the drowsiness of the occupant 3, the longer the delay time Td can be.

[0058] Alternatively, the head information may be the maximum value (hereinafter also referred to as "maximum head amplitude") or average value (hereinafter also referred to as "average head amplitude") of the amplitude of fluctuation of the head 31 of the occupant 3 relative to a predetermined position during the above-mentioned predetermined period T1. Furthermore, the head information may be the variance (hereinafter also referred to as "head position variance") of the position of the head 31 of the occupant 3 relative to a predetermined position during the above-mentioned predetermined period T1. The maximum value of the head amplitude, the average value of the head amplitude, and the variance of the head position are all parameters that can vary depending on the drowsiness of the occupant 3, similar to the delay time Td, and each of them can become larger the greater the drowsiness of the occupant 3.

[0059] In other words, the head information only needs to include at least one of the delay time Td until the head 31 moves during a predetermined period T1 from the time when vibration of the vehicle 2 occurs, the maximum value of the amplitude of the movement of the head 31 relative to the predetermined position, the average value of the amplitude of the movement of the head 31 relative to the predetermined position, and the variance of the position of the head 31 relative to the predetermined position.

[0060] The third acquisition unit 13 acquires seated height information indicating the seated height of the occupant 3. The third acquisition unit 13 is an entity that executes a third acquisition step ST3 in the drowsiness determination method. The "seated height" here may be, for example, a seated height defined by a physical measurement in a medical checkup, or may be a height from a part of the seat 4 where the occupant 3 sits to the top of the head 31 of the occupant 3. In the latter case, it may change depending on the way the occupant 3 sits in the seat 4, such as the occupant 3 sitting shallowly in the seat 4 or the occupant 3 sitting deep in the seat 4.

[0061] In the first embodiment, the third acquisition unit 13 estimates the sitting height based on a captured image of the head 31 of the occupant 3, thereby acquiring the sitting height information. Specifically, the third acquisition unit 13 executes appropriate image processing on the captured image to extract a feature point of the head 31 of the occupant 3, and calculates the coordinates of the feature point. In this case, the feature point is the top of the head 31 of the occupant 3. Then, the third acquisition unit 13 estimates the sitting height of the occupant 3 by calculating the difference in height between the coordinates indicating the position of the seating surface of the seat 4, which have been acquired in advance, and the coordinates of the calculated feature point.

[0062] When the camera 22 mounted on the vehicle 2 can capture an image of not only the occupant 3 but also the seat 4, the third acquisition unit 13 may extract the location of the seat 4 where the occupant 3 sits from the captured image, and calculate the coordinates of the extracted location as coordinates indicating the position of the seating surface of the seat 4. In this case, the third acquisition unit 13 can estimate the seat height only from the captured image, and does not need to acquire in advance the coordinates indicating the position of the seating surface of the seat 4.

[0063] In the first embodiment, the third acquisition unit 13 estimates the sitting height of the occupant 3 at the time when the occupant 3 sits in the seat 4. The third acquisition unit 13 may estimate the sitting height of the occupant 3 in real time using a captured image every time an image is captured by the camera 22 mounted on the vehicle 2. In the latter case, it is easy to accurately estimate the sitting height of the occupant 3 even if the posture of the occupant 3 changes from moment to moment.

[0064] When the vibration information indicates the occurrence of vibration of the vehicle 2, the judgment unit 14 judges whether the occupant 3 is drowsy based on the head information and sitting height information. The judgment unit 14 is the entity that executes the judgment step ST4 in the drowsiness judgment method. As described above, the head information (here, the delay time Td) varies depending on the drowsiness of the occupant 3. Therefore, in the first embodiment, the judgment unit 14 judges whether the occupant 3 is drowsy by comparing the head information with a threshold value.

[0065] Specifically, the determination unit 14 first waits until the amount of change in the attitude of the vehicle 2 (i.e., the magnitude of vibration of the vehicle 2) exceeds a predetermined value by referring to the vibration information acquired by the first acquisition unit 11. When the amount of change exceeds the predetermined value, the determination unit 14 starts measuring a predetermined period T1. Then, the determination unit 14 refers to the vibration information acquired by the first acquisition unit 11 and the head information acquired by the second acquisition unit 12 to determine whether the direction of vibration of the vehicle 2 is the same as the direction of vibration of the head 31 of the occupant 3. If these directions are different from each other, the determination unit 14 does not execute a process for determining drowsiness of the occupant 3. On the other hand, if these directions are the same, the determination unit 14 calculates a delay time Td as the head information.

[0066] The determination unit 14 compares the calculated delay time Td with a threshold value, and determines that the occupant 3 is not drowsy if the delay time Td is equal to or less than the threshold value. On the other hand, the determination unit 14 determines that the occupant 3 is drowsy if the delay time Td is greater than the threshold value. Then, when the determination unit 14 determines that the occupant 3 is drowsy, it outputs a notification command to the notification unit 15.

[0067] As already mentioned, when the occupant 3 feels drowsy, the reaction to the change in the posture of the vehicle 2 is delayed, so the delay time Td becomes longer, but there are individual differences in this reaction delay. In other words, there is a problem that it is difficult to uniquely determine a threshold value because there may be individual differences in the head information (here, the delay time Td) depending on the occupant 3.

[0068] The inventors of the present application have found that the above-mentioned individual differences depend on the sitting height of the occupant 3. That is, the amplitude of vibration of the head 31 of the occupant 3 changes depending on the distance between the head 31 (top of the head) of the occupant 3 and the location where the occupant 3 sits on the seat 4, which is the origin of the vibration of the head 31 of the occupant 3. In other words, the higher the sitting height of the occupant 3, the longer the delay time Td can be.

[0069] FIG. 5 is a scatter diagram showing the correlation between the drowsiness level of the occupant 3, the delay time Td, and the sitting height of the occupant 3. In FIG. 5, the vertical axis represents the drowsiness level of the occupant 3, and the horizontal axis represents the delay time Td. The drowsiness level is a level defined by, for example, the KSS (Kwansei Gakuin Sleepiness Scale), the KSS (Karolinska Sleepiness Scale), or a facial expression evaluation. Also, the dots in FIG. 5 represent data of a large number of occupants 3 having different sitting heights. In FIG. 5, the first data group surrounded by the dotted line A1 represents a data group of occupants 3 whose sitting height is lower than a first predetermined value, and the third data group surrounded by the dashed line A3 represents a data group of occupants 3 whose sitting height is higher than a second predetermined value (>first predetermined value). Also, in FIG. 5, the second data group surrounded by the dashed line A2 represents a data group of occupants 3 whose sitting height is in the range between the first predetermined value and the second predetermined value.

[0070] 5, even for the same drowsiness level, the corresponding delay time Td differs depending on the sitting height of the occupant 3. For example, when the drowsiness level is "L1", the delay time Td of the occupant 3 belonging to the first data group, the delay time Td of the occupant 3 belonging to the second data group, and the delay time Td of the occupant 3 belonging to the third data group are different from one another.

[0071] Therefore, in the first embodiment, the determination unit 14 determines a threshold value to be used when determining the drowsiness of the occupant 3 from among a plurality of threshold value candidates, based on the seated height information acquired by the third acquisition unit 13. Specifically, if the seated height of the occupant 3 is lower than a first predetermined value, the determination unit 14 determines a first threshold value from among three threshold value candidates as a threshold value to be used when determining the drowsiness of the occupant 3. Furthermore, if the seated height of the occupant 3 is between the first predetermined value and the second predetermined value, the determination unit 14 determines a second threshold value (>first threshold value) from among three threshold value candidates as a threshold value to be used when determining the drowsiness of the occupant 3. Furthermore, if the seated height of the occupant 3 is higher than the second predetermined value, the determination unit 14 determines a third threshold value (>second threshold value) from among three threshold value candidates as a threshold value to be used when determining the drowsiness of the occupant 3.

[0072] The determination unit 14 resets the timing of the predetermined period T1 when the predetermined period T1 has elapsed or when the amount of change in the attitude of the vehicle 2 (i.e., the magnitude of vibration of the vehicle 2) exceeds a predetermined value. Specifically, when the predetermined period T1 has elapsed since the determination unit 14 started timing the predetermined period T1, the determination unit 14 stops the process of determining the drowsiness of the occupant 3. Furthermore, after starting to time the predetermined period T1, if the magnitude of the vibration of the vehicle 2 exceeds a predetermined value midway through the predetermined period T1, that is, if a new change in the attitude of the vehicle 2 occurs, the determination unit 14 resets the timing of the predetermined period T1 to set the time point at which this change occurred as the starting point of the predetermined period T1.

[0073] Furthermore, the determination unit 14 does not determine whether the occupant 3 is drowsy from when the clocking of the predetermined period T1 is reset until the amount of change in the attitude of the vehicle 2 (i.e., the magnitude of vibration of the vehicle 2) exceeds a predetermined value. In other words, the determination unit 14 does not immediately start clocking a new predetermined period T1 after the predetermined period T1 has elapsed, but does not execute the process of determining whether the occupant 3 is drowsy until a new change in the attitude of the vehicle 2 occurs.

[0074] In addition, the determination unit 14 decelerates the vehicle 2 when the time when the vehicle 2 does not vibrate continues for a predetermined time or more. Here, "the vehicle 2 does not vibrate" includes the vehicle 2 not vibrating at all, and also includes the magnitude of the vibration of the vehicle 2 being below a predetermined value. Specifically, the determination unit 14 measures the time when the vehicle 2 does not vibrate by monitoring the vibration information. Then, when the measured time when the vehicle 2 does not vibrate reaches or exceeds the predetermined time, the determination unit 14 transmits a command to decelerate the vehicle 2 to an ECU (Electronic Control Unit) mounted on the vehicle 2. The ECU that receives the command decelerates the vehicle 2, for example, by performing a downshift control. This forcibly changes the posture of the vehicle 2, thereby making it possible to execute a process of determining the drowsiness of the occupant 3 by the determination unit 14. The deceleration of the vehicle 2 is to an extent that induces the movement of the head 31 of the occupant 3, and does not interfere with the driving of the vehicle 2.

[0075] After executing the process of determining the drowsiness of the occupant 3, the determination unit 14 may transmit a command to the ECU to return the speed of the vehicle 2 to the original speed. The ECU that receives the command returns the speed of the vehicle 2 to the original speed by, for example, performing a shift-up control.

[0076] When the notification unit 15 receives a notification command from the determination unit 14, it issues a notification to urge the occupant 3 to wake up, for example by outputting an attention-calling sound from a speaker.

[0077] <Operation> Hereinafter, the operation of the drowsiness determination system 100 in the first embodiment will be described with reference to Fig. 6. Fig. 6 is a flowchart showing an example of the operation of the drowsiness determination system 100 in the first embodiment. In the following, a case where the head information is the delay time Td will be described.

[0078] First, when the vehicle 2 starts traveling, the third acquisition unit 13 acquires sitting height information (S101). The process S101 corresponds to the third acquisition step ST3 in the drowsiness determination method. Then, the determination unit 14 determines a threshold value according to the sitting height of the occupant 3 from among a plurality of threshold value candidates by referring to the sitting height information acquired by the third acquisition unit 13 (S102). The process S102 corresponds to the determination step ST4 in the drowsiness determination method. Next, the first acquisition unit 11 acquires vibration information (S103). The process S103 corresponds to the first acquisition step ST1 in the drowsiness determination method. Furthermore, the second acquisition unit 12 acquires head information (S104). The process S104 corresponds to the second acquisition step ST2 in the drowsiness determination method. The processes S103 and S104 are constantly executed while the vehicle 2 is traveling.

[0079] The determination unit 14 monitors whether or not the magnitude of the vibration of the vehicle 2 exceeds a predetermined value based on the vibration information acquired by the first acquisition unit 11 (S105). If the magnitude of the vibration of the vehicle 2 exceeds the predetermined value (S105: Yes), the determination unit 14 further determines whether or not the direction of the vibration of the vehicle 2 is the same as the direction of the vibration of the head 31 of the occupant 3 based on the head information acquired by the second acquisition unit 12 (S106). On the other hand, if the magnitude of the vibration of the vehicle 2 does not exceed the predetermined value (S105: No), the determination unit 14 does not execute anything in particular.

[0080] When the direction of vibration of the vehicle 2 and the direction of vibration of the head 31 of the occupant 3 are the same (S106: Yes), the determination unit 14 calculates the delay time Td (S107). Then, the determination unit 14 compares the calculated delay time Td with the threshold determined in the process S102 (S108). When the delay time Td is greater than the threshold (S108: Yes), the determination unit 14 determines that the occupant 3 is drowsy (S109). Then, the determination unit 14 outputs a notification command to the notification unit 15 to cause the notification unit 15 to perform a notification to prompt the occupant 3 to wake up (S110). On the other hand, when the delay time Td is equal to or less than the threshold (S108: No), the determination unit 14 determines that the occupant 3 is not drowsy (S111). Moreover, if the direction of vibration of the vehicle 2 and the direction of vibration of the head 31 of the occupant 3 are not the same (S106: No), the determination unit 14 does not execute anything in particular. Processes S105 to S109 and S111 correspond to determination step ST4 in the drowsiness determination method. The drowsiness determination system 100 repeats the above series of processes until the vehicle 2 stops traveling (S112: No). Then, when the vehicle 2 stops traveling (S112: Yes), the drowsiness determination system 100 ends its operation.

[0081] <Action and effect> As described above, the drowsiness determination system 100 in the first embodiment includes the first acquisition unit 11, the second acquisition unit 12, the third acquisition unit 13, and the determination unit 14. The first acquisition unit 11 acquires vibration information indicating vibration of the vehicle 2. The second acquisition unit 12 acquires head information indicating a movement of the head 31 of the occupant 3. The third acquisition unit 13 acquires sitting height information indicating the sitting height of the occupant 3. When the vibration information indicates the occurrence of vibration of the vehicle 2, the determination unit 14 determines whether the occupant 3 is drowsy based on the head information and sitting height information.

[0082] According to this, in consideration of the fact that the head information depends on the sitting height of the occupant 3, not only the head information but also the sitting height information is referred to in order to determine whether the occupant 3 is drowsy. Therefore, compared to the case where whether the occupant 3 is drowsy is determined by referring only to the head information, there is an advantage that the influence of individual differences in the reaction of the occupant 3 to the vibration of the vehicle 2 can be reduced, and the accuracy of determining the drowsiness of the occupant 3 can be improved.

[0083] Moreover, in the drowsiness determination system 100 in the first embodiment, the third acquisition unit 13 estimates the sitting height based on a captured image of the head 31, thereby acquiring the sitting height information.

[0084] This has the advantage that the seated height information can be used without the occupant 3 having to input the seated height information in advance, reducing the effort required by the occupant 3 and making it easier to improve convenience.

[0085] In the drowsiness determination system 100 in the first embodiment, the determination unit 14 compares the head information with a threshold value to determine whether the occupant 3 is drowsy. The determination unit 14 determines a threshold value from a plurality of candidate threshold values ​​based on the sitting height information.

[0086] According to this, since the drowsiness of the occupant 3 can be determined by a simple method of comparing the head information with an appropriate threshold value according to the sitting height information, there is an advantage that the system can be easily designed.

[0087] Furthermore, in the drowsiness determination system 100 in embodiment 1, the head information includes at least one of the delay time Td until the head 31 moves during a predetermined period T1 from the time when vibration of the vehicle 2 occurs, the maximum value of the amplitude of the movement of the head 31 relative to the predetermined position, the average value of the amplitude of the movement of the head 31 relative to the predetermined position, and the variance of the position of the head 31 relative to the predetermined position.

[0088] According to this, since the drowsiness of the occupant 3 can be determined using one or more parameters that can vary depending on the drowsiness of the occupant 3, there is an advantage that the accuracy of determining the drowsiness of the occupant 3 can be easily improved.

[0089] Furthermore, in the drowsiness determination system 100 in the first embodiment, the determination unit 14 resets the counting of the predetermined period T1 when the predetermined period T1 has elapsed or the magnitude of the vibration of the vehicle 2 exceeds a predetermined value.

[0090] This has the advantage that it becomes easier to eliminate fluctuations in the head 31 of the occupant 3 that occur regardless of vibrations of the vehicle 2, and it becomes easier to further improve the accuracy of determining the drowsiness of the occupant 3.

[0091] In addition, in the drowsiness determination system 100 in the first embodiment, the determination unit 14 does not determine whether the occupant 3 is drowsy or not until the magnitude of the vibration of the vehicle 2 exceeds a predetermined value after the timing of the predetermined period T1 is reset.

[0092] This has the advantage that it becomes easier to eliminate fluctuations in the head 31 of the occupant 3 that occur regardless of vibrations of the vehicle 2, and it becomes easier to further improve the accuracy of determining the drowsiness of the occupant 3.

[0093] Furthermore, in the drowsiness determination system 100 in the first embodiment, the determination unit 14 decelerates the vehicle 2 when the time during which no vibration occurs in the vehicle 2 continues for a predetermined time or longer.

[0094] This has the advantage that even when a long period of time continues in which no vibration of the vehicle 2 occurs due to, for example, a continuous flat road surface, it is possible to forcibly generate vibration of the vehicle 2 and determine the drowsiness of the occupant 3. In particular, since the drowsiness of the occupant 3 is likely to increase during a period in which no vibration of the vehicle 2 occurs, there is also the advantage that by determining the drowsiness of the occupant 3 during such a period, it becomes easier to take measures such as encouraging the occupant 3 to wake up.

[0095] Moreover, the drowsiness determination method in the first embodiment includes a first acquisition step ST1, a second acquisition step ST2, a third acquisition step ST3, and a determination step ST4. In the first acquisition step ST1, vibration information indicating vibration of the vehicle 2 is obtained. In the second acquisition step ST2, head information indicating a movement of the head 31 of the occupant 3 is obtained. In the third acquisition step ST3, seated height information indicating the seated height of the occupant 3 is obtained. In the determination step ST4, when the vibration information indicates the occurrence of vibration of the vehicle 2, it is determined whether or not the occupant 3 is drowsy based on the head information and the seated height information.

[0096] According to this, in consideration of the fact that the head information depends on the sitting height of the occupant 3, not only the head information but also the sitting height information is referred to in order to determine whether the occupant 3 is drowsy. Therefore, compared to the case where whether the occupant 3 is drowsy is determined by referring only to the head information, there is an advantage that the influence of individual differences in the reaction of the occupant 3 to the vibration of the vehicle 2 can be reduced, and the accuracy of determining the drowsiness of the occupant 3 can be improved.

[0097] (Embodiment 2) <Configuration> 7 is a block diagram showing an overview of a drowsiness determination system 100A in embodiment 2. The drowsiness determination system 100A in embodiment 2 differs from the drowsiness determination system 100 in embodiment 1 in that the determination unit 14 determines the drowsiness of the occupant 3 using a determination model 141. In the following, mainly only the differences from the drowsiness determination system 100 in embodiment 1 will be described, and a description of the points in common with the drowsiness determination system 100 in embodiment 1 will be omitted.

[0098] The determination unit 14 has a determination model 141 that receives vibration information and head information of the vehicle 2 as input and outputs a determination result as to whether or not the occupant 3 is drowsy. In the second embodiment, the determination model 141 is a trained model that has been machine-learned to output a determination result when the vibration information and head information of the vehicle 2 are input. The determination model 141 is, for example, configured by a neural network having a multi-layer structure, and is a model in which machine learning has been completed to output a determination result as to whether or not the occupant 3 is drowsy in accordance with the vibration information and head information of the vehicle 2 when the vibration information and head information of the vehicle 2 are input. Here, the head information input to the determination unit 14 includes at least one, and preferably two or more, of the delay time Td, the maximum value of the head amplitude, the average value of the head amplitude, and the variance of the head position, which have been described above.

[0099] For example, the determination model 141 is machine-learned by supervised learning using a large number of learning datasets. Each learning dataset includes vibration information and head information of the vehicle 2 as input data (explanatory variables) and drowsiness level (here, two levels indicating whether the occupant 3 is drowsy or not) as ground-truth data (objective variable). Note that the determination model 141 may be a model obtained by multiple regression analysis using, for example, the drowsiness level as the objective variable and the vibration information of the vehicle 2 and the head information as explanatory variables.

[0100] Further, similarly to the first embodiment, the judgment unit 14 determines the judgment model 141 from among a plurality of candidates of the judgment model 141 based on the seated height information acquired by the third acquisition unit 13. Specifically, the judgment unit 14 stores three different judgment models, a first judgment model, a second judgment model, and a third judgment model, according to the seated height of the occupant 3, as candidates of the plurality of judgment models 141. The first judgment model is a model for which machine learning has been completed for the occupant 3 whose seated height is lower than a first predetermined value. The second judgment model is a model for which machine learning has been completed for the occupant 3 whose seated height is between the first predetermined value and the second predetermined value. The third judgment model is a model for which machine learning has been completed for the occupant 3 whose seated height is higher than the second predetermined value.

[0101] Then, if the seated height of the occupant 3 is lower than the first predetermined value, the determination unit 14 determines a first determination model as the determination model 141 to be used when determining the drowsiness of the occupant 3 from among the three candidates of the determination model 141. If the seated height of the occupant 3 is between the first predetermined value and the second predetermined value, the determination unit 14 determines a second determination model as the determination model 141 to be used when determining the drowsiness of the occupant 3 from among the three candidates of the determination model 141. If the seated height of the occupant 3 is higher than the second predetermined value, the determination unit 14 determines a third determination model as the threshold to be used when determining the drowsiness of the occupant 3 from among the three candidates of the determination model 141.

[0102] <Operation> Hereinafter, the operation of the drowsiness determination system 100A in the second embodiment will be described with reference to Fig. 8. Fig. 8 is a flowchart showing an example of the operation of the drowsiness determination system 100A in the second embodiment.

[0103] First, when the vehicle 2 starts traveling, the third acquisition unit 13 acquires the seated height information (S201). The process S201 corresponds to the third acquisition step ST3 in the drowsiness determination method. Then, the determination unit 14 determines the determination model 141 according to the seated height of the occupant 3 from among a plurality of candidates of the determination model 141 by referring to the seated height information acquired by the third acquisition unit 13 (S202). The process S202 corresponds to the determination step ST4 in the drowsiness determination method. Next, the first acquisition unit 11 acquires vibration information (S203). The process S203 corresponds to the first acquisition step ST1 in the drowsiness determination method. Furthermore, the second acquisition unit 12 acquires head information (S204). The process S204 corresponds to the second acquisition step ST2 in the drowsiness determination method. The processes S203 and S204 are constantly executed while the vehicle 2 is traveling.

[0104] The determination unit 14 uses the determination model 141 determined in the process S202 to determine whether the occupant 3 is drowsy (S205). At this time, the determination unit 14 uses the vibration information and the head information to determine whether the occupant 3 is drowsy, including a coincidence determination between the direction of vibration of the vehicle 2 based on the vibration information and the head information and the direction of vibration of the head 31 of the occupant 3. If the determination unit 14 determines that the occupant 3 is drowsy (S206: Yes), the determination unit 14 outputs a notification command to the notification unit 15 to cause the notification unit 15 to perform a notification to prompt the occupant 3 to wake up (S207). On the other hand, if the determination unit 14 determines that the occupant 3 is not drowsy (S206: No), the determination unit 14 does not execute anything in particular. The processes S205 to S207 correspond to the determination step ST4 in the drowsiness determination method. The drowsiness determination system 100A repeats the above series of processes until the vehicle 2 stops traveling (S208: No). Then, when the vehicle 2 stops traveling (S208: Yes), the drowsiness determination system 100A ends the operation.

[0105] <Action and effect> As described above, in the drowsiness determination system 100A in the second embodiment, the determination unit 14 has the determination model 141 that receives vibration information and head information as input and outputs a determination result as to whether or not the occupant 3 is drowsy. The determination unit 14 determines the determination model 141 from a plurality of candidates for the determination model 141 based on the sitting height information.

[0106] This has the advantage that it is possible to expect further improvement in the accuracy of determining drowsiness of the occupant 3 compared to methods of determining drowsiness of the occupant 3 by comparing vibration information with a predetermined value and head information with a threshold value.

[0107] (Other variations) The drowsiness determination system according to the present disclosure has been described above based on the above-mentioned embodiments 1 and 2, but the present disclosure is not limited to these embodiments 1 and 2. Various modifications conceivable by those skilled in the art to embodiments 1 and 2 may also be included within the scope of the present disclosure, as long as they do not deviate from the spirit of the present disclosure.

[0108] In the above-mentioned first embodiment, the determination unit 14 determines whether the occupant 3 is drowsy by comparing the delay time Td as the head information with a threshold value, but this is not limited to the above. For example, the determination unit 14 may determine whether the occupant 3 is drowsy by comparing any one of the maximum value of the head amplitude, the average value of the head amplitude, and the variance of the head position with a threshold value.

[0109] In addition, in the above-described first embodiment, the determination unit 14 may determine whether the occupant 3 is drowsy by comparing each of a plurality of parameters (such as the delay time Td) as head information with a corresponding threshold value. In this case, the determination unit 14 may calculate a drowsiness level for each parameter, and determine whether the occupant 3 is drowsy by referring to the most prevalent drowsiness level or the highest drowsiness level among the plurality of drowsiness levels.

[0110] In the above-mentioned second embodiment, the determination unit 14 determines one determination model 141 from a plurality of mutually different determination model candidates according to the sitting height of the occupant 3, but this is not limited to the above. For example, the determination model 141 may be a trained model in which machine learning has been completed using the sitting height of the occupant 3 as input data (explanatory variable). In this case, it is sufficient for the determination unit 14 to have one determination model 141, and a process of determining the determination model 141 from a plurality of determination model 141 candidates based on the sitting height information is not necessary.

[0111] In each of the above-described embodiments, the third acquisition unit 13 acquires the sitting height information by estimating the sitting height of the occupant 3 based on a captured image of the head 31 of the occupant 3, but is not limited to this. For example, the third acquisition unit 13 may be an input interface that accepts an operation input by the occupant 3 and acquires the sitting height input by the occupant 3 as the sitting height information. In this case, the third acquisition unit 13 does not need to have a function of estimating the sitting height of the occupant 3.

[0112] In each of the above-described embodiments, the determination unit 14 determines the drowsiness of the occupant 3 based on two levels of drowsiness, that is, whether or not the occupant 3 is drowsy, but this is not limited to the above. For example, the determination unit 14 may determine the drowsiness of the occupant 3 based on three or more levels of drowsiness.

[0113] As an example, the determination unit 14 may determine the drowsiness of the occupant 3 based on the head information (e.g., the delay time Td) in five stages of drowsiness levels from "Level 1" to "Level 5". "Level 1" is a level at which the occupant 3 is estimated to be not drowsy at all. "Level 2" is a level at which the occupant 3 is estimated to be slightly drowsy. "Level 3" is a level at which the occupant 3 is estimated to be drowsy. "Level 4" is a level at which the occupant 3 is estimated to be quite drowsy. "Level 5" is a level at which the occupant 3 is estimated to be very drowsy. The determination unit 14 determines that the drowsiness level is higher the longer the delay time Td is.

[0114] When the determination unit 14 determines the drowsiness of the occupant 3 at three or more drowsiness levels as described above, the notification unit 15 may notify in a manner according to the drowsiness level. For example, the notification unit 15 may output a warning sound at a louder volume as the drowsiness level increases. In addition, the notification unit 15 may change the notification content according to the drowsiness level, such as outputting a warning sound such as "beep" when the drowsiness level is low and outputting an alarm sound such as "beep beep" when the drowsiness level is high. Furthermore, the notification unit 15 may combine a means that stimulates a sense other than the hearing of the occupant 3, such as vibration of a seat belt or blowing air from an air conditioner, according to the drowsiness level. In this way, the notification unit 15 can appropriately alert the occupant 3 by notifying according to the drowsiness level.

[0115] In each of the above-described embodiments, the determination unit 14 calculates the delay time Td based on the time when the peak of the vibration of the vehicle 2 occurs and the time when the peak of the vibration of the head 31 of the occupant 3 occurs, but this is not limited to the above. For example, the determination unit 14 may calculate the delay time Td based on the correlation between the vibration of the vehicle 2 and the vibration of the head 31 of the occupant 3. Specifically, when the pattern of the vibration of the vehicle 2 and the pattern of the vibration of the head 31 of the occupant 3 are similar to each other, the determination unit 14 may calculate the delay time Td by finding the difference between the time when the pattern of the vibration of the vehicle 2 occurs (for example, the start time) and the time when the pattern of the vibration of the head 31 of the occupant 3 occurs.

[0116] (Embodiment 3) The drowsiness determination system in embodiment 3 further differs from drowsiness determination system 100 in embodiment 1 in the operation of determining whether or not the occupant 3 is drowsy based on the peak value Vp of the amount of variation in the posture of the vehicle 2 (the magnitude of the vibration information of the vehicle 2). The configuration of the drowsiness determination system in embodiment 3 is similar to the configuration of drowsiness determination system 100 in embodiment 1 shown in Fig. 1. Below, only the differences from drowsiness determination system 100 in embodiment 1 will be mainly described, and a description of the points in common with drowsiness determination system 100 in embodiment 1 will be omitted.

[0117] <Operation> Fig. 9 is a scatter diagram showing the correlation between the drowsiness level of the occupant based on the peak value Vp of the amount of change in the posture of the vehicle 2, the delay time Td, and the seated height of the occupant in the drowsiness determination system in the third embodiment, where (a) is a case where the peak value Vp is small, (b) is a case where the peak value Vp is medium, and (c) is a case where the peak value Vp is large. In (a) to (c) of Fig. 9, the vertical axis represents the drowsiness level of the occupant 3, and the horizontal axis represents the delay time Td. In each figure, the meanings of the first data group surrounded by the dotted line A1, the second data group surrounded by the dashed line A2, and the third data group surrounded by the dashed line A3 are the same as those in Fig. 5 of the first embodiment.

[0118] First, the case where the peak value Vp of the amount of change in the attitude of the vehicle 2 is small will be described with reference to Fig. 9(a). When the peak value Vp is small, the change in the head 31 accompanying the vibration becomes small regardless of the seat height. That is, as shown in Fig. 9(a), a correlation in which the first data group A1 to the third data group A3 are compressed to the left is shown.

[0119] Next, the case where the peak value Vp of the amount of change in the attitude of the vehicle 2 is medium, that is, the peak value Vp is not as small as in Fig. 9(a) but not as large as in Fig. 9(c) to be described later, will be described with reference to Fig. 9(b). The correlation in this case is, for example, the same as in Fig. 5, showing a correlation expanded to the right compared to Fig. 9(a).

[0120] Next, the case where the peak value Vp of the amount of change in the attitude of the vehicle 2 is larger than that in Fig. 9(b) will be described with reference to Fig. 9(c). When the peak value Vp is large, the change in the head 31 accompanying the vibration becomes large regardless of the seat height. That is, as shown in Fig. 9(c), the first data group A1 to the third data group A3 are shifted to the right and show a correlation expanded to the right.

[0121] From these figures, for example, for the occupant 3 in the first data group A1 with a low seat height, the threshold value Tdn (n = 1 to 9) of the delay time Td until the same drowsiness level L1 is reached is the threshold value Td1 in Fig. 9(a), the threshold value Td4 in Fig. 9(b), and the threshold value Td7 in Fig. 9(c). And as is clear from these figures, the relationship of each threshold value Tdn is Td1 < Td4 < Td7. Therefore, even with the same seat height, the threshold value Tdn differs depending on the magnitude of the peak value Vp, and it can be seen that the larger the peak value Vp, the larger the threshold value Tdn. Similarly, the relationship of the threshold value Tdn of the delay time Td in the case where the seat height is in the second data group A2 is Td2 < Td5 < Td8, and the relationship of the threshold value Tdn of the delay time Td in the case where the seat height is in the third data group A3 is Td3 < Td6 < Td9.

[0122] For these reasons, in the third embodiment, the judgment unit 14 stores the correlations of (a), (b), and (c) in Fig. 9 in a memory (not shown), and selects one of the correlations of (a) to (c) in Fig. 9 according to the peak value Vp to determine the threshold value Tdn. In this way, the threshold value Tdn is determined taking into account the effect of the peak value Vp on the delay time Td, so that an error in the delay time Td can be corrected.

[0123] In the above description, the sitting height has three data groups, and the peak value Vp has three levels, large, medium, and small, but the present invention is not limited to this and may have more levels or may have only two levels. In this case, the maximum value of n in the threshold value Tdn will be different from 9.

[0124] Next, a specific operation will be described with reference to Fig. 10. Fig. 10 is a flowchart showing an example of the operation of the drowsiness determination system in the embodiment 3. Note that in Fig. 10, the same operations as those in Fig. 6 of the embodiment 1 are denoted by the same step numbers and detailed description thereof will be omitted.

[0125] First, the operation (S101) in which the third acquisition unit 13 acquires sitting height information is the same as in Fig. 6. As a result, the determination unit 14 determines which data group to use from the first data group A1 to the third data group A3.

[0126] Next, based on the vibration information acquired by the first acquisition unit 11 (corresponding to the first acquisition step ST1 in the drowsiness determination method), the determination unit 14 obtains a peak value Vp of the amount of fluctuation in the posture of the vehicle 2 (S302). In the process S302, for example, the amounts of fluctuation acquired sequentially by the first acquisition unit 11 are compared, and if the current value is lower than the previous value, the previous value is set as the peak value Vp.

[0127] Next, the judgment unit 14 determines a threshold value Tdn (n=1 to 9) (S303) from the data group judged based on the process S101 and the peak value Vp obtained in the process S302. The process S303 corresponds to the judgment step ST4 in the drowsiness judgment method.

[0128] A specific example of process S303 will be described. For example, based on process S101, it is assumed that the sitting height is relatively high from the sitting height information. In this case, the judgment unit 14 judges to use the third data group A3. It is assumed that the peak value Vp is relatively large in the next process S302. In this case, the correlation between the delay time Td and the drowsiness level is as shown in FIG. 9(c). Therefore, the judgment unit 14 refers to the correlation in FIG. 9(c) from the memory. Here, by referring to the third data group A3 in FIG. 9(c), a threshold value Td9 for the delay time Td is determined.

[0129] The subsequent operations are the same as those in Fig. 6 of the first embodiment, except that the threshold value to be compared with the delay time Td in step S108 is the threshold value Tdn (n = 1 to 9) determined in step S303.

[0130] In the above description, the magnitude relationship of the peak value Vp is simply divided into three stages, i.e., when the peak value Vp is small, when the peak value Vp is medium, and when the peak value Vp is large. However, the range of the peak value Vp (the magnitude of the vibration information) in which the determination unit 14 determines that the peak value Vp is small may be wider than the range in which the peak value Vp is determined to be large. A specific description will be given with reference to Fig. 11. Fig. 11 is a correlation diagram between the peak value Vp of the amount of change in the posture of the vehicle and the delay time Td in the drowsiness determination system in the third embodiment. In Fig. 11, the horizontal axis indicates the peak value Vp, and the vertical axis indicates the delay time Td.

[0131] As shown in Fig. 11, at any seat height, as the peak value Vp caused by the vibration of the vehicle 2 increases, the delay time Td with respect to the movement of the head 31 also increases, but the relationship between the two is often nonlinear. That is, while the peak value Vp is small, the delay time Td does not change much even if the peak value Vp increases, but as the peak value Vp increases, the delay time Td increases sharply. For this reason, when determining the magnitude relationship of the peak value Vp, the range in which the determination unit 14 determines that the peak value Vp is small may be made wider than the range in which the peak value Vp is determined to be large, as shown in Fig. 11. This allows the magnitude relationship of the peak value Vp to be determined more practically, thereby improving the accuracy of the final drowsiness determination.

[0132] <Action and effect> As described above, in the drowsiness determination system in the third embodiment, the determination unit 14 determines whether or not the occupant 3 is drowsy, further based on the magnitude (peak value Vp) of the vibration information.

[0133] According to this, since an error in the delay time Td based on the magnitude (peak value Vp) of the vibration information of the vehicle 2 can be corrected, there is an advantage that drowsiness determination with higher accuracy is possible.

[0134] In addition, in the drowsiness determination system of embodiment 3, when the determination unit 14 determines whether the occupant 3 is drowsy, the range in which the vibration information is determined to be small (peak value Vp), i.e., the range in which the peak value Vp is small, is wider than the range in which the vibration information is determined to be large, i.e., the range in which the peak value Vp is large.

[0135] According to this, the delay time Td between the movement of the head 31 in response to the actual vibration of the vehicle 2 does not vary much when the vibration of the vehicle 2 is small, but increases sharply as the vibration of the vehicle 2 increases. Therefore, since the magnitude of the vibration information (peak value Vp) can be determined taking into account such nonlinear behavior, there is an advantage that drowsiness can be determined with even higher accuracy.

[0136] In the drowsiness determination method in the third embodiment, the determination step further determines whether or not the occupant 3 is drowsy, based on the magnitude (peak value Vp) of the vibration information.

[0137] According to this, since an error in the delay time Td based on the magnitude (peak value Vp) of the vibration information of the vehicle 2 can be corrected, there is an advantage that drowsiness determination with higher accuracy is possible.

[0138] The configuration of the third embodiment may be applied to a configuration using the determination model 141 of the second embodiment. In this case, the determination model 141 may be a model that has been subjected to machine learning so as to output a determination result when vibration information including the magnitude (peak value Vp) of the vibration information of the vehicle 2 and head information are input.

[0139] (Embodiment 4) The drowsiness determination system in the fourth embodiment differs from the drowsiness determination system in the third embodiment in that the drowsiness determination system further calculates a representative value Pe of the eyelid position of the occupant 3 within a predetermined period from information on the head 31 based on an image of the head 31, and determines whether the occupant 3 is drowsy by taking the representative value Pe of the eyelid position into consideration. In the following, it is described that the representative value Pe of the eyelid position of the occupant 3 within a predetermined period is the average eyelid position of the occupant 3 within the predetermined period. The configuration of the drowsiness determination system in the fourth embodiment is similar to that of the drowsiness determination system 100 in the first embodiment shown in FIG. 1. In the following, mainly differences from the drowsiness determination system in the third embodiment will be described, and descriptions of points in common with the drowsiness determination systems in the first and third embodiments will be omitted.

[0140] <Operation> Fig. 12 is a flowchart showing an example of the operation of the drowsiness determination system in the embodiment 4. Note that in Fig. 12, the same operations as those in Fig. 10 of the embodiment 3 are denoted by the same step numbers and detailed description thereof will be omitted.

[0141] First, steps S101 to S107 are the same as those in FIG.

[0142] Next, the determination unit 14 extracts an eyelid image from the image of the head 31 of the occupant 3 acquired by the second acquisition unit 12 using the camera 22 (S401). The process S401 corresponds to the determination step ST4 in the drowsiness determination method.

[0143] Next, the determination unit 14 calculates an average eyelid position Pe by averaging the eyelid positions in a preset period (S402). Specifically, the preset period is, for example, 1 second, and the determination unit 14 obtains and averages the eyelid positions from the images every 0.1 seconds, for example. The eyelid position is defined as the degree of eye opening, where open eyes are 100% and closed eyes are 0%. Note that the process S402 corresponds to the determination step ST4 in the drowsiness determination method.

[0144] Next, the determination unit 14 first determines head sleepiness LTd based on the delay time Td determined in step S107. As described above in "Other Modifications," the head sleepiness LTd determines the drowsiness of the occupant 3 as one of five drowsiness levels, "Level 1" to "Level 5," based on the delay time Td. The determination method is as described above. As a result, the head sleepiness LTd becomes one of the values ​​1 to 5.

[0145] Similarly, the determination unit 14 calculates eyelid sleepiness LPe based on the average eyelid position Pe calculated in step S402. The eyelid sleepiness LPe is correlated with the average eyelid position Pe at a certain percentage, which corresponds to a sleepiness level of 1 to 5. Therefore, the eyelid sleepiness LPe is also a value of 1 to 5.

[0146] Next, the determination unit 14 calculates the final sleepiness level Li by weighting the head sleepiness LTd and the eyelid sleepiness LPe (S403). The process S403 corresponds to the determination step ST4 in the sleepiness determination method.

[0147] A specific example of the drowsiness level Li will be described. For example, the weighting is set so that the influence of eyelid drowsiness LPe is 70% and the influence of head drowsiness LTd is 30% for the final drowsiness level Li. The weighting coefficient (%) is determined in advance by an experiment or the like. Thus, the function for calculating drowsiness L is defined as L=0.7*LPe+0.3*LTd. Drowsiness L can be calculated by substituting head drowsiness LTd and eyelid drowsiness LPe into this formula. However, since drowsiness L is a numerical value including a decimal point, the drowsiness level Li (a numerical value between 1 and 5) can be calculated by rounding off the decimal point of drowsiness L. Since the drowsiness level Li is calculated in this way, the influence of eyelid drowsiness LPe, which directly indicates drowsiness, on the drowsiness level Li becomes large, and the accuracy of drowsiness judgment is further increased.

[0148] Next, the determination unit 14 determines whether the drowsiness level Li is 2 or more (S404). Process S404 corresponds to determination step ST4 in the drowsiness determination method. If the drowsiness level Li is 2 or more, drowsiness including early drowsiness is present, and the process proceeds to process S109. On the other hand, if the drowsiness level Li is less than 2, that is, 1, drowsiness is not present, and the process proceeds to process S111. By performing such processes, a drowsiness determination system capable of notifying early drowsiness can be realized.

[0149] In addition, if the occupant 3 is wearing glasses and it is difficult to identify the eyelid position, the weighting may be changed. For example, if the type of glasses is tinted sunglasses, or if the type of glasses is glasses that easily reflect external light, etc., a situation may occur in which it is difficult to identify the eyelid position. As a specific example, if the determination unit 14 determines from the image of the camera 22 that the occupant 3 is wearing tinted sunglasses, the weighting of eyelid drowsiness LPe is lowered and the weighting of head drowsiness LTd is increased accordingly to obtain drowsiness L. For example, the above weighting is reversed and the function for obtaining drowsiness L is defined as L=0.3*LPe+0.7*LTd. This allows the error of eyelid drowsiness LPe due to sunglasses to be covered by head drowsiness LTd, so that the accuracy of drowsiness determination can be improved. The same applies to the case where it is determined that the occupant 3 is wearing glasses that easily reflect external light.

[0150] Regarding the variation of the weighting due to sunglasses, the determination unit 14 may weight the eyelid drowsiness LPe lower as the sunglasses become darker, according to the shade of the sunglasses recognized from the image of the camera 22. This allows the drowsiness determination to be performed so as to reduce the influence of the eyelid drowsiness LPe, which tends to become unclear as the sunglasses become darker, thereby improving the accuracy of the drowsiness determination.

[0151] The weighting coefficients described above are all examples, and a function may be defined using a correlation based on experiments with eyelid sleepiness LPe and head sleepiness LTd as parameters and sleepiness L. Also, the correlation between eyelid sleepiness LPe and head sleepiness LTd and the sleepiness level Li determined by experiments may be stored in memory as a table.

[0152] <Action and effect> As described above, in the drowsiness determination system of embodiment 4, the determination unit 14 further obtains a representative value Pe of the eyelid position of the occupant 3 within a specified period from head information based on an image of the head 31, and determines whether the occupant 3 is drowsy by taking into account the representative value Pe of the eyelid position.

[0153] According to this, since the representative value Pe of the eyelid position is also used in the drowsiness judgment, there is an advantage that the drowsiness judgment can be made with higher accuracy.

[0154] In addition, in the drowsiness determination system of embodiment 4, the determination unit 14 determines head drowsiness LTd based on head information and sitting height information, and eyelid drowsiness LPe based on the representative value Pe of the eyelid position, and weights the head drowsiness LTd and eyelid drowsiness LPe to determine the drowsiness L of the occupant 3.

[0155] According to this, for example, by making the weight of eyelid sleepiness LPe, which represents direct drowsiness, greater than that of head sleepiness LTd, the influence of eyelid sleepiness LPe on overall drowsiness L can be increased, which has the advantage of further increasing the accuracy of drowsiness judgment.

[0156] Furthermore, in the drowsiness determination system of embodiment 4, the determination unit 14 determines whether or not the occupant 3 is wearing glasses based on the head information, and if the occupant 3 is wearing glasses, the type of glasses, and determines the occupant's drowsiness by weighting the head drowsiness LTd and eyelid drowsiness LPe according to the determination result.

[0157] This has the advantage that the error in eyelid drowsiness LPe due to whether or not glasses are worn, and, if glasses are worn, the type of glasses, can be compensated for by the head drowsiness LTd, thereby improving the accuracy of drowsiness determination.

[0158] Furthermore, in the drowsiness determination system of embodiment 4, if the occupant 3 is not wearing sunglasses, the determination unit 14 increases the weighting of eyelid drowsiness LPe relative to head drowsiness LTd, and decreases the weighting of eyelid drowsiness LPe relative to head drowsiness LTd if the occupant 3 is wearing sunglasses.

[0159] This has the advantage that the error in eyelid drowsiness LPe due to sunglasses can be compensated for by head drowsiness LTd, thereby improving the accuracy of drowsiness determination.

[0160] Furthermore, in the drowsiness determination method of embodiment 4, the determination step further calculates a representative value Pe of the eyelid position of the occupant 3 within a specified period from head information based on an image of the head 31, and determines whether the occupant 3 is drowsy by taking into account the representative value Pe of the eyelid position.

[0161] According to this, since the representative value Pe of the eyelid position is also used in the drowsiness judgment, there is an advantage that the drowsiness judgment can be made with higher accuracy.

[0162] In addition, in embodiment 3, the representative value Pe of the eyelid position of occupant 3 within a specified period is the average eyelid position of occupant 3 within a specified period, but is not limited to this and may be, for example, the median or mode of the eyelid position of occupant 3 within a specified period.

[0163] In addition, in embodiment 4, the representative eyelid position value Pe is taken into account in the configuration of embodiment 3 to determine whether or not the occupant 3 is drowsy, but this configuration is not limited to this, and for example, the configuration of embodiment 4 may be applied to embodiments 1 and 2.

[0164] (Modification) Although the drowsiness determination system according to the present disclosure has been described based on the above-mentioned embodiments 1 to 4, the present disclosure is not limited to these embodiments 1 to 4. As long as it does not deviate from the gist of the present disclosure, various modifications conceivable by a person skilled in the art may also be included in the scope of the present disclosure.

[0165] In each of the above-described embodiments, the vibration direction of the vehicle 2 is exemplified as the pitch direction with respect to the road surface, but is not limited thereto. For example, the vibration direction of the vehicle 2 may be the roll direction or the yaw direction with respect to the road surface.

[0166] Each unit included in the drowsiness determination system according to each of the above embodiments is typically realized as an LSI, which is an integrated circuit. These units may be individually implemented as single chips, or may be integrated into a single chip so as to include some or all of the units.

[0167] The integrated circuit is not limited to an LSI, but may be realized by a dedicated circuit or a general-purpose processor. A field programmable gate array (FPGA) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connections and settings of the circuit cells inside the LSI may also be used.

[0168] In each of the above embodiments, each component may be implemented by dedicated hardware or by executing a software program suitable for each component. Each component may be implemented by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0169] Furthermore, all the numbers used above are merely examples for specifically explaining the present disclosure, and each embodiment of the present disclosure is not limited to the exemplified numbers.

[0170] In addition, the division of functional blocks in the block diagram is an example, and multiple functional blocks may be realized as one functional block, one functional block may be divided into multiple blocks, or some functions may be transferred to another functional block. Furthermore, the functions of multiple functional blocks having similar functions may be processed in parallel or in a time-sharing manner by a single piece of hardware or software.

[0171] In addition, the order in which each step is performed in the flowchart is merely for illustrative purposes and may be other than the above. In addition, some of the steps may be performed simultaneously (in parallel) with other steps.

[0172] In addition, this disclosure also includes forms obtained by applying various modifications to each embodiment that a person skilled in the art may think of, and forms realized by arbitrarily combining the components and functions of the embodiments within the scope that does not deviate from the spirit of this disclosure. [Industrial Applicability]

[0173] The present disclosure can be used in a system for assisting driving of a moving object such as a vehicle. [Explanation of symbols]

[0174] 100,100A Drowsiness Judgment System 11 First acquisition part 12 Second acquisition part 13 Third acquisition part 14 Judgment section 141 Decision Model 2 Vehicles 3 Crew 31 Head ST1 First acquisition step ST2 Second acquisition step ST3 Third acquisition step ST4 Judgment step T1 Predetermined period Td Delay time Vp Peak value Tdn Threshold Pe average eyelid position LTd Head drowsiness LPe Eyelid drowsiness L. Sleepiness Li Drowsiness level

Claims

1. a first acquisition unit that acquires vibration information indicating a vibration of the vehicle, the vibration information being an amount of variation in the attitude of the vehicle with respect to a road surface; A second acquisition unit that acquires head information indicating a movement of the occupant's head; A third acquisition unit that acquires seated height information indicating a seated height of the occupant; a determination unit that determines whether the occupant is sleepy based on the head information, the sitting height information, and the magnitude of the vibration information, The determination unit, in determining whether the occupant is drowsy, determining a magnitude relationship between peak values ​​from the vibration information, and determining a threshold value from the magnitude relationship between the sitting height information and the peak values; When the vibration information exceeds a predetermined value, it is determined that vibration has occurred in the vehicle, and when the head of the occupant moves in the same direction as the direction of change in the vehicle posture, the head information is compared with the threshold value to determine drowsiness of the occupant; In the determination of the magnitude relationship, determining that the peak value is small when the peak value is equal to or smaller than a first upper limit value, and determining that the peak value is large when the peak value is greater than a second upper limit value and equal to or smaller than a third upper limit value; the second upper limit value is greater than the first upper limit value, and the third upper limit value is greater than the second upper limit value; The first upper limit value is greater than a difference between the second upper limit value and the third upper limit value. Drowsiness assessment system.

2. the third acquisition unit calculates coordinates of a feature point of the head from an image of the head, and calculates a difference in height between a coordinate indicating a position of the seating surface of the occupant and the coordinate of the feature point of the head, thereby estimating the sitting height and acquiring the sitting height information. The drowsiness determination system according to claim 1 .

3. the determination unit has a determination model that receives the vibration information and the head information as input and outputs a determination result of whether the occupant is drowsy, The determination unit determines the determination model from a plurality of determination model candidates based on the sitting height information. The drowsiness determination system according to claim 1 .

4. The head information includes at least one of a delay time until the head moves during a predetermined period from the time when the vibration of the vehicle occurs, a maximum value of the amplitude of the head movement relative to a predetermined position, an average value of the amplitude of the head movement relative to the predetermined position, and a variance of the head position relative to the predetermined position. The drowsiness determination system according to any one of claims 1 to 3.

5. The determination unit resets the counting of the predetermined period when the predetermined period has elapsed or when the magnitude of the vibration of the vehicle exceeds a predetermined value. The drowsiness determination system according to claim 4 .

6. The determination unit does not determine whether the occupant is drowsy during a period from when the time measurement of the predetermined period is reset until the magnitude of the vibration of the vehicle exceeds the predetermined value. The drowsiness determination system according to claim 5 .

7. The determination unit decelerates the vehicle when a period during which no vibration is occurring in the vehicle continues for a predetermined period of time or longer. The drowsiness determination system according to any one of claims 1 to 6.

8. The determination unit further obtains a representative value of an eyelid position of the occupant within a predetermined period from the head information based on an image of the head, and determines whether the occupant is drowsy by taking the representative value of the eyelid position into account. The drowsiness determination system according to any one of claims 1 to 7.

9. The determination unit determines head drowsiness based on the head information, the sitting height information, and the magnitude of the vibration information, and eyelid drowsiness by comparing the representative value of the eyelid position with the eyelid position when the eyes are open, and determines the drowsiness of the occupant by weighting the head drowsiness and the eyelid drowsiness. The drowsiness determination system according to claim 8 .

10. The determination unit determines whether the occupant is wearing glasses based on the head information, and if the occupant is wearing glasses, the type of glasses, and determines the drowsiness of the occupant by weighting the head drowsiness and the eyelid drowsiness according to the determination result. The drowsiness determination system according to claim 9 .

11. The determination unit increases a weighting of the eyelid sleepiness relative to the head sleepiness when the occupant is not wearing sunglasses, and decreases a weighting of the eyelid sleepiness relative to the head sleepiness when the occupant is wearing sunglasses. The drowsiness determination system according to claim 10.

12. a first acquisition step of acquiring vibration information indicating vibration of the vehicle, the vibration information being a variation in the attitude of the vehicle with respect to a road surface; a second acquisition step of acquiring head information indicating a movement of the occupant's head; a third acquisition step of acquiring seated height information indicating a seated height of the occupant; and determining whether or not the occupant is drowsy based on the head information, the sitting height information, and the magnitude of the vibration information, In the determination step, determining a magnitude relationship between peak values ​​from the vibration information, and determining a threshold value from the magnitude relationship between the sitting height information and the peak values; When the vibration information exceeds a predetermined value, it is determined that vibration has occurred in the vehicle, and when the head of the occupant moves in the same direction as the direction of change in the vehicle posture, the head information is compared with the threshold value to determine drowsiness of the occupant; In the determination of the magnitude relationship, determining that the peak value is small when the peak value is equal to or smaller than a first upper limit value, and determining that the peak value is large when the peak value is greater than a second upper limit value and equal to or smaller than a third upper limit value; the second upper limit value is greater than the first upper limit value, and the third upper limit value is greater than the second upper limit value; The first upper limit value is greater than a difference between the second upper limit value and the third upper limit value. Methods for assessing drowsiness.

13. In the determination step, a representative value of an eyelid position of the occupant within a predetermined period is obtained from the head information based on an image of the head, and whether or not the occupant is drowsy is determined by taking the representative value of the eyelid position into consideration. The method for determining drowsiness according to claim 12.

Images