Occupant sleep abnormality detection system, occupant sleep abnormality detection device, and occupant sleep abnormality detection method

Abstract

This occupant sleep abnormality detection system comprises: an occupant sleep state detection device (11) that detects information about the sleep state of a vehicle occupant; an occupant breathing state detection device (12) that detects information about the breathing state of the vehicle occupant; and an occupant sleep abnormality detection device (13). The occupant sleep abnormality detection device (13) comprises: a sleep state detection unit (131) that detects the sleep state of the occupant on the basis of the result of the detection by the occupant sleep state detection device (11); a breathing state acquisition unit (132) that acquires the information about the breathing state of the occupant on the basis of the result of the detection by the occupant breathing state detection device (12); and a sleep abnormality detection unit (133) that detects whether or not the occupant is having a sleep abnormality on the basis of the sleep state of the occupant detected by the sleep state detection unit (131) and the information about the breathing state of the occupant acquired by the breathing state acquisition unit (132).

Description

Occupant sleep abnormality detection system, occupant sleep abnormality detection device, and occupant sleep abnormality detection method

[0001] The present disclosure relates to an occupant sleep abnormality detection system, an occupant sleep abnormality detection device, and an occupant sleep abnormality detection method for detecting an occupant's sleep abnormality.

[0002] A conventional occupant support method has been disclosed that detects the sleep state of an occupant and adjusts the interior environment of the vehicle to allow for comfortable sleep (see, for example, Patent Document 1). Patent Document 1 discloses a method of darkening the interior of the vehicle to allow for comfortable sleep.

[0003] Japanese Patent Application Laid-Open No. 2023-055197

[0004] While the conventional technology has been able to make sleeping occupants sleep more comfortably as described above, it does not detect sleep abnormalities such as apneic sleep, and therefore does not contribute to resolving sleep abnormalities in occupants.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to provide an occupant sleep abnormality detection system that is capable of detecting occupant sleep abnormalities.

[0006] The occupant sleep abnormality detection system of the present disclosure comprises an occupant sleep state detection device that detects information regarding the sleep state of an occupant of a mobile body, an occupant respiratory state detection device that detects information regarding the respiratory state of an occupant of the mobile body, and an occupant sleep abnormality detection device, and the occupant sleep abnormality detection device is characterized by comprising a sleep state detection unit that detects the sleep state of the occupant based on the detection result by the occupant sleep state detection device, a respiratory state acquisition unit that acquires information regarding the respiratory state of the occupant based on the detection result by the occupant respiratory state detection device, and a sleep abnormality detection unit that detects whether the occupant has a sleep abnormality based on the sleep state of the occupant detected by the sleep state detection unit and the information regarding the respiratory state of the occupant acquired by the respiratory state acquisition unit.

[0007] According to the present disclosure, the above-described configuration makes it possible to detect sleep abnormalities in occupants.

[0008] 3A and 3B are diagrams illustrating an example of detection of an apnea sleep state by the occupant sleep abnormality detection device according to embodiment 1. FIG. 3A is a diagram illustrating an example of an occupant's respiratory signal acquired by the respiratory state acquisition unit, and FIG. 3B is a diagram illustrating an example of the presence or absence of apnea of ​​the occupant detected by the sleep abnormality detection unit. FIG. 3B is a flowchart illustrating another example of operation of the occupant sleep abnormality detection device according to embodiment 1. FIG. 3A is a diagram illustrating an example of a configuration of an occupant sleep abnormality detection system including an occupant sleep abnormality detection device according to embodiment 2. FIG. 3B is a flowchart illustrating an example of operation of the occupant sleep abnormality detection device according to embodiment 2. FIG. 3B is a diagram illustrating an example of a configuration of an occupant sleep abnormality detection system including an occupant sleep abnormality detection device according to embodiment 3. FIG. 3C is a flowchart illustrating an example of operation of the occupant sleep abnormality detection device according to embodiment 3. FIG. 3D is a diagram illustrating an example of a configuration of an occupant sleep abnormality detection system including an occupant sleep abnormality detection device according to embodiment 4. FIG. 3D is a flowchart illustrating an example of operation of the occupant sleep abnormality detection device according to embodiment 4. FIG. 3E is a diagram illustrating an example of operation of the occupant sleep abnormality detection device according to embodiment 4, showing an image of a notification screen. 12A and 12B are diagrams illustrating an example of the hardware configuration of an occupant sleep abnormality detection device according to the first to fourth embodiments.

[0009] Hereinafter, embodiments will be described in detail with reference to the drawings. Embodiment 1. Fig. 1 is a diagram showing an example of the configuration of an occupant sleep abnormality detection system 1 including an occupant sleep abnormality detection device 13 according to embodiment 1. The occupant sleep abnormality detection system 1 is a system for detecting sleep abnormalities of occupants of a moving body such as a vehicle. This occupant sleep abnormality detection system 1 includes, for example, an occupant sleep state detection device 11, an occupant respiratory state detection device 12, and an occupant sleep abnormality detection device 13, as shown in Fig. 1.

[0010] The occupant sleep state detecting device 11 detects information related to the sleep state of the occupant. For example, the occupant sleep state detecting device 11 has an imaging unit capable of acquiring an image of the occupant, and an image processing unit capable of detecting information related to the sleep state of the occupant based on the image acquired by the imaging unit.

[0011] Here, the occupant sleep state detecting device 11 may detect information that enables detection of whether the occupant is asleep as the information regarding the occupant's sleep state. Examples of this information include the degree of eye opening of the occupant or the inclination of the occupant's body. Furthermore, the occupant sleep state detecting device 11 may detect whether the occupant is asleep as the information regarding the occupant's sleep state.

[0012] Data indicating the detection result by the occupant sleep state detection device 11 is output to the occupant sleep abnormality detection device 13 .

[0013] The occupant respiratory condition detection device 12 detects information related to the occupant's respiratory condition. For example, a millimeter wave radar is used as this occupant respiratory condition detection device 12. When the occupant respiratory condition detection device 12 is a millimeter wave radar, the occupant respiratory condition detection device 12 detects, for example, a respiratory signal indicating expansion and contraction of the occupant's chest as information related to the occupant's respiratory condition.

[0014] The occupant respiratory condition detecting device 12 may also detect the occupant's pulse rate, blood pressure, or respiratory rate as information related to the occupant's respiratory condition. The occupant respiratory condition detecting device 12 may also detect information related to the occupant's surrounding environment, such as oxygen concentration, instead of information related to the occupant's own condition, as information related to the occupant's respiratory condition. The occupant respiratory condition detecting device 12 may also detect multiple types of information from the above (respiratory signal, pulse rate, blood pressure, respiratory rate, or oxygen concentration) as information related to the occupant's respiratory condition.

[0015] Data indicating the detection result by the occupant respiratory condition detection device 12 is output to the occupant sleep abnormality detection device 13.

[0016] The occupant sleep abnormality detection device 13 detects whether or not the occupant has a sleep abnormality based on information related to the occupant's sleep state detected by the occupant sleep state detection device 11 and information indicating the occupant's respiratory state detected by the occupant respiratory state detection device 12. The occupant sleep abnormality detection device 13 includes, for example, a sleep state detection unit 131, a respiratory state acquisition unit 132, and a sleep abnormality detection unit 133, as shown in FIG.

[0017] The sleeping state detection unit 131 detects whether the occupant is asleep or not based on the detection result by the occupant sleeping state detection device 11. Data indicating the detection result by the sleeping state detection unit 131 is output to the sleeping abnormality detection unit 133.

[0018] The respiratory condition acquisition unit 132 acquires information about the respiratory condition of the occupant based on the detection result by the occupant respiratory condition detection device 12. Data indicating the acquisition result by the respiratory condition acquisition unit 132 is output to the sleep abnormality detection unit 133.

[0019] The sleep abnormality detection unit 133 detects whether or not the occupant has a sleep abnormality based on the sleeping state of the occupant detected by the sleeping state detection unit 131 and information related to the respiratory state of the occupant acquired by the respiratory state acquisition unit 132. In this case, the sleep abnormality detection unit 133 detects whether or not the occupant has a sleep abnormality by monitoring information related to the respiratory state acquired by the respiratory state acquisition unit 132 for the occupant detected to be asleep by the sleeping state detection unit 131.

[0020] The types of sleep abnormality detected by the sleep abnormality detection unit 133 include, for example, apnea sleep, cardiopulmonary collapse, sleep paralysis, teeth grinding, and mental stress.

[0021] For example, the symptom of apneic sleep can be determined as a state in which an apneic state continues for several seconds per hour. Therefore, for example, the sleep abnormality detection unit 133 may detect an apneic sleep state when an apneic state lasting several seconds is detected based on information on the occupant's respiratory state acquired by the respiratory state acquisition unit 132.

[0022] Furthermore, for example, the sleep abnormality detection unit 133 may detect an apnea sleep state when detecting a state in which the occupant takes a quick, deep breath after escaping from an apnea state, based on information about the occupant's respiratory state acquired by the respiratory state acquisition unit 132. Note that in this case, even if the apnea state lasts for less than the above-mentioned several seconds, the sleep abnormality detection unit 133 may detect an apnea sleep state when detecting a state in which the occupant takes a quick, deep breath after escaping from an apnea state. Furthermore, the sleep abnormality detection unit 133 may detect the "state in which the occupant takes a quick, deep breath" from captured images or audio information, which will be described later, acquired as information about the respiratory state by the respiratory state acquisition unit 132.

[0023] Furthermore, for example, the sleep abnormality detection unit 133 may detect an apneic sleep state when an apneic state lasting several seconds is detected a predetermined number of times during one sleep session based on information about the occupant's respiratory state acquired by the respiratory state acquisition unit 132. An example of the predetermined number of times is five times. For an occupant who has previously experienced apneic sleep symptoms, the predetermined number of times may be less than five.

[0024] Furthermore, for example, the sleep abnormality detection unit 133 may detect an apneic sleep state when a hypoventilation state is detected based on information on the respiratory state of the occupant acquired by the respiratory state acquisition unit 132. The hypoventilation state is a state in which the respiratory volume is half or less of that of normal breathing.

[0025] Furthermore, for example, the sleep abnormality detection unit 133 may detect that the occupant is in an apneic sleep state when it detects that a hypopneic state has continued for a predetermined period of time based on information about the occupant's respiratory state acquired by the respiratory state acquisition unit 132. The predetermined period of time may be, for example, 15 minutes. Furthermore, for an occupant who has previously experienced apneic sleep symptoms, the predetermined period of time may be shorter than 15 minutes.

[0026] Furthermore, for example, the sleep abnormality detection unit 133 may detect a state of cardiopulmonary collapse when it detects that the occupant is breathing weakly or that irregular breathing has continued for a predetermined period of time based on the information on the occupant's breathing state acquired by the breathing state acquisition unit 132. The predetermined period may be, for example, five minutes. Note that an example of a method for detecting an irregular breathing state is a detection method using pattern recognition.

[0027] Furthermore, valvular heart disease and the like can cause shortness of breath. Therefore, for example, the sleep abnormality detection unit 133 may detect a state of cardiopulmonary collapse when it detects shortness of breath based on information about the occupant's respiratory state acquired by the respiratory state acquisition unit 132. Note that the sleep abnormality detection unit 133 may detect "shortness of breath" from captured images or audio information, which will be described later, acquired by the respiratory state acquisition unit 132 as information about the respiratory state.

[0028] Furthermore, for example, the sleep abnormality detection unit 133 may detect that the occupant is in a state of paralysis when it detects that weak breathing has continued for a predetermined period of time based on information regarding the occupant's breathing state acquired by the breathing state acquisition unit 132.

[0029] Furthermore, for example, the sleep abnormality detection unit 133 may detect that the occupant is in a state of teeth grinding or mental stress when it detects that the pulse rate and blood pressure have risen above a predetermined threshold based on information regarding the occupant's respiratory state acquired by the respiratory state acquisition unit 132.

[0030] Furthermore, for example, the sleep abnormality detection unit 133 may detect that the occupant is in a state of teeth grinding or mental stress when it detects that the respiratory rate has increased above a predetermined threshold based on information regarding the occupant's respiratory state acquired by the respiratory state acquisition unit 132.

[0031] If the sleep abnormality detection unit 133 detects that a sleep abnormality has occurred in the occupant, it outputs information related to the sleep abnormality of the occupant. Specific examples of how the information related to the sleep abnormality output by the sleep abnormality detection unit 133 is used will be described in the second to fourth embodiments below.

[0032] Note that the occupant to be detected by the occupant sleep abnormality detection system 1 according to the first embodiment is not limited to the driver, and may be an occupant other than the driver. Furthermore, when the occupant to be detected is the driver, the occupant sleep abnormality detection system 1 desirably controls the driver so as not to fall asleep in a situation other than when the vehicle in which the driver is riding is performing fully automated driving. On the other hand, when the vehicle is performing fully automated driving, the occupant sleep abnormality detection system 1 may allow the driver to fall asleep.

[0033] The imaging unit included in the occupant sleep state detecting device 11 may be a single wide-angle imaging unit capable of detecting the sleep states of all occupants, or two imaging units may be provided in the vehicle, one for the front seat occupants and one for the rear seat occupants, or imaging units corresponding in number to the number of seats may be provided in the vehicle so that each occupant can be detected individually. The occupant respiratory state detecting device 12 may also be a single unit or multiple units may be provided in the vehicle.

[0034] Next, an example of the operation of the occupant sleep abnormality detection device 13 according to the first embodiment shown in Fig. 1 will be described with reference to Fig. 2. The occupant sleep state detection device 11 detects information related to the occupant's sleep state and outputs data indicating the information to the occupant sleep abnormality detection device 13. The occupant respiratory state detection device 12 detects information related to the occupant's respiratory state and outputs data indicating the information to the occupant sleep abnormality detection device 13.

[0035] 2, in an example of the operation of the occupant sleep abnormality detection device 13 according to the first embodiment shown in Fig. 1, first, the sleep state detection unit 131 detects whether or not the occupant is asleep based on the detection result by the occupant sleep state detection device 11 (step ST101). For example, when the vehicle starts traveling, the sleep state detection unit 131 acquires information on the occupant's sleep state from the occupant sleep state detection device 11 and detects whether or not the occupant is asleep.

[0036] In addition, when the occupant sleep state detection device 11 detects whether the occupant is asleep or not as information regarding the occupant's sleep state, the sleep state detection unit 131 can use the output result of the occupant sleep state detection device 11 as the detection result as is.

[0037] Next, the respiratory condition acquisition unit 132 acquires information about the occupant's respiratory condition based on the detection result by the occupant respiratory condition detection device 12 (step ST102). For example, when the vehicle starts traveling, the respiratory condition acquisition unit 132 acquires information about the occupant's respiratory condition from the occupant respiratory condition detection device 12.

[0038] Next, the sleep abnormality detection unit 133 detects whether or not the occupant has a sleep abnormality based on the sleeping state of the occupant detected by the sleeping state detection unit 131 and the information on the respiratory state of the occupant acquired by the respiratory state acquisition unit 132 (step ST103). At this time, the sleep abnormality detection unit 133 detects whether or not the occupant has a sleep abnormality by monitoring the information on the respiratory state acquired by the respiratory state acquisition unit 132 for the occupant detected to be asleep by the sleeping state detection unit 131.

[0039] For example, the sleep abnormality detection unit 133 may detect an apnea sleep state when an apnea state lasting several seconds is detected based on the information about the occupant's respiratory state acquired by the respiratory state acquisition unit 132. Furthermore, for example, the sleep abnormality detection unit 133 may detect an apnea sleep state when a state in which the occupant takes a quick, deep breath after escaping from an apnea state is detected based on the information about the occupant's respiratory state acquired by the respiratory state acquisition unit 132. Furthermore, for example, the sleep abnormality detection unit 133 may detect an apnea sleep state when an apnea state lasting several seconds is detected a predetermined number of times during one sleep session based on the information about the occupant's respiratory state acquired by the respiratory state acquisition unit 132.

[0040] 3A and 3B are diagrams illustrating an example of the operation of the occupant sleep abnormality detection device 13 according to the first embodiment, illustrating an image of detecting an apneic sleep state. Fig. 3A shows an example of the occupant's respiratory signal acquired by the respiratory state acquisition unit 132, and Fig. 3B shows an example of the presence or absence of apnea in the occupant detected by the sleep abnormality detection unit 133. In the example of Fig. 3 , when the sleep abnormality detection unit 133 detects that the respiratory signal is equal to or less than a first threshold value as indicated by reference numeral 101 for several seconds and then becomes equal to or greater than a second threshold value as indicated by reference numeral 102, the sleep abnormality detection unit 133 determines that the occupant is in an apneic state by determining that an abnormally deep breath has been taken in after the apnea. The first threshold value is set to a value capable of detecting an apneic state, and the second threshold value is set to a value greater than the first threshold value and capable of detecting a deep breath.

[0041] Furthermore, for example, the sleep abnormality detection unit 133 may detect an apnea sleep state when it detects a hypoventilation state based on information about the occupant's respiratory state acquired by the respiratory state acquisition unit 132. Furthermore, for example, the sleep abnormality detection unit 133 may detect an apnea sleep state when it detects that a hypoventilation state has continued for a predetermined period of time based on information about the occupant's respiratory state acquired by the respiratory state acquisition unit 132.

[0042] Furthermore, for example, the sleep abnormality detection unit 133 may detect a state of cardiopulmonary collapse when it detects that the occupant's breathing is weak or that irregular breathing has continued for a predetermined period of time based on information regarding the occupant's breathing state acquired by the breathing state acquisition unit 132.

[0043] Furthermore, for example, the sleep abnormality detection unit 133 may detect that the occupant is in a state of cardiopulmonary collapse when it detects breathing that sounds like shortness of breath based on information regarding the occupant's respiratory state acquired by the respiratory state acquisition unit 132.

[0044] Furthermore, for example, the sleep abnormality detection unit 133 may detect that the occupant is in a state of paralysis when it detects that weak breathing has continued for a predetermined period of time based on information regarding the occupant's breathing state acquired by the breathing state acquisition unit 132.

[0045] Furthermore, for example, the sleep abnormality detection unit 133 may detect that the occupant is in a teeth grinding state or a mental stress state when it detects that the pulse rate and blood pressure have increased above predetermined thresholds based on the information about the occupant's respiratory state acquired by the respiratory state acquisition unit 132. Furthermore, for example, the sleep abnormality detection unit 133 may detect that the occupant is in a teeth grinding state or a mental stress state when it detects that the respiratory rate has increased above predetermined thresholds based on the information about the occupant's respiratory state acquired by the respiratory state acquisition unit 132.

[0046] In step ST103, if the sleep abnormality detection unit 133 detects that the occupant has a sleep abnormality, it outputs information about the sleep abnormality of the occupant (step ST104).

[0047] On the other hand, if the sleep abnormality detection unit 133 detects in step ST103 that the occupant is not suffering from a sleep abnormality, the sequence proceeds to step ST105.

[0048] Next, the occupant sleep abnormality detection device 13 determines whether the vehicle has finished traveling (step ST105).

[0049] In step ST105, if the occupant sleep abnormality detection device 13 determines that the traveling of the vehicle has not ended, the sequence returns to step ST101, and the above processing is repeated.

[0050] On the other hand, if the occupant sleep abnormality detection device 13 determines in step ST105 that the traveling of the vehicle has ended, the sequence ends.

[0051] As described above, in the occupant sleep abnormality detection device 13 according to the first embodiment, the sleep state detection unit 131 detects the sleep state of the occupant, the respiratory state acquisition unit 132 acquires information about the respiratory state of the occupant, and the sleep abnormality detection unit 133 detects whether the occupant has a sleep abnormality based on the information about the sleep state and the respiratory state. This enables the occupant sleep abnormality detection device 13 according to the first embodiment to detect when the occupant has fallen into a sleep abnormality such as apnea. As a result, the occupant sleep abnormality detection device 13 according to the first embodiment can output information indicating that a specific occupant has fallen into a sleep abnormality, enabling appropriate measures to be taken.

[0052] In the above description, the occupant respiratory condition detecting device 12 is a millimeter-wave radar. However, the occupant respiratory condition detecting device 12 is not limited to this. For example, the occupant respiratory condition detecting device 12 may be an imaging unit capable of acquiring an image of an occupant, and the sleep abnormality detecting unit 133 may detect the occupant's respiratory condition from the image acquired by the imaging unit.

[0053] For example, the occupant respiratory condition detection device 12 may acquire an image showing the occupant's chest, and the sleep abnormality detection unit 133 may detect the occupant's respiratory condition by detecting the expansion and contraction of the occupant's chest from this image.

[0054] Furthermore, for example, the occupant respiratory condition detection device 12 may acquire a captured image showing the occupant's face, and the sleep condition detection unit 131 may detect the occupant's respiratory condition by performing facial expression recognition on the captured image. For example, the sleep abnormality detection unit 133 may detect an apneic sleep state when it detects a painful expression in the captured image that shows the facial muscles moving. For example, the sleep abnormality detection unit 133 may detect a cardiopulmonary collapse state or a sleep paralysis state when it detects a painful expression only in the eyes on the captured image. For example, the sleep abnormality detection unit 133 may detect a teeth grinding state or a mental stress state when it detects an expression in which the occupant clenches and moves their teeth on the captured image.

[0055] Also, for example, the occupant respiratory condition detecting device 12 may obtain a captured image showing the face of the occupant, and the sleep state detecting unit 131 may detect the pulse or blood pressure from this captured image.

[0056] The occupant respiratory condition detection device 12 may also be a smart watch. In this case, for example, the occupant respiratory condition detection device 12 may detect pulse, blood pressure, or oxygen concentration as information related to the respiratory condition.

[0057] The occupant respiratory condition detection device 12 is not limited to this, and may detect, for example, audio information as information related to the respiratory condition. For example, the sleep abnormality detection unit 133 may detect an apneic sleep state when it detects from the audio information that heavy snoring has suddenly stopped and then detects a sudden inhalation like a puff. For example, the sleep abnormality detection unit 133 may detect an apneic sleep state when it detects gasping or choking from the audio information. For example, the sleep abnormality detection unit 133 may detect a bruxism state or a mental stress state when it detects teeth grinding sounds from the audio information.

[0058] The occupant respiratory condition detecting device 12 may detect multiple types of information related to the occupant's respiratory condition, including a respiratory signal, a pulse rate, blood pressure, a respiratory rate, an oxygen concentration, a captured image, and / or audio information. The sleep abnormality detecting unit 133 may then use these multiple detection results to comprehensively detect sleep abnormalities.

[0059] Furthermore, the occupant sleep state detection device 11 may detect that the occupant is asleep when detecting from the captured image that the occupant's face is covered. For example, the occupant sleep state detection device 11 may detect that the occupant is asleep when the occupant's face is covered with a blanket or when the occupant's face is not visible due to other coverings.

[0060] In addition, even if the occupant is covered, if the occupant sleep state detection device 11 can detect a relaxed or collapsed state of the body due to sleep, it may determine whether the occupant is asleep without immediately determining that the occupant is asleep.

[0061] In the above description, the occupant sleep state detection device 11 includes an imaging unit and an image processing unit that detects information about the occupant's sleep state from an image captured by the imaging unit. However, the occupant sleep state detection device 11 is not limited to this. For example, the occupant sleep state detection device 11 may be a millimeter-wave radar. That is, when an occupant is asleep, the occupant generally takes deep breaths and the number of breaths decreases. Therefore, for example, the occupant sleep state detection device 11 may detect whether the occupant is asleep by detecting a difference in breathing between when awake and when asleep using a breathing signal from the millimeter-wave radar.

[0062] The occupant sleep state detecting device 11 may also detect information about the occupant's sleep state from both an image captured by the imaging unit and a respiratory signal captured by the millimeter wave radar.

[0063] The configurations of the occupant sleep state detection device 11 and the occupant respiratory state detection device 12 may be changed depending on the seat position of the occupant targeted by the occupant sleep abnormality detection device 13. For example, when a front-seat occupant is the target of detection, the occupant sleep state detection device 11 and the occupant respiratory state detection device 12 may detect information about the sleep state and information about the respiratory state from captured images, respectively. For example, when a rear-seat occupant is the target of detection, the occupant sleep state detection device 11 and the occupant respiratory state detection device 12 may detect information about the sleep state and information about the respiratory state using millimeter-wave radar, respectively. That is, typically, a driver detection device for image processing is provided in the front seats, which can detect the occupant status even of the passenger seat occupant, and a millimeter-wave radar is often provided in the rear seats for occupant abandonment warning. Therefore, by changing the configuration as described above, it is possible to utilize existing devices, thereby realizing the occupant sleep abnormality detection system 1 with reduced cost.

[0064] The sleep abnormality detection unit 133 may also detect the presence or absence of a sleep abnormality using a trained machine learning model. This trained machine learning model uses information about the sleep state and information about the respiratory state as input variables and the presence or absence of the occupant's sleep abnormality as an output variable. For example, the occupant sleep abnormality detection device 13 transmits sensor information, such as captured images or respiratory signals, indicating the occupant's sleep abnormality to a server (not shown). The server may then analyze the sleep abnormalities of multiple people and learn sleep abnormality detection logic based on their age, gender, body shape, or medical history. Learning also includes machine learning. The learning results from the server may be downloaded to the occupant sleep abnormality detection device 13 for use. Furthermore, adopting learning results based on the occupant's age, gender, body shape, or medical history improves the accuracy of detecting the presence or absence of a sleep abnormality. The sleep abnormality detection logic corresponding to the occupant of the vehicle may be learned not on the server side but on the occupant sleep abnormality detection system 1 installed in the vehicle.

[0065] Furthermore, the occupant sleep abnormality detection device 13 may include an emergency abnormality detection unit (not shown) in addition to the sleep abnormality detection unit 133. The emergency abnormality detection unit detects whether an emergency abnormality has occurred in an occupant based on at least one of the detection results from the occupant sleep state detection device 11 and the detection results from the occupant respiratory state detection device 12. Examples of types of emergency abnormalities detected by the emergency abnormality detection unit include sleep abnormalities due to serious illness or respiratory arrest. When the emergency abnormality detection unit detects that an emergency abnormality has occurred in an occupant, it outputs information about the emergency abnormality of the occupant. For example, if the occupant in whom the emergency abnormality has occurred is a occupant other than the driver, the emergency abnormality detection unit notifies the driver that an emergency abnormality has occurred in the occupant. Furthermore, for example, if the occupant in whom the emergency abnormality has occurred is the driver, the emergency abnormality detection unit automatically stops the vehicle or issues an emergency call.

[0066] In this configuration, as shown in FIG. 4, for example, step ST106 is added to the flowchart shown in FIG. 2, and step ST104 is changed to steps ST107 to ST109.

[0067] In the flowchart shown in FIG. 4 , first, the emergency abnormality detection unit detects whether an emergency abnormality has occurred in the occupant based on at least one of the detection results from the occupant sleep state detection device 11 and the detection results from the occupant respiratory state detection device 12 (step ST106). This process is performed before the process for detecting the presence or absence of a sleep abnormality, and is for detecting an emergency abnormality that can be detected without detecting a sleep abnormality, such as respiratory arrest. If the emergency abnormality detection unit detects that no emergency abnormality has occurred in step ST106, the sequence proceeds to step ST101. On the other hand, if the emergency abnormality detection unit detects that an emergency abnormality has occurred in step ST106, it outputs information about the emergency abnormality (step ST109).

[0068] In the flowchart shown in FIG. 4 , when the sleep abnormality detection unit 133 determines that an occupant has a sleep abnormality, the emergency abnormality detection unit determines whether the occupant has an emergency abnormality based on at least one of the detection results from the occupant sleep state detection device 11 and the occupant respiratory state detection device 12 (step ST107). This process is performed after the process for detecting the presence or absence of a sleep abnormality and is intended to detect an emergency abnormality that can be detected after detecting a sleep abnormality, such as a sleep abnormality caused by a serious illness. If the emergency abnormality detection unit detects that no emergency abnormality has occurred in step ST107, the sleep abnormality detection unit 133 outputs information about the sleep abnormality (step ST108). This process is similar to the process in step ST104. On the other hand, if the emergency abnormality detection unit detects that an emergency abnormality has occurred in step ST107, it outputs information about the emergency abnormality (step ST109).

[0069] 4 illustrates a case where the occupant sleep abnormality detection device 13 performs both the process of step ST106 and the processes of steps ST107 and ST109 as the process related to an emergency abnormality. However, the present invention is not limited to this, and the occupant sleep abnormality detection device 13 may perform only one of the process of step ST106 and the process of steps ST107 and ST109 as the process related to an emergency abnormality.

[0070] As described above, according to the first embodiment, the occupant sleep abnormality detection system 1 includes an occupant sleep state detection device 11 that detects information about the sleeping state of a vehicle occupant, an occupant respiratory state detection device 12 that detects information about the respiratory state of the vehicle occupant, and an occupant sleep abnormality detection device 13. The occupant sleep abnormality detection device 13 includes a sleeping state detection unit 131 that detects the sleeping state of the occupant based on the detection result by the occupant sleep state detection device 11, a respiratory state acquisition unit 132 that acquires information about the respiratory state of the occupant based on the detection result by the occupant respiratory state detection device 12, and a sleep abnormality detection unit 133 that detects whether the occupant has a sleep abnormality based on the sleeping state of the occupant detected by the sleeping state detection unit 131 and the information about the respiratory state of the occupant acquired by the respiratory state acquisition unit 132. Furthermore, according to the first embodiment, the occupant sleep state detection device 11 detects information about the sleeping state of the occupant from a captured image, and the occupant respiratory state detection device 12 is a millimeter-wave radar. According to the first embodiment, the occupant sleep state detection device 11 detects information about the occupant's sleep state from the captured image, and the occupant respiratory state detection device 12 detects audio information as information about the occupant's respiratory state. According to the first embodiment, the occupant sleep state detection device 11 detects information about the occupant's sleep state from the captured image, and the occupant respiratory state detection device 12 detects information about the occupant's respiratory state from the captured image. According to the first embodiment, the occupant sleep state detection device 11 detects that the occupant's face is covered when detecting from the captured image that the occupant's face is covered. As a result, the occupant sleep abnormality detection system 1 according to the first embodiment can detect the occupant's sleep abnormality. As a result, the occupant sleep abnormality detection system 1 according to the first embodiment can assist in resolving the occupant's sleep abnormality.

[0071] Furthermore, according to this embodiment 1, the sleep abnormality detected by the sleep abnormality detection unit 133 is an apnea sleep state, a sleep paralysis state, a cardiopulmonary collapse state, a teeth grinding state, or a mental stress state. As a result, the occupant sleep abnormality detection system 1 according to embodiment 1 can detect an apnea sleep state, a sleep paralysis state, a cardiopulmonary collapse state, a teeth grinding state, or a mental stress state as an occupant sleep abnormality.

[0072] Furthermore, according to this embodiment 1, the sleep abnormality detection unit 133 detects the presence or absence of a sleep abnormality using a trained machine learning model that uses information about the sleeping state and breathing state as input variables and the presence or absence of the occupant's sleep abnormality as an output variable. This further improves the accuracy of detecting occupant sleep abnormalities in the occupant sleep abnormality detection system 1 according to embodiment 1.

[0073] Furthermore, according to this embodiment 1, the occupant sleep abnormality detection device 13 includes a sleep state detection unit 131 that detects the sleep state of the vehicle occupant based on the detection result by the occupant sleep state detection device 11 that detects information about the sleep state of the vehicle occupant, a respiratory state acquisition unit 132 that acquires information about the respiratory state of the occupant based on the detection result by the occupant respiratory state detection device 12 that detects information about the respiratory state of the vehicle occupant, and a sleep abnormality detection unit 133 that detects whether the occupant has a sleep abnormality based on the sleep state of the occupant detected by the sleep state detection unit 131 and the information about the respiratory state of the occupant acquired by the respiratory state acquisition unit 132. This makes it possible for the occupant sleep abnormality detection device 13 according to embodiment 1 to detect a sleep abnormality in the occupant. As a result, the occupant sleep abnormality detection device 13 according to embodiment 1 can assist in resolving the occupant's sleep abnormality.

[0074] Furthermore, according to this embodiment 1, the occupant sleep abnormality detection method includes the steps of: a sleep state detection unit 131 detecting the sleep state of the vehicle occupant based on a detection result by the occupant sleep state detection device 11 that detects information about the sleep state of the vehicle occupant; a respiratory state acquisition unit 132 acquiring information about the occupant's respiratory state based on a detection result by the occupant respiratory state detection device 12 that detects information about the occupant's respiratory state; and a sleep abnormality detection unit 133 detecting whether the occupant has a sleep abnormality based on the occupant's sleep state detected by the sleep state detection unit 131 and the information about the occupant's respiratory state acquired by the respiratory state acquisition unit 132. This makes it possible for the occupant sleep abnormality detection method according to embodiment 1 to detect the occupant's sleep abnormality. As a result, the occupant sleep abnormality detection method according to embodiment 1 can assist in resolving the occupant's sleep abnormality.

[0075] Embodiment 2. Figure 5 is a diagram showing an example of the configuration of an occupant sleep abnormality detection system 1 according to embodiment 2. In the occupant sleep abnormality detection system 1 according to embodiment 2 shown in Figure 5, an occupant support device 14 is added to the occupant sleep abnormality detection system 1 according to embodiment 1 shown in Figure 1, and an occupant support unit 134 is added to the occupant sleep abnormality detection device 13. The other configuration example of the occupant sleep abnormality detection system 1 according to embodiment 2 shown in Figure 5 is similar to the configuration example of the occupant sleep abnormality detection system 1 according to embodiment 1 shown in Figure 1, and the same reference numerals are used, and only the different parts will be described.

[0076] The occupant support device 14 is a device for resolving sleep abnormalities. This occupant support device 14 operates in accordance with drive control by the occupant support unit 134. Examples of this occupant support device 14 include a reclining seat provided in a seat of a mobile body, an automatic driving control device that enables automatic driving control of a mobile body, a tactile stimulus generating device that provides tactile stimulus to an occupant of a mobile body by vibration or the like, an air conditioning stimulus generating device that provides tactile stimulus to an occupant of a mobile body by temperature or wind, an optical stimulus generating device that provides visual stimulus to an occupant of a mobile body, or an audio stimulus generating device that provides auditory stimulus to an occupant of a mobile body.

[0077] The occupant support unit 134 controls the occupant support device 14 so as to eliminate the sleep abnormality of the occupant whose sleep abnormality has been detected by the sleep abnormality detection unit 133 .

[0078] For example, if the occupant support device 14 is a reclining seat, the occupant support unit 134 may perform drive control to change the reclining state of the occupant support device 14 .

[0079] Also, for example, if the occupant assistance device 14 is an automatic driving control device, the occupant assistance unit 134 may drive and control the occupant assistance device 14 to generate acceleration in at least one of the lateral and longitudinal directions of the vehicle.

[0080] Furthermore, for example, if the occupant support device 14 is a tactile stimulus generator, an air conditioning stimulus generator, a light stimulus generator, or a sound stimulus generator, the occupant support unit 134 may drive and control the occupant support device 14 to provide a stimulus to the occupant. That is, for example, the occupant support unit 134 may provide vibration to the occupant using a tactile stimulus generator having a vibration or massage function installed in the seat. For example, the occupant support unit 134 may provide a stimulus to the occupant using a tactile stimulus generator having a seatbelt tensioning function. For example, the occupant support unit 134 may provide air conditioning to the occupant's face using an air conditioning stimulus generator to increase the amount of oxygen. For example, the occupant support unit 134 may provide a light stimulus to the occupant by darkening the cabin during sleep and brightening it during abnormal sleep using a light stimulus generator capable of controlling a liquid crystal window or curtain. For example, the occupant support unit 134 may provide a stimulus to the occupant using voice or relaxing music using a sound stimulus generator.

[0081] Next, an example of the operation of the occupant sleep abnormality detection device 13 according to the second embodiment shown in Fig. 5 will be described with reference to Fig. 6. The occupant sleep state detection device 11 detects information related to the occupant's sleep state and outputs data indicating the information to the occupant sleep abnormality detection device 13. The occupant respiratory state detection device 12 detects information related to the occupant's respiratory state and outputs data indicating the information to the occupant sleep abnormality detection device 13.

[0082] In an example of operation of the occupant sleep abnormality detection device 13 according to embodiment 2 shown in Figure 5, as shown in Figure 6, first, the sleep state detection unit 131 detects whether the occupant is asleep or not based on the detection result by the occupant sleep state detection device 11 (step ST201).

[0083] Next, the respiratory condition acquisition unit 132 acquires information about the respiratory condition of the occupant based on the detection result by the occupant respiratory condition detection device 12 (step ST202).

[0084] Next, the sleep abnormality detection unit 133 detects whether or not the occupant has a sleep abnormality based on the sleeping state of the occupant detected by the sleeping state detection unit 131 and the information on the respiratory state of the occupant acquired by the respiratory state acquisition unit 132 (step ST203). At this time, the sleep abnormality detection unit 133 detects whether or not the occupant has a sleep abnormality by monitoring the information on the respiratory state acquired by the respiratory state acquisition unit 132 for the occupant detected to be asleep by the sleeping state detection unit 131.

[0085] In step ST203, if the sleep abnormality detection unit 133 detects that the occupant has a sleep abnormality, it outputs information about the sleep abnormality of the occupant to the occupant support unit 134 (step ST204).

[0086] Next, the occupant support unit 134 controls the drive of the occupant support device 14 so as to eliminate the sleep abnormality of the occupant whose sleep abnormality has been detected by the sleep abnormality detection unit 133 (step ST205).

[0087] For example, if the occupant support device 14 is a reclining seat, the occupant support unit 134 may perform drive control to change the reclining state of the occupant support device 14 .

[0088] Also, for example, when the occupant support device 14 is an automatic driving control device, the occupant support unit 134 may control the drive of the occupant support device 14 so that the vehicle generates acceleration in the lateral or longitudinal direction.

[0089] Furthermore, for example, if the occupant support device 14 is a tactile stimulus generator, an air conditioning stimulus generator, a light stimulus generator, or a sound stimulus generator, the occupant support unit 134 may drive and control the occupant support device 14 to provide a stimulus to the occupant. That is, for example, the occupant support unit 134 may provide vibration to the occupant using a tactile stimulus generator having a vibration or massage function installed in the seat. For example, the occupant support unit 134 may provide a stimulus to the occupant using a tactile stimulus generator having a seatbelt tensioning function. For example, the occupant support unit 134 may provide air conditioning to the occupant's face using an air conditioning stimulus generator to increase the amount of oxygen. For example, the occupant support unit 134 may provide a light stimulus to the occupant by darkening the cabin during sleep and brightening it during abnormal sleep using a light stimulus generator capable of controlling a liquid crystal window or curtain. For example, the occupant support unit 134 may provide a stimulus to the occupant using voice or relaxing music using a sound stimulus generator.

[0090] On the other hand, if the sleep abnormality detection unit 133 detects in step ST203 that the occupant is not suffering from a sleep abnormality, the sequence proceeds to step ST206.

[0091] Next, the occupant sleep abnormality detection device 13 determines whether the traveling of the vehicle has ended (step ST206).

[0092] In step ST206, if the occupant sleep abnormality detection device 13 determines that the traveling of the vehicle has not ended, the sequence returns to step ST201, and the above processing is repeated.

[0093] On the other hand, if the occupant sleep abnormality detection device 13 determines in step ST206 that the traveling of the vehicle has ended, the sequence ends.

[0094] As described above, in the occupant sleep abnormality detection device 13 according to the second embodiment, the occupant support unit 134 controls the occupant support device 14 so as to resolve the sleep abnormality of an occupant whose sleep abnormality has been detected by the sleep abnormality detection unit 133. As a result, the occupant sleep abnormality detection device 13 according to the second embodiment can resolve the sleep abnormality when the sleep abnormality occurs in the occupant.

[0095] If the occupant support device 14 is capable of executing multiple occupant support methods, the occupant support unit 134 may select an occupant support method to be performed by the occupant support device 14 and then perform drive control, depending on the type of sleep abnormality detected by the sleep abnormality detection unit 133. If multiple types of occupant support devices 14 are provided, the occupant support unit 134 may select an occupant support device 14 and then perform drive control, depending on the type of sleep abnormality detected by the sleep abnormality detection unit 133.

[0096] For example, when the sleep abnormality detection unit 133 detects that the occupant is in an apneic sleeping state, it is easier for the occupant to breathe by lying on their side rather than on their back. Therefore, for example, when the sleep abnormality detection unit 133 detects that the occupant is in an apneic sleeping state and the occupant is lying on their back, the occupant support unit 134 may tilt a reclining seat that is structured to tilt the occupant left or right to one side. Also, for example, when the occupant is already leaning to one side, the occupant support unit 134 may tilt the reclining seat in the opposite direction. Also, for example, when the sleep abnormality detection unit 133 detects that the occupant is in an apneic sleeping state, the occupant support unit 134 may control the driving of the vehicle by using an automatic driving control device to apply lateral acceleration to the vehicle in order to lean the occupant left or right in the same manner as described above.

[0097] Furthermore, for example, when the sleep abnormality detection unit 133 detects that the occupant has cardiopulmonary collapse, the occupant can breathe more easily by tilting their upper body upward. Therefore, for example, when the sleep abnormality detection unit 133 detects that the occupant has cardiopulmonary collapse, the occupant support unit 134 may tilt the reclining seat upward. Furthermore, for example, when the sleep abnormality detection unit 133 detects that the occupant has cardiopulmonary collapse, the occupant support unit 134 may control the air conditioning stimulus generator to blow fresh air to the occupant's face.

[0098] Furthermore, for example, when the sleep abnormality detection unit 133 detects that the occupant is paralyzed, the occupant may be released from the paralysis by a stimulus. Therefore, for example, when the sleep abnormality detection unit 133 detects that the occupant is paralyzed, the occupant support unit 134 may provide a tactile stimulus. That is, for example, the occupant support unit 134 may generate vibrations or vibrations using a reclining seat. Furthermore, the occupant support unit 134 may provide an air conditioning stimulus using an air conditioning stimulus generating device.

[0099] Furthermore, for example, if the sleep abnormality detection unit 133 detects that the occupant is grinding their teeth or is in a state of mental stress, these states are symptoms that occur during REM sleep, and the occupant is aware of the outside world, albeit unconsciously. Therefore, for example, if the sleep abnormality detection unit 133 detects that the occupant is grinding their teeth or is in a state of mental stress, the occupant support unit 134 may generate relaxing music using a sound stimulation generator. Also, for example, the occupant support unit 134 may provide gentle vibrations to the reclining seat to help the occupant relax.

[0100] Furthermore, when the occupant assistance device 14 is capable of executing multiple occupant assistance methods, the occupant assistance unit 134 may adaptively change the occupant assistance method executed by the occupant assistance device 14. For example, the occupant assistance unit 134 may execute a specific occupant assistance method executed by the occupant assistance device 14, and then, based on a change in the occupant's sleep abnormality detected by the sleep abnormality detection unit 133, execute another occupant assistance method if the effect of the specific occupant assistance method is insufficient or low. For example, when the occupant assistance device 14 is a reclining seat, the occupant assistance unit 134 may first drive and control the occupant assistance device 14 to tilt it to a specific angle, and then, if there is no or only a small change in the occupant's sleep abnormality, drive and control the occupant assistance device 14 to tilt it further.

[0101] Furthermore, when multiple types of occupant support devices 14 are provided, the occupant support unit 134 may adaptively change the occupant support device 14. For example, the occupant support unit 134 may perform occupant support using a specific occupant support device 14, and then, based on a change in the occupant's sleep abnormality detected by the sleep abnormality detection unit 133, if the support effect of the specific occupant support device 14 is ineffective or low, perform occupant support using another occupant support device 14. For example, when the occupant support devices 14 are a reclining seat and a tactile stimulus generator, the occupant support unit 134 may first drive and control the reclining seat, and then, if there is no or only a small change in the occupant's sleep abnormality, drive and control the tactile stimulus generator.

[0102] Furthermore, when the occupant support device 14 is capable of executing multiple occupant support methods, the occupant support unit 134 may learn the support effect of each occupant support method by the occupant support device 14 for each occupant, and may preferentially use the occupant support method that is most effective for that occupant. Furthermore, when multiple types of occupant support devices 14 are provided, the occupant support unit 134 may learn the support effect of each occupant support device 14 for each occupant, and may preferentially use the occupant support device 14 that is most effective for that occupant.

[0103] Furthermore, the occupant support device 14 is a device for resolving the driver's sleep abnormality, and the occupant support unit 134 may drive and control the occupant support device 14 to resolve the sleep abnormality when the sleep abnormality detection unit 133 detects the driver's sleep abnormality while the vehicle is in fully automated driving mode. Without support from the occupant support device 14, if the driver experiences a sleep abnormality, the driver's fatigue caused by the sleep abnormality will not be resolved even after waking up, and the driver's driving aptitude will decrease when manual driving is resumed. In contrast, with support from the occupant support device 14, the above problem can be resolved by providing occupant support for the driver's sleep abnormality.

[0104] Furthermore, when the occupant is a driver and the vehicle in which the driver is riding is not in a fully automated driving state, it is desirable that the occupant sleep abnormality detection system 1 perform control so as not to cause the driver to become de-awakened or fall asleep. For example, when the driver becomes de-awakened or falls asleep, it is desirable that the occupant assistance unit 134 perform control so as to warn the driver of the de-awakening or to provide driver assistance to increase the driver's level of wakefulness, or to intervene in automated driving to park the vehicle in an appropriate location.

[0105] Furthermore, the sleep abnormality detection unit 133 may detect the presence or absence of a sign of a sleep abnormality using a trained machine learning model. The trained machine learning model is a trained machine learning model that uses information about the sleep state and breathing state as input variables and a sign of a sleep abnormality of the occupant as an output variable. In this case, the occupant assistance unit 134 may drive and control the occupant assistance device 14 so as to eliminate the sign of a sleep abnormality for an occupant in whom the sleep abnormality detection unit 133 has detected the occurrence of a sign of a sleep abnormality. That is, in the first embodiment, the case where the sleep abnormality detection logic is learned is described. In contrast, in the second embodiment, the sleep abnormality detection device 13 may also learn a sleep abnormality sign detection logic. Then, the occupant sleep abnormality detection device 13 may perform occupant assistance when a sign of a sleep abnormality is detected based on the learning result. This makes it possible to suppress the occurrence of a sleep abnormality itself.

[0106] As described above, according to the second embodiment, the occupant sleep abnormality detection device 13 includes the occupant assistance device 14 for eliminating the sleep abnormality, and the occupant sleep abnormality detection device 13 includes the occupant assistance unit 134 that drives and controls the occupant assistance device 14 so as to eliminate the sleep abnormality of an occupant whose sleep abnormality has been detected by the sleep abnormality detection unit 133. As a result, the occupant sleep abnormality detection device 13 according to the second embodiment can eliminate the sleep abnormality of an occupant in addition to the effects of the first embodiment.

[0107] According to the second embodiment, the occupant support device 14 is a reclining seat, and the occupant support unit 134 controls the occupant support device 14 to change its reclining state. According to the second embodiment, the occupant support device 14 is an automatic driving control device, and the occupant support unit 134 controls the occupant support device 14 to accelerate the vehicle in at least one of the lateral direction and the longitudinal direction. According to the second embodiment, the occupant support device 14 is a tactile stimulus generator, an air conditioning stimulus generator, a light stimulus generator, or a sound stimulus generator, and the occupant support unit 134 controls the occupant support device 14 to provide a stimulus to the occupant. As a result, the occupant sleep abnormality detection device 13 according to the second embodiment can eliminate the occupant's sleep abnormality by using the reclining seat, the automatic driving control device, the tactile stimulus generator, the air conditioning stimulus generator, the light stimulus generator, or the sound stimulus generator, in addition to the effects of the first embodiment.

[0108] Furthermore, according to the second embodiment, the occupant assistance device 14 is capable of executing a plurality of occupant assistance methods, and the occupant assistance unit 134 performs drive control after selecting an occupant assistance method to be performed by the occupant assistance device 14 in accordance with the type of sleep abnormality detected by the sleep abnormality detection unit 133. Furthermore, according to the second embodiment, a plurality of types of occupant assistance devices 14 are provided, and the occupant assistance unit 134 performs drive control after selecting an occupant assistance device 14 in accordance with the type of sleep abnormality detected by the sleep abnormality detection unit 133. As a result, the occupant sleep abnormality detection device 13 according to the second embodiment can more appropriately resolve the occupant's sleep abnormality.

[0109] Furthermore, according to the second embodiment, the occupant support device 14 can execute a plurality of occupant support methods, and the occupant support unit 134 learns the support effect of each occupant support method by the occupant support device 14 for each occupant, and preferentially uses the occupant support method by the occupant support device 14 that is most effective for that occupant. Furthermore, according to the second embodiment, a plurality of types of occupant support devices 14 are provided, and the occupant support unit 134 learns the support effect of each occupant support device 14 for each occupant, and preferentially uses the occupant support device 14 that is most effective for that occupant. As a result, the occupant sleep abnormality detection device 13 according to the second embodiment can more appropriately resolve occupant sleep abnormalities.

[0110] Furthermore, according to the second embodiment, the occupant sleep abnormality detection device 13 includes an occupant assistance device 14 for eliminating the driver's sleep abnormality, and the occupant sleep abnormality detection device 13 includes an occupant assistance unit 134 that controls the operation of the occupant assistance device 14 to eliminate the sleep abnormality when the sleep abnormality detection unit 133 detects the driver's sleep abnormality while the vehicle is in fully autonomous driving mode. As a result, the occupant sleep abnormality detection device 13 according to the second embodiment can eliminate the driver's sleep abnormality in addition to the effects of the first embodiment.

[0111] Furthermore, according to the second embodiment, the sleep abnormality detection unit 133 detects the presence or absence of a sign of a sleep abnormality using a trained machine learning model that uses information about the sleeping state and breathing state as input variables and a sign of the occupant's sleep abnormality as an output variable, and the occupant assistance unit 134 controls the drive of the occupant assistance device 14 so as to eliminate the sign of the sleep abnormality for an occupant in whom the sleep abnormality detection unit 133 has detected the occurrence of a sign of a sleep abnormality. In this way, the occupant sleep abnormality detection device 13 according to the second embodiment can eliminate the occurrence of the sleep abnormality itself.

[0112] Embodiment 3. Figure 7 is a diagram showing an example of the configuration of an occupant sleep abnormality detection system 1 according to embodiment 3. In the occupant sleep abnormality detection system 1 according to embodiment 3 shown in Figure 7, a recording device 15 is added to the occupant sleep abnormality detection system 1 according to embodiment 1 shown in Figure 1, and a recording control unit 135 is added to the occupant sleep abnormality detection device 13. The other example of the configuration of the occupant sleep abnormality detection system 1 according to embodiment 3 shown in Figure 7 is similar to the example of the configuration of the occupant sleep abnormality detection system 1 according to embodiment 1 shown in Figure 1, and the same reference numerals are used and only the different parts will be described.

[0113] The recording device 15 records information about the occupant's sleep abnormality detected by the occupant sleep abnormality detection device 13 under the control of the recording control unit 135. The recording device 15 is installed inside the vehicle or outside the vehicle, such as on a server.

[0114] When the sleep abnormality detection unit 133 detects that an occupant has a sleep abnormality, the recording control unit 135 causes the recording device 15 to record information about the occupant's sleep abnormality.

[0115] In addition to the above, the occupant sleep abnormality detection system 1 shown in FIG. 7 is also provided with a display device 16 that can display information related to the occupant's sleep abnormality recorded in the recording device 15.

[0116] Next, an example of the operation of the occupant sleep abnormality detection device 13 according to the third embodiment shown in Fig. 7 will be described with reference to Fig. 8. The occupant sleep state detection device 11 detects information related to the occupant's sleep state and outputs data indicating the information to the occupant sleep abnormality detection device 13. The occupant respiratory state detection device 12 detects information related to the occupant's respiratory state and outputs data indicating the information to the occupant sleep abnormality detection device 13.

[0117] In an example of operation of the occupant sleep abnormality detection device 13 according to embodiment 3 shown in Figure 7, as shown in Figure 8, first, the sleep state detection unit 131 detects whether the occupant is asleep or not based on the detection result by the occupant sleep state detection device 11 (step ST301).

[0118] Next, the respiratory condition acquisition unit 132 acquires information about the respiratory condition of the occupant based on the detection result by the occupant respiratory condition detection device 12 (step ST302).

[0119] Next, the sleep abnormality detection unit 133 detects whether or not the occupant has a sleep abnormality based on the sleeping state of the occupant detected by the sleeping state detection unit 131 and the information on the respiratory state of the occupant acquired by the respiratory state acquisition unit 132 (step ST303). At this time, the sleep abnormality detection unit 133 detects whether or not the occupant has a sleep abnormality by monitoring the information on the respiratory state acquired by the respiratory state acquisition unit 132 for the occupant detected to be asleep by the sleeping state detection unit 131.

[0120] In step ST303, if the sleep abnormality detection unit 133 detects that the occupant has a sleep abnormality, it outputs information about the sleep abnormality of the occupant to the recording control unit 135 (step ST304).

[0121] Next, when the sleep abnormality detection unit 133 detects the occurrence of a sleep abnormality in an occupant, the recording control unit 135 causes the recording device 15 to record information related to the sleep abnormality in the occupant (step ST305). That is, the recording device 15 records information related to the sleep abnormality of the occupant detected by the occupant sleep abnormality detection device 13 under the control of the recording control unit 135. In addition, the display device 16 may display the information related to the sleep abnormality of the occupant.

[0122] On the other hand, if the sleep abnormality detection unit 133 detects in step ST304 that the occupant is not suffering from a sleep abnormality, the sequence proceeds to step ST306.

[0123] Next, the occupant sleep abnormality detection device 13 determines whether the traveling of the vehicle has ended (step ST306).

[0124] In step ST306, if the occupant sleep abnormality detection device 13 determines that the traveling of the moving body has not ended, the sequence returns to step ST301, and the above processing is repeated.

[0125] On the other hand, if the occupant sleep abnormality detection device 13 determines in step ST306 that the traveling of the vehicle has ended, the sequence ends.

[0126] As described above, when the occupant sleep abnormality detection device 13 according to the third embodiment detects the occurrence of a sleep abnormality in an occupant, the recording device 15 is caused to record information about the sleep abnormality. That is, just as a drive recorder records video or situations inside and outside a vehicle before and after a specific event, when the sleep abnormality detection unit 133 detects the occurrence of a sleep abnormality in an occupant, the recording control unit 135 causes the recording device 15 to record the occupant's state before and after the detection, for example, captured images of the occupant at times before and after the occurrence of the sleep abnormality. Furthermore, in addition to the captured images described above, the recording control unit 135 may also record, for example, the type of sleep abnormality, a breathing signal, time, or vehicle location information. Furthermore, the display device 16 is capable of displaying information stored in the recording device 15 through a predetermined operation.

[0127] As a result, the occupant sleep abnormality detection device 13 according to the third embodiment is capable of recording information relating to the occupant's sleep abnormality, allowing those involved in driving to understand the state of the occupant's sleep abnormality.

[0128] In the above description, the recording device 15 and the display device 16 are added to the occupant sleep abnormality detection system 1 according to the first embodiment shown in Fig. 1 , and the recording control unit 135 is added to the occupant sleep abnormality detection device 13. However, the present invention is not limited to this. The recording device 15 and the display device 16 may be added to the occupant sleep abnormality detection system 1 according to the second embodiment shown in Fig. 5 , and the recording control unit 135 may be added to the occupant sleep abnormality detection device 13, and the same effects as those described above may be obtained. In this case, in addition to the effects described above, those involved in driving may also be able to know the occupant assistance effect.

[0129] As described above, according to the third embodiment, the occupant sleep abnormality detection device 13 includes the recording control unit 135 that records information about the occupant's sleep abnormality in a storage device when the occurrence of the occupant's sleep abnormality is detected by the sleep abnormality detection unit 133. As a result, the occupant sleep abnormality detection device 13 according to the third embodiment not only achieves the effects of the first and second embodiments, but also enables a person involved in driving to understand the state of the occupant's sleep abnormality.

[0130] Embodiment 4. Figure 9 is a diagram showing an example configuration of an occupant sleep abnormality detection system 1 according to embodiment 4. In the occupant sleep abnormality detection system 1 according to embodiment 4 shown in Figure 9, an alarm device 17 is added to the occupant sleep abnormality detection system 1 according to embodiment 1 shown in Figure 1, and an alarm control unit 136 is added to the occupant sleep abnormality detection device 13. The other configuration example of the occupant sleep abnormality detection system 1 according to embodiment 4 shown in Figure 9 is similar to the configuration example of the occupant sleep abnormality detection system 1 according to embodiment 1 shown in Figure 1, and the same reference numerals are used, and only the different parts will be described.

[0131] The notification device 17, under the control of the notification control unit 136, notifies information about the occupant's sleep abnormality detected by the occupant sleep abnormality detection device 13. The notification device 17 may be, for example, a display device or an audio output device installed inside or outside the vehicle.

[0132] When the sleep abnormality detection unit 133 detects that an occupant has a sleep abnormality, the notification control unit 136 causes the notification device 17 to notify other occupants or the outside of the vehicle of information regarding the occupant's sleep abnormality.

[0133] 9, the notification control unit 136 can issue a notification based on the detection result by the occupant sleep state detection device 11. For example, if the occupant sleep state detection device 11 has an imaging unit, the notification control unit 136 can issue a notification of information related to the occupant's sleep abnormality using an image captured by the imaging unit.

[0134] Next, an example of the operation of the occupant sleep abnormality detection device 13 according to the fourth embodiment shown in Fig. 9 will be described with reference to Fig. 10. The occupant sleep state detection device 11 detects information related to the occupant's sleep state and outputs data indicating the information to the occupant sleep abnormality detection device 13. The occupant respiratory state detection device 12 detects information related to the occupant's respiratory state and outputs data indicating the information to the occupant sleep abnormality detection device 13.

[0135] In an example of operation of the occupant sleep abnormality detection device 13 according to embodiment 4 shown in Figure 9, as shown in Figure 10, first, the sleep state detection unit 131 detects whether the occupant is asleep or not based on the detection result by the occupant sleep state detection device 11 (step ST401).

[0136] Next, the respiratory condition acquisition unit 132 acquires information about the respiratory condition of the occupant based on the detection result by the occupant respiratory condition detection device 12 (step ST402).

[0137] Next, the sleep abnormality detection unit 133 detects whether or not the occupant has a sleep abnormality based on the sleeping state of the occupant detected by the sleeping state detection unit 131 and the information on the respiratory state of the occupant acquired by the respiratory state acquisition unit 132 (step ST403). At this time, the sleep abnormality detection unit 133 detects whether or not the occupant has a sleep abnormality by monitoring the information on the respiratory state acquired by the respiratory state acquisition unit 132 for the occupant detected to be asleep by the sleeping state detection unit 131.

[0138] In step ST403, if the sleep abnormality detection unit 133 detects that the occupant has a sleep abnormality, it outputs information related to the occupant's sleep abnormality to the notification control unit 136 (step ST404).

[0139] Next, when the sleep abnormality detection unit 133 detects the occurrence of a sleep abnormality in an occupant, the notification control unit 136 causes the notification device 17 to notify other occupants or the outside of the vehicle of information related to the sleep abnormality in the occupant (step ST405). Then, the notification device 17 notifies information related to the sleep abnormality in the occupant detected by the occupant sleep abnormality detection device 13 in response to control by the notification control unit 136.

[0140] For example, Fig. 11 is a diagram showing an example of information about an occupant's sleep abnormality notified by the notification control unit 136 via the notification device 17. Fig. 11 shows a case where the notification device 17 is a display device, and a case where the information about the occupant's sleep abnormality is displayed as a notification screen. The notification screen shown in Fig. 11 displays the type of sleep abnormality indicated by reference numeral 201, an image (a facial image in the figure) acquired by the imaging unit of the occupant sleep state detection device 11 at the time of the sleep abnormality indicated by reference numeral 202, and a respiratory signal indicated by reference numeral 203. The display screen shown in Fig. 11 also displays the number of times the occupant's sleep abnormality has occurred as indicated by reference numeral 204.

[0141] In the case of fully autonomous driving, if the sleep abnormality detection unit 133 detects a sleep abnormality in the driver, the notification control unit 136 may notify other occupants other than the driver. This allows the other occupants to take appropriate measures for the driver. Furthermore, if there are no occupants other than the driver, the notification control unit 136 may notify a server or a predetermined communication terminal outside the vehicle.

[0142] On the other hand, if the sleep abnormality detection unit 133 detects in step ST404 that the occupant is not suffering from a sleep abnormality, the sequence proceeds to step ST406.

[0143] Next, the occupant sleep abnormality detection device 13 determines whether the traveling of the vehicle has ended (step ST406).

[0144] In step ST406, if the occupant sleep abnormality detection device 13 determines that the traveling of the moving body has not ended, the sequence returns to step ST401, and the above processing is repeated.

[0145] On the other hand, if the occupant sleep abnormality detection device 13 determines in step ST406 that the traveling of the vehicle has ended, the sequence ends.

[0146] In this way, in the fourth embodiment, when the sleep abnormality detection unit 133 detects that an occupant has a sleep abnormality, information related to the sleep abnormality is reported. As a result, the occupant sleep abnormality detection device 13 according to the fourth embodiment allows other occupants or outside persons to recognize the sleep abnormality and take appropriate measures.

[0147] In the above description, the alarm device 17 is added to the occupant sleep abnormality detection system 1 according to the first embodiment shown in Fig. 1 , and the alarm control unit 136 is added to the occupant sleep abnormality detection device 13. However, the present invention is not limited to this. The alarm device 17 may be added to the occupant sleep abnormality detection system 1 according to the second embodiment shown in Fig. 5 or the occupant sleep abnormality detection system 1 according to the third embodiment shown in Fig. 7 , and the alarm control unit 136 may be added to the occupant sleep abnormality detection device 13, and the same effects as those described above may be obtained.

[0148] As described above, according to the fourth embodiment, the occupant sleep abnormality detection device 13 includes the notification control unit 136 that, when the sleep abnormality detection unit 133 detects the occurrence of a sleep abnormality in an occupant, causes the notification device 17 to notify other occupants or the outside of the vehicle of information related to the sleep abnormality in the occupant. As a result, the occupant sleep abnormality detection device 13 according to the fourth embodiment can take appropriate measures against the sleep abnormality in the occupant in addition to the effects of the first to third embodiments.

[0149] Finally, with reference to FIG. 12 , an example of the hardware configuration of the occupant sleep abnormality detection device 13 according to the first to fourth embodiments will be described. Hereinafter, an example of the hardware configuration of the occupant sleep abnormality detection device 13 according to the first embodiment will be described, but the same applies to the example of the hardware configuration of the occupant sleep abnormality detection device 13 according to the second to fourth embodiments. The functions of the sleep state detection unit 131, the respiratory state acquisition unit 132, and the sleep abnormality detection unit 133 in the occupant sleep abnormality detection device 13 are realized by a processing circuit 51. The processing circuit 51 may be dedicated hardware as shown in FIG. 12A , or may be a CPU (also referred to as a central processing unit, processing unit, arithmetic unit, microprocessor, microcomputer, processor, or DSP (Digital Signal Processor)) 52 that executes a program stored in a memory 53 as shown in FIG. 12B .

[0150] When the processing circuitry 51 is dedicated hardware, the processing circuitry 51 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. The functions of the sleep state detection unit 131, the respiratory state acquisition unit 132, and the sleep abnormality detection unit 133 may be realized individually by the processing circuitry 51, or the functions of these units may be realized collectively by the processing circuitry 51.

[0151] When the processing circuit 51 is a CPU 52, the functions of the sleep state detection unit 131, the respiratory state acquisition unit 132, and the sleep abnormality detection unit 133 are realized by software, firmware, or a combination of software and firmware. The software and firmware are written as programs and stored in the memory 53. The processing circuit 51 realizes the functions of each unit by reading and executing the programs stored in the memory 53. That is, the occupant sleep abnormality detection device 13 includes the memory 53 for storing programs that, when executed by the processing circuit 51, result in the execution of, for example, each step shown in FIG. 2 . It can also be said that these programs cause a computer to execute the procedures and methods of the sleep state detection unit 131, the respiratory state acquisition unit 132, and the sleep abnormality detection unit 133. Here, examples of the memory 53 include non-volatile or volatile semiconductor memories such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable ROM), and EEPROM (Electrically EPROM), as well as magnetic disks, flexible disks, optical disks, compact disks, minidisks, and DVDs (Digital Versatile Discs).

[0152] Note that the functions of the sleep state detection unit 131, the respiratory state acquisition unit 132, and the sleep abnormality detection unit 133 may be partially implemented by dedicated hardware and partially implemented by software or firmware. For example, the sleep state detection unit 131 may be implemented by a processing circuit 51 as dedicated hardware, and the respiratory state acquisition unit 132 and the sleep abnormality detection unit 133 may be implemented by the processing circuit 51 reading and executing programs stored in a memory 53.

[0153] In this way, the processing circuitry 51 can realize each of the above-described functions by hardware, software, firmware, or a combination of these.

[0154] It should be noted that the embodiments may be freely combined, or any of the components in each embodiment may be modified, or any of the components in each embodiment may be omitted.

[0155] The occupant sleep abnormality detection system 1 according to the present disclosure is capable of detecting occupant sleep abnormalities, and is suitable for use as an occupant sleep abnormality detection system that detects occupant sleep abnormalities.

[0156] 1 Occupant sleep abnormality detection system, 11 Occupant sleep state detection device, 12 Occupant respiratory state detection device, 13 Occupant sleep abnormality detection device, 14 Occupant assistance device, 15 Recording device, 16 Display device, 17 Notification device, 51 Processing circuit, 52 CPU, 53 Memory, 131 Sleep state detection unit, 132 Respiratory state acquisition unit, 133 Sleep abnormality detection unit, 134 Occupant assistance unit, 135 Recording control unit, 136 Notification control unit.

Claims

1. An occupant sleep abnormality detection system comprising: an occupant sleep state detection device that detects information regarding the sleep state of an occupant of a mobile body; an occupant respiratory state detection device that detects information regarding the respiratory state of an occupant of the mobile body; and an occupant sleep abnormality detection device, wherein the occupant sleep abnormality detection device comprises: a sleep state detection unit that detects the sleep state of the occupant based on the detection result by the occupant sleep state detection device; a respiratory state acquisition unit that acquires information regarding the occupant's respiratory state based on the detection result by the occupant respiratory state detection device; and a sleep abnormality detection unit that detects whether the occupant has a sleep abnormality based on the occupant's sleep state detected by the sleep state detection unit and the information regarding the occupant's respiratory state acquired by the respiratory state acquisition unit.

2. The occupant sleep abnormality detection system according to claim 1, characterized in that the occupant sleep state detection device detects information relating to the occupant's sleep state from a captured image, and the occupant respiratory state detection device is a millimeter-wave radar.

3. The occupant sleep abnormality detection system according to claim 1, characterized in that the occupant sleep state detection device detects that the occupant is asleep when it detects from the captured image that the occupant's face is covered.

4. The occupant sleep abnormality detection system according to claim 1, characterized in that the occupant sleep state detection device detects information relating to the occupant's sleep state from the captured image, and the occupant respiratory state detection device detects audio information as information relating to the occupant's respiratory state.

5. The occupant sleep abnormality detection system according to claim 1, characterized in that the occupant sleep state detection device detects information relating to the occupant's sleep state from the captured image, and the occupant respiratory state detection device detects information relating to the occupant's respiratory state from the captured image.

6. The occupant sleep abnormality detection system according to claim 1, characterized in that the sleep abnormality detected by the sleep abnormality detection unit is an apneic sleep state.

7. The occupant sleep abnormality detection system according to claim 1, wherein the sleep abnormality detected by the sleep abnormality detection unit is a state of sleep paralysis or a state of cardiopulmonary collapse.

8. The passenger sleep abnormality detection system according to claim 1, wherein the sleep abnormality detected by the sleep abnormality detection unit is teeth grinding or mental stress.

9. An occupant sleep abnormality detection system as described in claim 1, characterized in that it comprises an occupant assistance device for eliminating sleep abnormalities, and the occupant sleep abnormality detection device comprises an occupant assistance unit that drives and controls the occupant assistance device so as to eliminate the sleep abnormality for an occupant whose sleep abnormality has been detected by the sleep abnormality detection unit.

10. The occupant sleep abnormality detection system according to claim 9, wherein the occupant support device is a reclining seat, and the occupant support unit controls the drive to change the reclining state of the occupant support device.

11. The occupant sleep abnormality detection system according to claim 9, characterized in that the occupant assistance device is an automatic driving control device, and the occupant assistance unit controls the occupant assistance device to generate acceleration in at least one of the lateral and longitudinal directions in the moving body.

12. The occupant sleep abnormality detection system according to claim 9, characterized in that the occupant support device is a tactile stimulus generator, an air conditioning stimulus generator, a light stimulus generator, or a sound stimulus generator, and the occupant support unit drives and controls the occupant support device to provide the stimulus to the occupant.

13. The occupant sleep abnormality detection system according to claim 9, characterized in that the occupant assistance device is capable of executing a plurality of occupant assistance methods, and the occupant assistance unit selects an occupant assistance method to be used by the occupant assistance device according to the type of sleep abnormality detected by the sleep abnormality detection unit, and then performs drive control.

14. An occupant sleep abnormality detection system as described in claim 1, characterized in that it comprises an occupant assistance device for eliminating a driver's sleep abnormality, and the occupant sleep abnormality detection device comprises an occupant assistance unit that drives and controls the occupant assistance device so as to eliminate the sleep abnormality when the sleep abnormality detection unit detects a driver's sleep abnormality while the mobile body is operating fully autonomously.

15. The occupant sleep abnormality detection system according to claim 1, characterized in that the sleep abnormality detection unit detects the presence or absence of a sleep abnormality using a trained machine learning model in which information on the sleep state and breathing state is used as input variables and the presence or absence of the occupant's sleep abnormality is used as an output variable.

16. The occupant sleep abnormality detection system of claim 9, characterized in that the sleep abnormality detection unit detects the presence or absence of signs of sleep abnormality using a trained machine learning model that uses information on the sleep state and breathing state as input variables and signs of the occupant's sleep abnormality as output variables, and the occupant assistance unit drives and controls the occupant assistance device so as to eliminate the signs of sleep abnormality for an occupant in whom the sleep abnormality detection unit has detected the occurrence of signs of sleep abnormality.

17. The occupant sleep abnormality detection system according to claim 1, further comprising a recording control unit that, when the sleep abnormality detection unit detects the occurrence of an occupant's sleep abnormality, records information regarding the occupant's sleep abnormality in a storage device.

18. An occupant sleep abnormality detection system as described in claim 1, characterized in that it is equipped with an alarm control unit that, when the sleep abnormality detection unit detects the occurrence of an occupant's sleep abnormality, alerts information regarding the occupant's sleep abnormality to occupants other than the occupant or to the outside of the vehicle using an alarm device.

19. An occupant sleep abnormality detection device comprising: a sleep state detection unit that detects the sleep state of an occupant of a mobile body based on the detection results of an occupant sleep state detection device that detects information regarding the sleep state of the occupant of the mobile body; a respiratory state acquisition unit that acquires information regarding the respiratory state of the occupant of the mobile body based on the detection results of an occupant respiratory state detection device that detects information regarding the respiratory state of the occupant of the mobile body; and a sleep abnormality detection unit that detects whether the occupant has a sleep abnormality based on the sleep state of the occupant detected by the sleep state detection unit and the information regarding the respiratory state of the occupant acquired by the respiratory state acquisition unit.

20. A method for detecting sleep abnormalities in occupants, comprising: a step in which a sleep state detection unit detects the sleep state of an occupant of a mobile body based on a detection result from an occupant sleep state detection device that detects information related to the sleep state of the occupant; a step in which a respiratory state acquisition unit acquires information related to the respiratory state of the occupant of the mobile body based on a detection result from an occupant respiratory state detection device that detects information related to the respiratory state of the occupant; and a step in which a sleep abnormality detection unit detects whether or not the occupant has a sleep abnormality based on the sleep state of the occupant detected by the sleep state detection unit and the information related to the respiratory state of the occupant acquired by the respiratory state acquisition unit.

Images