Occupant status detection system

Abstract

To provide an occupant state detection system that optimizes an occupant monitoring method when a vehicle is in a running state and an occupant monitoring method when the vehicle is in a stopped state, and can always accurately detect the physical states of occupants.SOLUTION: An occupant state detection system 1 includes: an occupant monitoring device 10 that monitors a physical state of an occupant of a vehicle; an occupant state detection device 200 that detects the physical state of the occupant of the vehicle based on occupant monitoring information from the occupant monitoring device 10; and a discrimination device 100 that discriminates between a running state and a stopped state of the vehicle. The occupant state detection device 200 includes a processor 210, and a memory 220 communicably connected to the processor 210. An information receiving unit 211 of the processor 210 receives the occupant monitoring information, and discrimination information from the discrimination device 100, stores the received occupant monitoring information and discrimination information in the memory 220, assigns a monitoring part of the occupant of the vehicle monitored by the occupant monitoring device 10 based on the discrimination information, and detects the physical state of the occupant.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an occupant condition detection system. [Background technology]

[0002] In recent years, so-called driver monitoring systems that monitor the state of the driver to support safe driving of a vehicle have been developed. In such driver monitoring systems, for example, a system detects where the driver is looking and issues a warning to the driver if the driver is not aware of the current situation.

[0003] For example, as one such device, an occupant monitoring device that monitors occupants including the driver has been disclosed that can reduce the number of imaging devices used even when the number of occupants to be monitored increases (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-202726 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology described in Patent Document 1 monitors the occupants, including the driver, while the vehicle is traveling, and has the problem that it does not monitor the occupants, including the driver, or determine the physical condition of the occupants when the vehicle is stopped. In addition, since the objects monitored by the occupant monitoring device are significantly different when the vehicle is moving and when it is stopped, there was also the problem that even if the same monitoring control was applied when the vehicle was stopped as when it was moving, the status of the occupants could not be correctly detected.

[0006] Therefore, the present invention has been made in consideration of the above-mentioned problems, and aims to provide an occupant condition detection system that can always accurately detect the physical condition of occupants by optimizing the monitoring method when the vehicle is in a traveling state and the monitoring method when the vehicle is in a stopped state. [Means for solving the problem]

[0007] Mode 1: One or more embodiments of the present invention include an occupant monitoring device that monitors the physical state of an occupant of a vehicle, an occupant state detection device that detects the physical state of an occupant of the vehicle based on occupant monitoring information from the occupant monitoring device, and a discrimination device that discriminates between a running state and a stopped state of the vehicle, the occupant state detection device including one or more processors and one or more memories communicatively connected to the one or more processors, the discrimination device acquires ignition information, and when the ignition is in an OFF state, transmits discrimination information to the one or more processors that the vehicle is in the stopped state, and when the ignition is in an ON state, transmits the discrimination information to the one or more processors that the vehicle is in the running state; the occupant monitoring device includes a camera that captures images of occupants of the vehicle, and transmits the occupant monitoring information acquired from image information capturing an image of the occupant's face or entire body to the one or more processors; the one or more processors receive the occupant monitoring information and the discrimination information from the discrimination device, and store the received occupant monitoring information and the discrimination information in the one or more memories; When the discrimination information indicates the stopped state, the passenger monitoring device detects at least one of the passenger's resting state and sleep quality by allocating a monitoring portion to be monitored by the passenger to the entire body, and when the discrimination information indicates the running state, the passenger monitoring device detects at least one of the passenger's sleep state and fatigue state by allocating a monitoring portion to be monitored by the passenger to the face. We propose an occupant status detection system. Effect of the Invention

[0015] According to one or more embodiments of the present invention, the monitoring method when the vehicle is moving and the monitoring method when the vehicle is stopped are optimized, and it is possible to always accurately detect the physical condition of the occupants. [Brief description of the drawings]

[0016] [Figure 1] 1 is a diagram showing a configuration of an occupant state detection system according to a first embodiment of the present invention. [Diagram 2] 1 is a diagram showing a configuration of an occupant state detection device in an occupant state detection system according to a first embodiment of the present invention. [Diagram 3] 3 is a diagram showing the relationship between monitoring allocation processing and occupant monitoring information in the occupant state detection system according to the first embodiment of the present invention. FIG. [Figure 4]1 is a process flow diagram of an occupant state detection system according to a first embodiment of the present invention. [Diagram 5] FIG. 4 is a diagram showing a configuration of an occupant state detection system according to a second embodiment of the present invention. [Figure 6] FIG. 11 is a diagram showing the relationship between monitoring allocation processing and occupant monitoring information in an occupant state detection system according to a second embodiment of the present invention. [Figure 7] FIG. 6 is a process flow diagram of an occupant state detection system according to a second embodiment of the present invention. [Figure 8] FIG. 11 is a diagram showing a configuration of an occupant state detection system according to a third embodiment of the present invention. [Figure 9] FIG. 11 is a diagram showing the relationship between monitoring allocation processing and occupant monitoring information in an occupant state detection system according to a third embodiment of the present invention. [Figure 10] FIG. 11 is a process flow diagram of an occupant state detection system according to a third embodiment of the present invention. [Figure 11] FIG. 13 is a diagram showing a configuration of an occupant state detection system according to a fourth embodiment of the present invention. [Figure 12] FIG. 13 is a diagram showing the relationship between monitoring allocation processing and occupant monitoring information in an occupant state detection system according to a fourth embodiment of the present invention. [Figure 13] FIG. 11 is a process flow diagram of an occupant state detection system according to a third embodiment of the present invention. [Figure 14] FIG. 13 is a diagram showing a configuration of an occupant state detection system according to another embodiment of the present invention. [Figure 15] FIG. 11 is a diagram showing the relationship between a monitoring allocation process and occupant monitoring information in an occupant state detection system according to another embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, an embodiment of the present invention will be described with reference to FIGS.

[0018] <First embodiment> An occupant state detection system 1 according to this embodiment will be described with reference to FIGS. 1 to 4. FIG.

[0019] <Configuration of occupant state detection system 1> As shown in FIG. 1, an occupant state detection system 1 according to this embodiment includes an occupant monitoring device 10, a discrimination device 100, and an occupant state detection device 200.

[0020] The occupant monitoring device 10 monitors the physical condition of the vehicle occupants using a camera that has an imaging element such as a CCD (Charge Coupled Device) or a CIS (CMOS Image Sensor) built in and captures images (including moving images and still images) of the occupants in the vehicle captured by the imaging element. The camera may be a dedicated camera, but if an occupant recognition device is provided, it is preferable to use a camera in the occupant recognition device. Also, it is preferable that the camera is composed of both an optical camera and a near-infrared camera. In the occupant state detection system 1 according to this embodiment, the occupant monitoring device 10 is suitable for monitoring information such as the facial expression, posture, and behavior of the occupant that can be observed from the outside.

[0021] Examples of information about occupants that can be primarily obtained from the occupant monitoring device 10 include the direction of gaze, face direction, number of blinks, eye opening, eye movement, posture, body movement, number of turns, sleep time, sleep state, rest time, etc. Here, the degree of eye opening can be used as an indicator for detecting drowsiness, the number of blinks can be used as an indicator for detecting the degree of eye fatigue or lack of sleep, and the degree of eye opening can be used as an indicator for detecting the sleeping state of the occupants, etc. The information monitored by the occupant monitoring device 10 will be described in detail later.

[0022] The discrimination device 100 discriminates between a traveling state and a stopped state of the vehicle. The discrimination device 100, for example, acquires ignition information from a CPU (Central Processing Unit) of the vehicle (not shown), and when the ignition is in an ON state, determines that the vehicle is in a running state, and when the ignition is in an OFF state, determines that the vehicle is in a stopped state. The discrimination device 100 may further monitor the vehicle speed pulse, the operation state of the parking brake, and the like, and discriminate between the running state and the stopped state of the vehicle in combination with the ON / OFF state of the ignition.

[0023] The occupant condition detection device 200 detects physical information of the vehicle occupant based on the occupant monitoring information from the occupant monitoring device 10. Specifically, when the occupant state detection device 200 receives information from the discrimination device 100 that the vehicle is stopped, it assigns the monitoring locations of the vehicle occupants to be monitored by the occupant monitoring device 10 and the acquired information. The occupant condition detection device 200 receives occupant monitoring information from the occupant monitoring device 10 to which the occupant's monitoring location and acquired information have been assigned, and detects the occupant's physical condition based on the occupant monitoring information.

[0024] <Configuration of occupant state detection device 200> As shown in FIG. 2, the occupant state detection device 200 according to this embodiment includes a processor 210 and a memory 220.

[0025] The processor 210 controls the entire occupant state detection device 200 in accordance with a control program stored in a memory 220, which will be described later. In this embodiment, the processor 210 particularly executes the functions of an information receiving unit 211, a monitoring allocation unit 212, an occupant state detection unit 213, and the like, which will be described later.

[0026] The memory 220 includes a Read Only Memory (ROM) and a Random Access Memory (RAM), and the like. The ROM stores the above-mentioned control programs and the like, and the RAM stores various data and the like. In this embodiment, for example, passenger monitoring information from the passenger monitoring device 10 and the like are stored in the RAM.

[0027] <Configuration of the processor 210> As shown in FIG. 2, the processor 210 according to this embodiment includes an information receiving unit 211, a monitoring allocation unit 212, and an occupant state detection unit 213.

[0028] The information receiving unit 211 receives information from the discrimination device 100 indicating that the vehicle is in a moving state or that the vehicle is in a stopped state. When the information receiving unit 211 receives information from the discrimination device 100 that the vehicle is in a stopped state, the information receiving unit 211 transmits the information to a monitoring allocation unit 212, which will be described later. Moreover, the information receiving unit 211 receives passenger monitoring information from the passenger monitoring device 10 and stores the received passenger monitoring information in the memory 220 .

[0029] The monitoring allocation unit 212 receives the discrimination result of the discrimination device 100 from the information receiving unit 211, and further receives information regarding the sleeping state of the occupants from the occupant state detection unit 213 described later, and allocates the monitoring areas and acquired information of the vehicle occupants detected in the occupant monitoring device 10. Specifically, as shown in FIG. 3, when the monitoring allocation unit 212 receives information from the information receiving unit 211 that the vehicle is stopped, and further receives information from the occupant state detection unit 213 that the occupant is not asleep, it assigns the monitoring part of the occupant monitoring device 10 to the "face" and assigns the acquired information to "facial expression" (allocation process A). In addition, when the monitoring allocation unit 212 receives, for example, information from the information receiving unit 211 that the vehicle is stopped, and further receives information from the occupant state detection unit 213 that the occupant is asleep, it assigns the monitoring area of ​​the occupant monitoring device 10 to "whole body" and assigns the acquired information to "behavior" (allocation process B).

[0030] The occupant condition detection unit 213 detects the physical condition of the occupant based on the occupant monitoring information from the occupant monitoring device 10 stored in the memory 220 . Specifically, as shown in FIG. 3, the occupant state detection unit 213 detects the sleep state, fatigue state, etc. of the vehicle occupant based on occupant monitoring information (information regarding the occupant's "facial expression") obtained when the allocation process A is executed, such as "direction of gaze, direction of face, eye opening, number of blinks, eye movement," etc. That is, when the monitoring part of the occupant monitor 10 is set to "face", the occupant state detector 213 detects in real time whether the occupant is dozing while the vehicle is stopped, the direction of the driver's gaze, and the like. For this purpose, the occupant state detection unit 213 continuously executes detection processing based on occupant monitoring information such as the occupant's eye movement and quick movements such as blinking.

[0031] In addition, the occupant status detection unit 213 detects the resting state and sleep quality of the occupant in the vehicle based on occupant monitoring information (information regarding the occupant's "behavior") obtained when the allocation process B is executed, such as "posture, body movement, sleeping time, number of turns, rest time," etc. Therefore, in the occupant status detection unit 213, when the monitoring area of ​​the occupant monitoring device 10 is set to "whole body," a low-load detection process is continuously executed based on occupant monitoring information such as the occupant turning over, slow body movements, etc., as opposed to the monitoring process for the "face."

[0032] <Processing of Occupant State Detection System 1> The processing of the occupant state detection system 1 according to this embodiment will be described with reference to FIG.

[0033] The monitoring allocation unit 212 of the processor 210 determines whether or not information indicating a stopped state of the vehicle with the ignition turned off has been received from the discrimination device 100 via the information receiving unit 211 (step S110). When the monitoring allocation unit 212 determines that it has not received information indicating that the vehicle is stopped with the ignition turned off ("NO" in step S110), it returns the process to step S110 and transitions to a standby state. In addition, in the standby state, for example, the occupant state detection process that is normally performed when the vehicle is traveling is continuously executed.

[0034] On the other hand, when the monitoring allocation unit 212 determines that information indicating a stationary state of the vehicle with the ignition turned off has been received ("YES" in step S110), the process proceeds to step S120.

[0035] The monitoring allocation unit 212 of the processor 210 determines whether or not information indicating that the occupant is asleep has been received from the occupant state detection unit 213 (step S120). When the monitoring allocation unit 212 receives information indicating a sleeping state ("YES" in step S120), the process proceeds to step S140. On the other hand, if the monitoring allocation unit 212 determines that the information indicating a sleeping state has not been received ("NO" in step S120), the process proceeds to step S130.

[0036] If the monitoring allocation unit 212 determines that information indicating a sleeping state has not been received ("NO" in step S120), the monitoring allocation unit 212 of the processor 210 executes allocation process A (step S130) and transitions to step S150. As described above, in the assignment process A (step S130), the monitoring assignment unit 212 executes a process of assigning the monitoring part of the occupant monitoring device 10 to the “face” and the acquired information to the “facial expression”, and transmitting a signal to that effect to the occupant monitoring device 10.

[0037] When the monitoring allocation unit 212 receives information indicating a sleeping state ("YES" in step S120), the monitoring allocation unit 212 of the processor 210 executes allocation process B (step S140) and transitions to step S150. As described above, in the assignment process B (step S140), the monitoring assignment unit 212 executes a process of assigning the monitoring body parts of the vehicle occupant detected by the occupant monitoring device 10 to “whole body”, assigning the acquired information to “behavior”, and transmitting a signal to that effect to the occupant monitoring device 10.

[0038] Here, after the monitoring allocation unit 212 executes the allocation process A (step S130) or the allocation process B (step S140), the processor 210 permits the occupant monitoring device 10 that has already been started to output information to the information receiving unit 211. The processor 210 may allow the information receiving unit 211 to receive the occupant monitoring information from the occupant monitoring device 10 after the monitoring allocation unit 212 executes the allocation process A (step S130) or the allocation process B (step S140). Alternatively, after executing allocation process A (step S130) or allocation process B (step S140) in the monitoring allocation unit 212, the processor 210 may allow the information receiving unit 211 to write information from the occupant monitoring device 10 to the memory 220.

[0039] The information receiving unit 211 receives the occupant monitoring information from the occupant monitoring device 10 (step S150). Then, the information receiving unit 211 stores the occupant monitoring information received from the occupant monitoring device 10 in the memory 220 (step S150).

[0040] The occupant condition detection unit 213 detects the physical condition of the occupant based on the occupant monitoring information from the occupant monitoring device 10 stored in the memory 220 (step S160). In addition, when the occupant monitoring device 10 assigns the monitoring part to "face" and the acquired information to "facial expression" through assignment process A (step S130), the occupant state detection unit 213 detects the state of the occupant while the vehicle is stopped, such as whether they are dozing off, the direction of their gaze, etc. In addition, when the occupant monitoring device 10 assigns the monitoring body part to “whole body” and the acquired information to “behavior” through assignment process B (step S140), the occupant state detection unit 213 detects the resting state, sleep quality, etc. of the occupant while the vehicle is stopped.

[0041] Then, the processor 210 determines whether or not to continue the occupant state detection process (step S170). When the processor 210 determines to continue the occupant state detection process ("YES" in step S170), the process returns to step S110, and the occupant state detection process continues. On the other hand, if the processor 210 determines not to continue the occupant state detection process ("NO" in step S170), the occupant state detection process ends.

[0042] <Actions and Effects> As described above, the occupant status detection system 1 of this embodiment comprises an occupant monitoring device 10 that monitors the physical status of vehicle occupants, an occupant status detection device 200 that detects the physical status of vehicle occupants based on occupant monitoring information from the occupant monitoring device 10, and a discrimination device 100 that discriminates between a running status and a stopped status of the vehicle, and the occupant status detection device 200 comprises a processor 210 and a memory 220 that is communicatively connected to the processor 210, and the information receiving unit 211 of the processor 210 receives the occupant monitoring information and discrimination information from the discrimination device 100, stores the received occupant monitoring information and discrimination information in the memory 220, and assigns monitoring areas of the vehicle occupants monitored by the occupant monitoring device 10 based on the discrimination information, and detects the physical status of the occupant. That is, the occupant condition detection system 1 detects the physical condition of a vehicle occupant using the occupant monitoring device 10 to which monitoring parts of the vehicle occupant and acquired information are assigned based on vehicle discrimination information. Therefore, in detecting the physical condition of the occupant, the monitoring method when the vehicle is stopped is optimized, so that the physical condition of the occupant can be detected accurately at all times.

[0043] Moreover, the discrimination device 100 in the occupant state detection system 1 according to this embodiment acquires ignition information, and transmits information to the processor 210 that the vehicle is in a stopped state when the ignition is in an OFF state. In other words, the discrimination device 100 discriminates whether the vehicle is in a running state or a stopped state based on the ON / OFF state of the ignition. Therefore, the occupant state detection system 1 can uniquely detect whether the vehicle is traveling or stopped without using any special sensor or the like.

[0044] In addition, the occupant monitoring device 10 in the occupant state detection system 1 of this embodiment includes a camera that captures images of the vehicle occupants, acquires occupant monitoring information including the behavior and facial expressions of the vehicle occupants, and transmits it to the processor 210. In other words, in the occupant condition detection system 1, the occupant monitoring device 10 acquires occupant monitoring information including the behavior and facial expressions of the vehicle occupants as information for detecting the physical condition of the occupants, taking into account the characteristics of the built-in camera. Therefore, the occupant state detection system 1 can obtain highly accurate information about the behavior, facial expressions, and the like of the vehicle occupant, thereby improving the accuracy of detecting the physical state of the vehicle occupant. Furthermore, if the camera is configured with both an optical camera and a near-infrared camera, images of the occupants can be obtained regardless of day or night. Therefore, the physical condition of the occupants can be accurately grasped at all times while the vehicle is stopped, regardless of the time of day.

[0045] In addition, when the processor 210 in the occupant state detection system 1 of this embodiment determines that the occupant is asleep based on the occupant monitoring information when the discrimination information indicates a stopped state, it assigns the monitoring area to the "whole body" of the vehicle occupant and detects the physical state of the vehicle occupant. That is, when the discrimination information indicates a stopped state and the occupant is determined to be asleep, the monitoring portion of the vehicle occupant is assigned to the occupant's "whole body" to detect the physical state of the occupant. Therefore, by allocating the monitoring area of ​​the vehicle occupant to the "whole body" of the occupant, the processing load of the processor 210 can be reduced, and the power consumption of the processor 210 can be reduced. Furthermore, the processor 210 can utilize the resources saved by the reduced processing load on the processor 210 for a new detection function or the like.

[0046] In addition, when the processor 210 in the occupant state detection system 1 of this embodiment determines that the occupant is asleep based on the occupant monitoring information when the discrimination information indicates a stopped state, it assigns the forward viewing area to a part other than the face of the vehicle occupant and detects the physical state of the vehicle occupant. That is, when the discrimination information indicates a stopped state and the occupant is determined to be asleep, the monitoring body part of the vehicle occupant is assigned to "whole body" and the physical state of the occupant is detected. Therefore, by allocating the monitoring area of ​​the vehicle occupant to the "whole body" of the occupant, the processing load of the processor 210 can be reduced, and the power consumption of the processor 210 can be reduced. Furthermore, the processor 210 can utilize the resources saved by the reduced processing load on the processor 210 for a new detection function or the like.

[0047] <Second embodiment> An occupant state detection system 1A according to this embodiment will be described with reference to FIGS.

[0048] <Configuration of occupant state detection system 1A> As shown in FIG. 5, an occupant state detection system 1A according to this embodiment includes an occupant monitoring device 10, a discrimination device 100, and an occupant state detection device 200A. In addition, since the components having the same reference numerals as those in the first embodiment have the same functions, detailed description thereof will be omitted.

[0049] <Configuration of occupant state detection device 200A> As shown in FIG. 5, the occupant state detection device 200A includes a processor 210A and a memory 220.

[0050] When the occupant state detection device 200A receives information from the discrimination device 100 that the vehicle is stopped, the occupant state detection device 200A assigns the monitoring locations of the vehicle occupants to be monitored by the occupant monitoring device 10 and the acquired information. The details of the process of allocating monitored portions and acquired information executed by the occupant state detection device 200A will be described later. In addition, the occupant state detection device 200A receives occupant monitoring information from the occupant monitoring device 10. The occupant condition detection device 200A detects the physical condition of the occupant based on the occupant monitoring information from the occupant monitoring device 10.

[0051] <Configuration of the Processor 210A> As shown in FIG. 5, the processor 210A includes an information receiving unit 211A, a monitoring allocation unit 212A, and an occupant state detection unit 213A.

[0052] The information receiving unit 211A stores the occupant monitoring information received from the occupant monitoring device 10 in the memory 220.

[0053] The monitoring allocation unit 212A allocates the monitoring locations of the vehicle occupants detected by the occupant monitoring device 10 and the acquired information when the monitoring allocation unit 212A receives information indicating that the vehicle state is a traveling state from the information receiving unit 211A. Specifically, as shown in FIG. 6, when the monitoring allocation unit 212A receives information from the information receiving unit 211A that the vehicle is in a traveling state, it assigns the monitoring target of the occupant monitoring device 10 to “face” and assigns the acquired information to “facial expression” (allocation process C). Furthermore, when the monitoring allocation unit 212A receives information from the information receiving unit 211A that the vehicle is stopped, it allocates the monitoring area of ​​the occupant monitoring device 10 to "whole body" and allocates the acquired information to "behavior" (allocation process D).

[0054] The occupant condition detection unit 213A detects the physical condition of the occupant based on the occupant monitoring information from the occupant monitoring device 10 stored in the memory 220. Specifically, as shown in FIG. 6, the occupant status detection unit 213A detects whether a vehicle occupant is drowsy, fatigued, looking away, etc., based on occupant monitoring information (information regarding the occupant's "facial expression") obtained when the allocation process C is executed, such as "direction of gaze, direction of face, eye opening, number of blinks, eye movement," etc. That is, when the occupant monitor 10 is set to "face", the occupant state detector 213A detects in real time whether the occupant is dozing while driving or the like. For this purpose, the occupant state detection unit 213A continuously executes detection processing based on occupant monitoring information such as the occupant's eye movement, blinking, and other quick movements.

[0055] In addition, the occupant status detection unit 213A detects the resting state, sleeping state, sleep quality, etc. of the occupant in the vehicle based on occupant monitoring information (information regarding the occupant's "behavior") obtained when the allocation process D is executed, such as "posture, body movement, sleeping time, number of turns, rest time," etc. Therefore, in the occupant status detection unit 213A, when the monitoring area of ​​the occupant monitoring device 10 is set to "whole body", a low-load detection process is executed based on occupant monitoring information such as the occupant turning over, body movements, and other slow movements, unlike monitoring of the "face".

[0056] <Processing of Occupant State Detection System 1A> The process of the occupant state detection system 1A according to this embodiment will be described with reference to FIG.

[0057] The monitoring allocation unit 212A of the processor 210A determines whether or not information indicating the traveling state of the vehicle with the ignition turned on has been received from the discrimination device 100 via the information receiving unit 211A (step S210). At this time, if the monitoring allocation unit 212A determines that it has received information indicating the traveling state of a vehicle with the ignition turned on ("YES" in step S210), it transitions to step S220 and executes allocation process C (step S220). As described above, in the allocation process C, the monitoring part of the occupant monitoring device 10 is assigned to the "face", and the acquired information is assigned to the "facial expression".

[0058] On the other hand, if the monitoring allocation unit 212A of the processor 210A determines that it has received information indicating a stopped state of the vehicle with the ignition turned off ("NO" in step S210), it transitions to step S230 and executes allocation process D (step S230). As described above, in the allocation process D, the monitoring body part of the occupant monitoring device 10 is assigned to "whole body", and the acquired information is assigned to "behavior".

[0059] The information receiving unit 211A receives the occupant monitoring information from the occupant monitoring device 10 (step S240). Then, the information receiving unit 211A stores the occupant monitoring information received from the occupant monitoring device 10 in the memory 220 (step S240).

[0060] The occupant condition detection unit 213A detects the physical condition of the occupant based on the occupant monitoring information of the occupant monitoring device 10 stored in the memory 220 (step S250). When the occupant monitoring device 10 assigns the monitoring part to "face" and the acquired information to "facial expression" through assignment process C (step S220), the occupant state detection unit 213A detects whether the occupant is dozing, tired, looking away, etc. while driving. In addition, when the occupant monitoring device 10 assigns the monitoring body part to “whole body” and the acquired information to “behavior” through assignment process D (step S230), the occupant state detection unit 213A detects the resting state, sleeping state, sleep quality, etc. of the occupant while the vehicle is stopped.

[0061] The processor 210A determines whether to continue the occupant state detection process (step S260). When the processor 210A determines to continue the occupant state detection process ("YES" in step S260), the process returns to step S210, and the occupant state detection process continues. On the other hand, if the processor 210A determines not to continue the occupant state detection process ("NO" in step S260), the occupant state detection process ends.

[0062] <Actions and Effects> As described above, the processor 210A in the occupant state detection system 1A of this embodiment assigns the monitoring area to the face of the vehicle occupant when the discrimination information indicates a moving state, and assigns the monitoring area to the entire body of the vehicle occupant when the discrimination information indicates a stopped state, thereby detecting the physical state of the vehicle occupant. That is, the occupant condition detection system 1 detects the physical condition of the vehicle occupant based on the vehicle discrimination information, using the occupant monitoring device 10 to which the monitoring parts of the vehicle occupant and the acquired information are respectively assigned. Therefore, when detecting the physical condition of an occupant, the monitoring method when the vehicle is moving and the monitoring method when the vehicle is stopped are optimized, so that the physical condition of the occupant can always be detected accurately. Furthermore, when the vehicle is stopped, the monitoring area of ​​the vehicle occupant is assigned to the "whole body" of the occupant, thereby reducing the processing load of the processor 210A and the power consumption of the processor 210A. Furthermore, the processor 210A can utilize the resources saved by the reduced processing load on the processor 210A for a new detection function or the like.

[0063] <Third embodiment> An occupant state detection system 1B according to this embodiment will be described with reference to FIGS.

[0064] <Configuration of occupant state detection system 1B> As shown in FIG. 8, an occupant state detection system 1B according to this embodiment includes an occupant monitoring device 10, a discrimination device 100, and an occupant state detection device 200B.

[0065] When the occupant state detection device 200B receives information from the discrimination device 100 that the vehicle is stopped, it assigns the monitoring locations of the vehicle occupants to be monitored by the occupant monitoring device 10 and the acquired information. The details of the process of allocating monitored portions and acquired information executed by the occupant state detection device 200B will be described later. In addition, the occupant state detection device 200B receives occupant monitoring information from the occupant monitoring device 10. The occupant condition detection device 200B detects the physical condition of the occupant based on the occupant monitoring information from the occupant monitoring device 10.

[0066] <Configuration of the Processor 210B> As shown in FIG. 8, the processor 210B includes an information receiving unit 211B, a monitoring allocation unit 212B, and an occupant state detection unit 213B.

[0067] The information receiving unit 211B stores the occupant monitoring information received from the occupant monitoring device 10 in the memory 220.

[0068] The monitoring allocation unit 212B allocates the monitoring locations of the vehicle occupants detected by the occupant monitoring device 10 and the acquired information when the monitoring allocation unit 212B receives information indicating that the vehicle state is a traveling state from the information receiving unit 211B. Specifically, as shown in FIG. 9, when the monitoring allocation unit 212B receives information from the information receiving unit 211B that the vehicle is in a traveling state, for example, in the occupant monitoring device 10, it assigns the monitoring part to the “driver” and the acquired information to the “facial expression” (allocation process E). In addition, when the monitoring allocation unit 212B receives information from the information receiving unit 211B that the vehicle is stopped, it assigns, for example, the monitoring area of ​​the occupant monitoring device 10 to “all occupants” and the acquired information to “behavior” (allocation process F).

[0069] The occupant condition detection unit 213B detects the physical condition of the occupant based on the occupant monitoring information from the occupant monitoring device 10 stored in the memory 220. Specifically, as shown in FIG. 9, the occupant status detection unit 213B detects whether the driver of the vehicle is drowsy, fatigued, looking away, etc., based on occupant monitoring information (information regarding the occupant's "facial expression") obtained when the allocation process E is executed, such as "direction of gaze, direction of face, eye opening, number of blinks, eye movement," etc. That is, when the monitoring part of the occupant monitor 10 is set to "driver", the occupant state detector 213B detects in real time whether the driver is dozing off or looking elsewhere while driving. For this purpose, the occupant state detection unit 213B continuously executes detection processing based on occupant monitoring information such as the driver's eye movement, blinking, and other quick movements.

[0070] In addition, the occupant status detection unit 213B detects the resting state, sleeping state, sleep quality, etc. of all occupants in the vehicle based on occupant monitoring information (information regarding the occupant's "behavior") obtained when the allocation process F is executed, such as "posture, body movement, sleeping time, number of turns, rest time," etc. Therefore, in the occupant status detection unit 213B, when the monitoring area of ​​the occupant monitoring device 10 is set to "all occupants," a low-load detection process is executed based on occupant monitoring information such as the occupant turning over, body movements, and other slow movements, unlike monitoring of the "driver."

[0071] <Processing of occupant state detection system 1B> The process of the occupant state detection system 1B according to this embodiment will be described with reference to FIG.

[0072] The monitoring allocation unit 212B of the processor 210B determines whether or not information indicating the traveling state of the vehicle with the ignition turned on has been received from the discrimination device 100 via the information receiving unit 211B (step S310).

[0073] When the monitoring allocation unit 212B determines that information indicating the traveling state of a vehicle with the ignition turned on has been received ("YES" in step S310), the process proceeds to step S320, and allocation process E is executed (step S320). As described above, in the assignment process E (step S320), the monitoring assignment unit 212B assigns the monitoring part of the occupant monitoring device 10 to “driver” and the acquired information to “facial expression”, and transmits a signal to that effect to the occupant monitoring device 10.

[0074] On the other hand, if the monitoring allocation unit 212B of the processor 210B determines that it has received information indicating a stopped state of the vehicle with the ignition turned off ("NO" in step S310), it transitions to step S330 and executes allocation process F (step S330). As described above, in the assignment process F (step S330), the monitoring assignment unit 212B executes a process of assigning the monitoring areas of the occupant monitoring device 10 to “all occupants” and the acquired information to “behavior”, and transmitting a signal to that effect to the occupant monitoring device 10.

[0075] The information receiving unit 211B receives the occupant monitoring information from the occupant monitoring device 10 (step S340). Then, the information receiving unit 211B stores the occupant monitoring information received from the occupant monitoring device 10 in the memory 220 (step S340).

[0076] The occupant condition detection unit 213B detects the physical condition of the occupant based on the occupant monitoring information from the occupant monitoring device 10 stored in the memory 220 (step S350). When the occupant monitoring device 10 assigns the monitoring part to the "driver" and the acquired information to the "facial expression" through the assignment process E (step S320), the occupant state detection unit 213B detects the driver's drowsiness, fatigue, distraction, etc. while driving. In addition, when the occupant monitoring device 10 assigns the monitoring area to “all occupants” and the acquired information to “behavior” through assignment process F (step S330), the occupant state detection unit 213B detects the resting state, sleeping state, sleep quality, etc. of all occupants while the vehicle is stopped.

[0077] The processor 210B determines whether to continue the occupant state detection process (step S360). If processor 210B determines to continue the occupant state detection process ("YES" in step S360), the process returns to step S310, and the occupant state detection process continues. On the other hand, if the processor 210B determines not to continue the occupant state detection process ("NO" in step S360), the occupant state detection process ends.

[0078] <Actions and Effects> As described above, the processor 210B in the occupant state detection system 1B of this embodiment assigns monitoring areas to the driver of the vehicle when the discrimination information indicates a driving state, and assigns monitoring areas to all occupants of the vehicle when the discrimination information indicates a stopped state, and detects the physical state of the vehicle occupants. When the discrimination information indicates a driving state, the processor 210B assigns the monitoring area to the driver, and when the discrimination information indicates a stopped state, the processor 210B assigns the monitoring area to all occupants of the vehicle and detects the physical condition of the vehicle occupants. That is, the occupant condition detection system 1B detects the physical condition of a vehicle occupant using the occupant monitoring device 10 to which the monitoring parts and acquired information of the vehicle occupant are respectively assigned based on the vehicle discrimination information. Therefore, when detecting the physical condition of an occupant, the monitoring method when the vehicle is moving and the monitoring method when the vehicle is stopped are optimized, so that the physical condition of the occupant can always be detected accurately. Furthermore, when the vehicle is stopped, the monitoring areas for the vehicle occupants are assigned to "all occupants", thereby reducing the processing load on the processor 210B and the power consumption of the processor 210B. Furthermore, the processor 210B can utilize the resources resulting from the reduction in the processing load of the processor 210B for new functions or the like.

[0079] <Fourth embodiment> An occupant state detection system 1C according to this embodiment will be described with reference to FIGS.

[0080] <Configuration of occupant state detection system 1C> As shown in FIG. 11, an occupant state detection system 1C according to this embodiment includes an occupant monitoring device 10, a discrimination device 100, and an occupant state detection device 200C.

[0081] When the occupant state detection device 200C receives information from the discrimination device 100 that the vehicle is in a stopped state, the occupant state detection device 200C assigns monitoring locations of the vehicle occupant to be monitored by the occupant monitoring device 10. The monitoring portion allocation process executed by the occupant state detection device 200C will be described in detail later. In addition, the occupant state detection device 200C receives occupant monitoring information from the occupant monitoring device 10. The occupant condition detection device 200C detects the physical condition of the occupant based on the occupant monitoring information from the occupant monitoring device 10.

[0082] <Configuration of the processor 210C> As shown in FIG. 8, the processor 210C includes an information receiving unit 211C, a monitoring allocation unit 212C, and an occupant state detection unit 213C.

[0083] The information receiving unit 211C stores the occupant monitoring information received from the occupant monitoring device 10 in the memory 220.

[0084] The monitoring allocation unit 212C allocates monitoring locations of the vehicle occupant detected by the occupant monitoring device 10 when the monitoring allocation unit 212C receives information indicating that the vehicle state is a traveling state from the information receiving unit 211C.

[0085] Specifically, as shown in FIG. 12, when the monitoring allocation unit 212C receives information from the information receiving unit 211C that the vehicle is in a traveling state, it allocates the monitoring area of ​​the occupant monitoring device 10 to the occupant's "face and whole body" (allocation process G). In other words, when the vehicle is traveling, in order to immediately detect abnormal posture, drowsiness, or distraction of the occupant, the monitoring area of ​​the occupant monitoring device 10 is assigned to the "face and whole body," and occupant monitoring information such as "direction of gaze, direction of face, number of blinks, eye opening, eye movement, posture, and body movement" is transmitted to the information receiving unit 211C. Furthermore, when the monitoring allocation unit 212C receives information from the information receiving unit 211C that the vehicle is stopped, it allocates the monitoring areas of the occupant monitoring device 10 to "part of the face and the whole body" of the occupant (allocation process H). In other words, when the vehicle is stopped, there is no need to immediately detect dangerous situations such as occupants falling asleep or looking away, so the monitoring areas of the occupant monitoring device 10 are assigned to "part of the face and the entire body," and occupant monitoring information such as "face direction, eye opening, posture, body movements," etc. are transmitted to the information receiving unit 211C.

[0086] <Processing of occupant state detection system 1C> The process of the occupant state detection system 1C according to this embodiment will be described with reference to FIG.

[0087] The monitoring allocation unit 212C of the processor 210C determines whether or not information indicating the traveling state of the vehicle with the ignition turned on has been received from the discrimination device 100 via the information receiving unit 211C (step S410).

[0088] When the monitoring allocation unit 212C determines that information indicating the traveling state of a vehicle with the ignition turned on has been received ("YES" in step S410), the process proceeds to step S320, and an allocation process G is executed (step S420). As described above, in the allocation process G (step S420), the monitoring allocation unit 212C executes a process of allocating the monitoring body part of the occupant monitoring device 10 to the “face and whole body” of the occupant, and transmitting a signal to that effect to the occupant monitoring device 10.

[0089] On the other hand, if the monitoring allocation unit 212C of the processor 210C determines that it has received information indicating a stopped state of the vehicle with the ignition turned off ("NO" in step S410), it transitions to step S430 and executes allocation process H (step S430). As described above, in the assignment process H (step S430), the monitoring assignment unit 212C assigns the monitoring area of ​​the occupant monitoring device 10 to “part of the face and the entire body” of the occupant, and transmits a signal to that effect to the occupant monitoring device 10.

[0090] The information receiving unit 211C receives occupant monitoring information from the occupant monitoring device 10 to which the monitoring location and acquired information of the vehicle occupant detected by the monitoring allocation unit 212C have been allocated (step S440). Then, the information receiving unit 211C stores the occupant monitoring information received by the occupant monitoring device 10 in the memory 220 (step S440).

[0091] The occupant condition detection unit 213C detects the physical condition of the occupant based on the occupant monitoring information from the occupant monitoring device 10 stored in the memory 220 (step S450). When the occupant monitoring device 10 has assigned the monitoring body part to the occupant's "face and whole body" in the assignment process G (step S420), the occupant condition detection unit 213C detects the occupant's drowsiness, fatigue, inattentiveness, etc. while driving. When the occupant monitoring device 10 has assigned the monitoring area to "part of face and whole body" in the assignment process H (step S430), the occupant state detection unit 213C detects the resting state, sleeping state, etc. of the occupant while the vehicle is stopped.

[0092] The processor 210C determines whether to continue the occupant state detection process (step S460). If processor 210C determines to continue the occupant state detection process ("YES" in step S460), the process returns to step S410, and the occupant state detection process continues. On the other hand, if the processor 210C determines not to continue the occupant state detection process ("NO" in step S460), the occupant state detection process ends.

[0093] <Actions and Effects> As described above, the processor 210C in the occupant state detection system 1C of this embodiment assigns the monitoring areas to the face and entire body of the vehicle occupant when the discrimination information indicates a moving state, and assigns the monitoring areas to a part of the face and the entire body of the vehicle occupant when the discrimination information indicates a stopped state, and detects the physical state of the vehicle occupant. In other words, when the vehicle is in a driving state, the occupant status detection system 1C needs to immediately detect dangerous conditions while driving, and therefore detects drowsiness, looking away, etc. while driving based on occupant monitoring information such as "direction of gaze, direction of face, number of blinks, eye opening, eye movement, posture, body movement, etc." In addition, when the vehicle is stopped, the occupant status detection system 1C does not need to immediately detect dangerous conditions such as drowsiness or looking away, so it detects the physical status of the vehicle occupants based on occupant monitoring information such as "face direction, eye opening, posture, body movement, etc." In other words, the occupant state detection system 1C allocates the monitoring parts so as not to obtain occupant monitor information that is not necessary for detecting the physical state of the occupant when the vehicle is stopped. Therefore, in detecting the physical condition of the occupant, the monitoring method when the vehicle is moving and the monitoring method when the vehicle is stopped are optimized, so that the physical condition of the occupant can be detected accurately at all times. Furthermore, when the vehicle is in motion, detailed information of the occupant's face is monitored, making it possible to detect the occupant's internal state, such as stress, drowsiness, absent-minded driving, fatigue, etc., which cannot be determined from the occupant's external state alone (body movements, posture, etc.). In addition, when the vehicle is stopped, the monitoring area of ​​the vehicle occupant can be assigned to "part of the face and the entire body" of the occupant, thereby reducing the processing load of the processor 210C and the power consumption of the processor 210C. Furthermore, the processor 210C can utilize the resources saved by the reduced processing load on the processor 210C for a new detection function or the like.

[0094] <Other embodiments> As described above, the occupant condition detection systems 1, 1A, 1B, and 1C detect the physical condition of the occupant based on occupant monitoring information obtained from image information captured by a camera provided in the occupant monitoring device 10, but the physical condition of the occupant may also be detected based on occupant monitoring information obtained from a first occupant monitoring device equipped with a camera and a second occupant monitoring device equipped with a millimeter-wave radar. An occupant state detection system 1D including the above-mentioned first occupant monitoring device and second occupant monitoring device will be described below with reference to FIGS.

[0095] <Configuration of occupant state detection system 1D> As shown in FIG. 14, the occupant state detection system 1D according to this embodiment includes a first occupant monitoring device 10a, a second occupant monitoring device 10b, a discrimination device 100, and an occupant state detection device 200D.

[0096] The first passenger monitoring device 10a is the same as the passenger monitoring device 10 provided in the passenger status detection system 1, 1A, 1B, 1C, and acquires passenger monitoring information by a camera. The second passenger monitoring device 10b acquires passenger monitoring information by means of a millimeter wave radar. The millimeter wave radar of the second occupant monitoring device 10b is disposed at a certain distance from the camera of the first occupant monitoring device 10a. Even if an image of the vehicle occupants cannot be obtained by the camera due to obstructions, the vehicle occupants can be monitored by using a millimeter wave radar. That is, the second occupant monitoring device 10b is suitable for obtaining occupant monitoring information regarding an occupant who is in a range that cannot be captured by the camera or in a range that can be partially captured by the camera. For example, the second occupant monitoring device 10b acquires occupant monitoring information of occupants who have reclined their seats to rest or sleep, and of occupants sitting in the rear seats. Furthermore, the second occupant monitoring device 10b can obtain "biological information" such as heart rate, heart rate variability, respiratory rate, and brain waves. That is, the second occupant monitoring device 10b is suitable for obtaining biological information for monitoring the health condition of an occupant that is difficult to observe from the outside.

[0097] <Configuration of occupant state detection device 200D> As shown in FIG. 14, the occupant state detection device 200D includes a processor 210D and a memory 220. The processor 210D and the memory 220 include a memory unit 220A.

[0098] When the occupant state detection device 200D receives information from the discrimination device 100 that the vehicle is stopped, it assigns the monitoring locations and acquired information of the vehicle occupants detected by the first occupant monitoring device 10a and the second occupant monitoring device 10b. The details of the process of allocating monitored portions and acquired information executed by the occupant state detection device 200D will be described later. Moreover, the occupant state detection device 200D receives occupant monitoring information from the first occupant monitoring device 10a and the second occupant monitoring device 10b. The occupant condition detection device 200D detects the physical conditions of the occupants based on the occupant monitoring information acquired by the first occupant monitoring device 10a and the second occupant monitoring device 10b.

[0099] <Configuration of the Processor 210D> As shown in FIG. 14, the processor 210D includes an information receiving unit 211D, a monitoring allocation unit 212D, and an occupant state detection unit 213D.

[0100] The information receiving unit 211D stores in the memory 220 the occupant monitoring information received from the first occupant monitoring device 10a and the second occupant monitoring device 10b.

[0101] When the monitoring allocation unit 212D receives information from the information receiving unit 211D indicating that the vehicle state is in a traveling state, it allocates the monitoring locations and acquired information of the vehicle occupants to be monitored by the first occupant monitoring device 10a and the second occupant monitoring device 10b. Specifically, as shown in FIG. 15, when the monitoring allocation unit 212D receives information from the information receiving unit 211D that the vehicle is in a traveling state, it assigns the monitoring area of ​​the first occupant monitoring device 10a and the second occupant monitoring device 10b to “face.” Then, the monitoring allocation unit 212D allocates the acquired information of the first occupant monitoring device 10a to "facial expression", and allocates the acquired information of the second occupant monitoring device 10b to "facial expression" and "biometric information" (allocation process I). Furthermore, when the monitoring allocation unit 212D receives information from the information receiving unit 211D that the vehicle is stopped, it allocates the monitoring regions of the first occupant monitoring device 10a and the second occupant monitoring device 10b to "whole body." Then, the monitoring allocation unit 212D allocates the acquired information of the first occupant monitoring device 10a to "behavior", and allocates the acquired information of the second occupant monitoring device 10b to "behavior" and "biological information" (allocation process J).

[0102] The occupant condition detection unit 213D detects the physical conditions of the occupants based on the occupant monitoring information stored in the memory 220 from the first occupant monitoring device 10a and the second occupant monitoring device 10b. Specifically, as shown in FIG. 15, the occupant status detection unit 213D detects whether the vehicle occupant is drowsy, fatigued, looking away, stressed, etc., based on occupant monitoring information (information regarding the occupant's "facial expression" and "biometric information") obtained when the allocation process I is executed, such as "direction of gaze, direction of face, eye opening, number of blinks, eye movement" and "heart rate, heart rate variability, respiratory rate, brain waves."

[0103] In addition, the occupant status detection unit 213D detects the resting state, sleeping state, sleep quality, etc. of the occupant in the vehicle based on occupant monitoring information (information regarding the occupant's "behavior" and "biological information") obtained when the allocation process J is executed, which is "posture, body movement, sleeping time, number of turns over, rest time" and "heart rate, heart rate variability, respiratory rate, brain waves."

[0104] <Actions and Effects> As described above, in the occupant state detection system 1D according to this embodiment, the second occupant monitoring device 10b includes a millimeter wave radar, so that detailed biological information of the occupant can be obtained. Therefore, by using detailed biological information of the occupant, the physical condition of the occupant can be detected with high accuracy. Furthermore, in the occupant state detection system 1D according to this embodiment, the second occupant monitoring device 10b is equipped with a millimeter wave radar, so that it is possible to obtain monitoring information of an occupant who is in a position that cannot be captured by a camera. Therefore, occupant monitoring information that cannot be obtained by a camera can be interpolated using occupant monitoring information obtained from the millimeter wave radar, making it possible to detect the physical condition of the occupants with high accuracy.

[0105] <Modification> For example, in the above embodiment, an example was given of providing a monitoring allocation unit 212 and an occupant state detection unit 213 within the occupant state detection device 200, but a configuration may also be adopted in which information from the occupant monitoring device 10 is transferred to a server connected to the vehicle, and the processing of the monitoring allocation unit 212 and the processing of the occupant state detection unit 213 are executed in the server. In this manner, a large amount of information can be quickly processed, and the physical condition of the occupants can be detected accurately at all times while the vehicle is stopped.

[0106] The occupant state detection systems 1, 1A, 1B, 1C, and 1D of the present invention can be realized by recording the processing of the processors 210, 210A, 210B, 210C, and 210D in a recording medium readable by a computer system, and having the processors 210, 210A, 210B, 210C, and 210D read and execute the programs recorded in the recording medium. The computer system here includes hardware such as an OS and peripheral devices.

[0107] Furthermore, "computer system" also includes the homepage providing environment (or display environment) if a WWW (World Wide Web) system is used. The above program may be transmitted from a computer system in which the program is stored in a storage device or the like to another computer system via a transmission medium, or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.

[0108] The program may be for implementing some of the above-mentioned functions, or may be a so-called differential file (differential program) that can implement the above-mentioned functions in combination with a program already recorded in the computer system.

[0109] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and designs that do not deviate from the gist of the present invention are also included. [Explanation of symbols]

[0110] 1; Occupant status detection system 1A: Occupant status detection system 1B: Occupant status detection system 1C: Occupant status detection system 1D; Occupant status detection system 10. Occupant monitoring device 10a; First occupant monitoring device 10b; Secondary occupant monitoring device 100;Discrimination device 200: Occupant status detection device 200A: Occupant status detection device 200B: Occupant status detection device 200C: Occupant status detection device 200D; Occupant status detection device 210; Processor 210A; Processor 210B Processor 210C; Processor 210D; Processor 211; Information receiving section 211A; Information receiving section 211B; Information receiving section 211C; Information receiving section 211D; Information receiving section 212; Surveillance Allocation Division 212A; Supervisory Allocation Division 212B; Surveillance Allocation Division 212C; Surveillance Allocation Division 212D; Surveillance Allocation Division 213: Occupant status detection unit 213A: Occupant status detection unit 213B: Occupant status detection unit 213C: Occupant status detection unit 213D: Occupant status detection unit 220;Memory

Claims

[Claim 1] An occupant monitoring device that monitors the physical condition of an occupant of a vehicle; an occupant condition detection device that detects a physical condition of an occupant of the vehicle based on occupant monitoring information from the occupant monitoring device; A discrimination device that discriminates between a traveling state and a stopped state of the vehicle; Equipped with The occupant state detection device includes one or more processors and one or more memories communicatively connected to the one or more processors, the discrimination device acquires ignition information, and when an ignition is in an OFF state, transmits discrimination information indicating that the vehicle is in the stopped state to the one or more processors, and when an ignition is in an ON state, transmits the discrimination information indicating that the vehicle is in the running state to the one or more processors; The occupant monitoring device includes a camera that captures an image of an occupant of the vehicle, and transmits the occupant monitoring information obtained from image information capturing a face or a whole body of the occupant to the one or more processors; The one or more processors receive the occupant monitoring information and the discrimination information from the discrimination device, store the received occupant monitoring information and the discrimination information in the one or more memories, and when the discrimination information is the stopped state, assign the monitoring area monitored by the occupant monitoring device to the entire body of the occupant to detect at least one of the occupant's resting state or sleep quality, and when the discrimination information is the driving state, assign the monitoring area monitored by the occupant monitoring device to the occupant's face to detect at least one of the occupant's sleeping state or fatigue state.

Images