Occupant status detection system

Abstract

To provide an occupant state detection system that can always accurately detect the physical states of occupants, by an optimal monitoring method in a running a state and a stopped state of a vehicle.SOLUTION: An occupant state detection system includes: an occupant monitoring device 10 that monitors a physical state of an occupant; a discrimination device 100 that discriminates between a running state and a stopped state of a vehicle; and an occupant state detection device 200 that detects the state of the occupant of the vehicle based on occupant monitoring information from the occupant monitoring device 10. The occupant state detection device 200 receives the occupant monitoring information, and discrimination information from the discrimination device 100, collates, based on the discrimination information, the received occupant monitoring information with comparison information at the time of the running state or comparison information at the time of the stopped state corresponding to the occupant monitoring information, and detects the state of the occupant of the vehicle.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an occupant state detection system. [Background technology]

[0002] In recent years, so-called driver monitoring systems that monitor the driver's state to support safe driving of a vehicle have been developed. Such driver monitoring systems, for example, detect where the driver is looking and issue a warning to the driver if the driver is not aware of the current situation.

[0003] For example, as an example of this type of device, an occupant monitoring device that monitors occupants including the driver has been disclosed that can reduce the number of imaging devices used even if the number of occupants to be monitored increases (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-202726 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the technology described in Patent Document 1 monitors the occupants, including the driver, while the vehicle is in motion, but has the problem that it does not monitor the occupants, including the driver, of the vehicle when the vehicle is stopped, or determine the physical condition of the occupants. Furthermore, since the objects monitored by the occupant monitoring device differ significantly between when the vehicle is moving and when it is stopped, there was also the problem that even if the same occupant monitoring control was applied when the vehicle was stopped as when it was moving, the occupant's status could not be detected correctly.

[0006] Therefore, the present invention has been made in consideration of the above-mentioned problems, and aims to provide an occupant condition detection system that optimizes the monitoring method when the vehicle is in a moving state and the monitoring method when the vehicle is stopped, and can always accurately detect the physical condition of the occupants. [Means for solving the problem]

[0007] Aspect 1: One or more embodiments of the present invention monitor the physical condition of a vehicle occupant. and obtains, from the body of the occupant via a predetermined device, first monitoring information including at least one of gaze direction, face direction, blink frequency, eye opening, eye movement, posture, and body movement, and second monitoring information including at least one of blood flow, pulse rate, respiratory rate, sleep time, sleep state, number of turns in sleep, and rest time. an occupant monitoring device; a discrimination device for discriminating between a running state and a stopped state of the vehicle; and The first monitoring information and the second monitoring information are included. an occupant status detection device that detects the status of an occupant of the vehicle based on occupant monitoring information, the occupant status detection device including one or more processors and one or more memories communicably connected to the one or more processors, the one or more memories storing collation data for detecting the status of an occupant of the vehicle in the traveling state; comparison information during a driving state including an index indicating a relationship between the first monitoring information and a behavior of the occupant including at least one of inattentiveness, drowsiness, fatigue, and abnormal posture; Verification data for detecting the state of an occupant of the vehicle in the stopped state comparison information during a stopped state including an index indicating a relationship between the second monitoring information and at least a physical abnormality of the occupant; and wherein the one or more processors: The monitoring target is switched depending on the running state of the vehicle, and when it is detected that the vehicle is running, the monitoring target is switched to the behavior of the occupant, and the first monitoring information is and the comparison information during the running state. When it is detected that the vehicle is stopped, the monitoring target is switched to the body of the occupant, and the second monitoring information and The comparison information during a stopped state; and detecting an abnormality in the body of the occupant by comparing the We propose an occupant status detection system. [Effects of the Invention]

[0013] According to one or more embodiments of the present invention, the monitoring method when the vehicle is moving and the monitoring method when the vehicle is stopped are optimized, which has the effect of always being able to accurately detect the physical condition of the occupants. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a diagram showing an installation state of an occupant state detection device in an occupant state detection system according to a first embodiment of the present invention. [Figure 2] 1 is a diagram showing a configuration of an occupant state detection system according to a first embodiment of the present invention. [Figure 3]1 is a diagram showing a configuration of an occupant state detection device in an occupant state detection system according to a first embodiment of the present invention. [Figure 4] 1 is a process flow diagram of an occupant state detection system according to a first embodiment of the present invention. [Figure 5] FIG. 4 is a diagram showing a configuration of an occupant state detection device in an occupant state detection system according to a second embodiment of the present invention. [Figure 6] FIG. 10 is a diagram showing the relationship between monitoring allocation processing and occupant monitoring information in an occupant state detection system according to a second embodiment of the present invention. [Figure 7] FIG. 6 is a process flow diagram of an occupant state detection system according to a second embodiment of the present invention. [Figure 8] FIG. 4 is a diagram showing a configuration of an occupant state detection device in an occupant state detection system according to a second embodiment of the present invention. [Figure 9] FIG. 10 is a diagram showing a configuration of an occupant state detection device in an occupant state detection system according to a third embodiment of the present invention. [Figure 10] FIG. 10 is a diagram showing the relationship between monitoring allocation processing and occupant monitoring information in an occupant state detection system according to a third embodiment of the present invention. [Figure 11] FIG. 10 is a process flow diagram of an occupant state detection system according to a third embodiment of the present invention. [Figure 12] FIG. 10 is a diagram showing a configuration of an occupant state detection device in an occupant state detection system according to a fourth embodiment of the present invention. [Figure 13] FIG. 10 is a diagram showing the relationship between the monitoring allocation process and the occupant monitoring information in the occupant state detecting system according to the fourth embodiment of the present invention. [Figure 14] FIG. 10 is a process flow diagram of an occupant state detection system according to a fourth embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, an embodiment of the present invention will be described with reference to FIGS. First Embodiment An occupant state detection system 1 according to this embodiment will be described with reference to FIGS. 1 to 4. FIG.

[0016] The occupant status detection system 1 of this embodiment is composed of an occupant monitoring device 10, a discrimination device 100, and an occupant status detection device 200. As shown in Figure 1, the occupant status detection device 200 is provided, for example, in the center portion in front of the driver's seat inside the vehicle, and receives occupant monitoring information obtained by the occupant monitoring device 10 described below and discrimination information from the discrimination device 100. Based on the discrimination information, the received occupant monitoring information is compared with the comparison information during driving or stopped state corresponding to the occupant monitoring information, thereby detecting the status of the occupants of the vehicle. The detailed configuration will be described separately.

[0017] <Configuration of occupant state detection system 1> As shown in FIG. 2, the occupant state detecting system 1 according to this embodiment includes an occupant monitoring device 10, a discrimination device 100, and an occupant state detecting device 200.

[0018] The occupant monitoring device 10 has an imaging element such as a CCD (Charge Coupled Device) or a CIS (CMOS Image Sensor) built in, and monitors the physical condition of the vehicle occupants using a camera that captures images (including moving images and still images) of the occupants inside the vehicle captured by the imaging element. The camera may be a dedicated camera, but if an occupant recognition device is provided, it is preferable to use the camera within the occupant recognition device. Also, it is preferable that the camera is configured with both an optical camera and a near-infrared camera. In the occupant state detection system 1 according to this embodiment, the occupant monitoring device 10 is suitable for monitoring information such as the facial expressions, behavior, and biological information of the occupant that can be observed from the outside.

[0019] Examples of occupant monitoring information that can be mainly obtained from the occupant monitoring device 10 include "behavior" including gaze direction, facial direction, number of blinks, eye opening, eye movement, posture, body movement, etc., and "biological information" including blood flow, pulse rate, respiratory rate, sleep time, sleep state, number of turns, rest time, etc. Here, the degree of eye opening can be used as an indicator for detecting drowsiness, the number of blinks can be used as an indicator for detecting the degree of eye strain or lack of sleep, and the degree of eye opening can be used as an indicator for detecting the sleeping state of the occupant, etc.

[0020] The discrimination device 100 discriminates between a running state and a stopped state of the vehicle. The discrimination device 100 acquires ignition information, for example, from a CPU (Central Processing Unit) of the vehicle (not shown), and determines that the vehicle is in a running state when the ignition is in an ON state, and determines that the vehicle is in a stopped state when the ignition is in an OFF state. The discrimination device 100 may further monitor the vehicle speed pulse, the operating state of the parking brake, and the like, and determine whether the vehicle is running or stopped in combination with the ON / OFF state of the ignition.

[0021] The occupant state detection device 200 receives information indicating whether the vehicle is in a running state or a stopped state from the discrimination device 100. Then, based on the discrimination information from the discrimination device 100, if the vehicle is in a running state, the occupant state detection device 200 compares the occupant monitoring information received from the occupant monitoring device 10 with the running state comparison information, and if the vehicle is in a stopped state, compares the occupant monitoring information received from the occupant monitoring device 10 with the stopped state comparison information to detect the state of the vehicle occupants.

[0022] <Configuration of occupant state detection device 200> As shown in FIG. 3, the occupant state detection device 200 according to this embodiment includes a processor 210 and a memory 220.

[0023] The processor 210 controls the entire occupant state detection device 200 in accordance with a control program stored in a memory 220, which will be described later. The processor 210 then receives information indicating that the vehicle is in a moving or stopped state from the discrimination device 100. The processor 210 then receives occupant monitoring information from the occupant monitoring device 10, and compares the received occupant monitoring information with moving state comparison information or stopped state comparison information stored in the memory 220 (described later), to detect the physical state of the occupant.

[0024] The memory 220 includes a read only memory (ROM), a random access memory (RAM), etc. The ROM stores the above-mentioned control programs, etc., and the RAM stores various data, etc. In this embodiment, for example, the occupant monitoring information received from the occupant monitoring device 10 and the like are stored in the memory 220.

[0025] The comparison information during driving is collation data that is referenced by the processor 210 to detect the physical state of the occupant when the vehicle is driving. The comparison information during a stopped state is collation data that is referenced by the processor 210 to detect the physical state of the occupant when the vehicle is stopped. Specifically, the comparison information when the vehicle is in a traveling state is information relating to the occupant's behavior and biometric information under normal circumstances when the vehicle is in a traveling state, and the comparison information when the vehicle is stopped is information relating to the occupant's behavior and biometric information under normal circumstances when the vehicle is stopped, and both are stored in memory 220. The processor 210 detects abnormal behavior and physical conditions of the occupants when the vehicle is moving or stopped, such as biometric information, by comparing the occupant monitoring information received from the occupant monitoring device 10 with the comparison information when the vehicle is moving or stopped.

[0026] <Processing of occupant state detection system 1> The processing of the occupant state detection system 1 according to this embodiment will be described with reference to FIG.

[0027] The processor 210 receives the occupant monitoring information from the occupant monitoring device 10 and stores it in the memory 220 (step S10).

[0028] Processor 210 determines whether or not information indicating the running state of a vehicle with its ignition turned on has been received from discriminator 100 (step S20).

[0029] If processor 210 determines that it has not received information indicating the running state of a vehicle with the ignition turned on ("NO" in step S20), it determines that the vehicle is stopped and transitions the processing to step S30.

[0030] The processor 210 compares the received occupant monitoring information with the vehicle-stopped state comparison information stored in the memory 220, and detects the occupant state (step S30). Specifically, the processor 210 compares occupant monitoring information ("biological information") such as "blood flow, pulse rate, respiratory rate" with the comparison information during a stopped state, and detects any abnormalities in the occupant's body.

[0031] On the other hand, if processor 210 determines that it has received information indicating the traveling state of a vehicle with the ignition turned on ("YES" in step S20), it shifts the processing to step S40. The processor 210 compares the passenger monitoring information received from the passenger monitoring device 10 with the comparison information during driving stored in the memory 220, and detects the passenger state (step S40). Specifically, processor 210 compares occupant monitoring information ("behavior") such as "direction of gaze, direction of face, number of blinks, eye opening, eye movement, posture, and body movement" with comparison information during driving conditions to detect abnormal conditions such as inattentiveness, drowsiness, fatigue, and posture.

[0032] Then, the processor 210 determines whether or not to continue the occupant state detection process (step S50). If processor 210 determines to continue the occupant state detection process ("YES" in step S50), the process returns to step S10, and the occupant state detection process continues. On the other hand, if the processor 210 determines not to continue the occupant state detection process ("NO" in step S50), the occupant state detection process ends.

[0033] <Actions and Effects> As described above, the occupant state detection system 1 according to this embodiment includes the occupant monitoring device 10 that monitors the physical state of the vehicle occupant, the discrimination device 100 that discriminates between the running state and the stopped state of the vehicle, and the occupant state detection device 200 that detects the state of the vehicle occupant based on the occupant monitoring information from the occupant monitoring device 10. The occupant state detection device 200 includes the processor 210 and the memory 220 that is communicably connected to the processor 210. The memory 220 stores collation data for detecting the state of the occupant of the vehicle in the running state. The processor 210 receives the occupant monitoring information and the discrimination information from the discrimination device 100, stores the received occupant monitoring information in the memory 220, and based on the discrimination information, when the vehicle is in a running state, compares the occupant monitoring information with the running state comparison information, and when the vehicle is in a stopped state, compares the occupant monitoring information with the stopped state comparison information to detect the state of the occupants of the vehicle. In other words, in the occupant condition detection system 1 according to this embodiment, based on the discrimination information that determines whether the vehicle is in a moving state or a stopped state, the occupant monitoring information from the occupant monitoring device 10 is compared with the comparison information when the vehicle is in a moving state or a stopped state, and the physical condition of the occupant is detected. Therefore, in detecting the physical condition of the occupant, the monitoring method is optimized when the vehicle is stopped or running, so that the physical condition of the occupant can be detected accurately at all times.

[0034] In addition, the discrimination device 100 in the occupant state detection system 1 of this embodiment acquires ignition information, and when the ignition is in the OFF state, transmits discrimination information to the processor 210 that the vehicle is in a stopped state, and when the ignition is in the ON state, transmits discrimination information to the processor 210 that the vehicle is in a running state. That is, the discrimination device 100 discriminates between a running state and a stopped state of the vehicle based on whether the ignition is on or off. Therefore, the occupant state detection system 1 can uniquely determine whether the vehicle is in a traveling state or a stopped state without using a special sensor or the like.

[0035] In addition, the occupant monitoring device 10 in the occupant state detection system 1 of this embodiment includes a camera that captures images of the vehicle occupants, acquires occupant monitoring information including the behavior and biometric information of the vehicle occupants, and transmits it to the processor 210. In other words, the occupant state detection system 1 causes the occupant monitoring device 10 to acquire occupant monitoring information including the behavior and biometric information of the vehicle occupants as information for detecting the physical state of the occupants, while taking into consideration the characteristics of the built-in camera. Therefore, the occupant state detection system 1 can obtain highly accurate information about the behavior, biological information, etc. of the vehicle occupant, thereby improving the accuracy of detecting the physical state of the vehicle occupant.

[0036] Furthermore, if the camera is configured with both an optical camera and a near-infrared camera, images of the occupant can be acquired day or night. Therefore, regardless of the time of day, the physical condition of the occupants can be always accurately grasped when the vehicle is running or stopped.

[0037] <Second embodiment> An occupant state detection system 1A according to this embodiment will be described with reference to FIGS. Note that components with the same reference numerals as those in the first embodiment have the same functions, and therefore detailed descriptions thereof will be omitted. <Configuration of Occupant State Detection System 1A> As shown in FIG. 5, an occupant state detection system 1A according to this embodiment includes an occupant monitoring device 10, a discrimination device 100, and an occupant state detection device 200A.

[0038] <Configuration of occupant state detection device 200A> As shown in FIG. 6, the occupant state detection device 200A according to this embodiment includes a processor 210A and a memory 220.

[0039] The processor 210A controls the entire occupant state detection device 200A in accordance with a control program stored in a memory 220, which will be described later. In this embodiment, the processor 210A particularly executes the functions of an information receiving unit 211A, a monitoring allocation unit 212A, an occupant state detection unit 213A, and the like, which will be described later.

[0040] <Configuration of Processor 210A> As shown in FIG. 6, the processor 210A according to this embodiment includes an information receiving unit 211A, a monitoring allocation unit 212A, and an occupant state detection unit 213A.

[0041] The information receiving unit 211A receives the passenger monitoring information from the passenger monitoring device 10 and stores the received passenger monitoring information in the memory 220.

[0042] The monitoring allocation unit 212A receives the discrimination information from the discrimination device 100 and allocates a monitoring target to be monitored by the occupant monitoring device 10. Specifically, as shown in FIG. 7, when the monitoring allocation unit 212A receives discrimination information from the discrimination device 100 indicating that the vehicle is in a traveling state, it allocates the monitoring target of the occupant monitoring device 10 to, for example, "occupant behavior" (allocation process A). In other words, when the vehicle is in a traveling state, in order to detect abnormal postures of occupants, the monitoring target of the occupant monitoring device 10 is assigned to, for example, "occupant behavior," and occupant monitoring information such as, for example, "direction of gaze, direction of face, number of blinks, eye opening, eye movement, posture, body movement" is transmitted to the information receiving unit 211A.

[0043] Furthermore, when the monitoring allocation unit 212A receives discrimination information from the discrimination device 100 that the vehicle is in a stopped state, it allocates the monitoring target of the occupant monitoring device 10 to, for example, "biological information of the occupant" (allocation process B). That is, when the vehicle is stopped, the occupant may lie down to sleep, rest, or the like, and the occupant monitoring device 10 may not be able to acquire facial information. Therefore, the monitoring target of the occupant monitoring device 10 is assigned to, for example, "occupant biological information," and occupant monitoring information such as "blood flow, pulse rate, respiratory rate," etc. is transmitted to the information receiving unit 211A. Moreover, the monitoring allocation unit 212A transmits the discrimination information received from the discrimination device 100 to the occupant state detection unit 213A.

[0044] Based on the discrimination information received from the monitoring allocation unit 212A, the occupant condition detection unit 213A compares the occupant monitoring information from the occupant monitoring device 10 stored in the memory 220 with the comparison information during driving or the comparison information during stopped state, and detects the physical condition of the occupant. Specifically, when the occupant state detection unit 213A receives determination information indicating a driving state from the monitoring allocation unit 212A, it compares occupant monitoring information such as "direction of gaze, direction of face, number of blinks, eye opening, eye movement, posture, body movement" that indicates the above-mentioned "occupant behavior" with the driving state comparison information corresponding to this occupant monitoring information, and detects abnormal states such as inattentiveness, drowsiness, fatigue, posture, etc.

[0045] In addition, when the occupant status detection unit 213A receives information from the monitoring allocation unit 212A indicating that the vehicle is stopped, it compares occupant monitoring information such as "blood flow, pulse rate, respiratory rate" which indicates the above-mentioned "occupant's biological information" with the comparison information corresponding to this occupant monitoring information when the vehicle is stopped, and detects any abnormalities in the occupant's body.

[0046] <Processing of occupant state detection system 1A> The processing of the occupant state detection system 1A according to this embodiment will be described with reference to FIG.

[0047] The monitoring allocation unit 212A of the processor 210A determines whether or not information indicating the running state of a vehicle with the ignition turned on has been received from the discrimination device 100 (step S110).

[0048] When the monitoring allocation unit 212A determines that it has received information indicating the traveling state of a vehicle with the ignition turned on ("YES" in step S110), it executes allocation process A (step S120). In the allocation process A (step S120), the monitoring allocation unit 212A allocates the monitoring target of the occupant monitoring device 10 to, for example, "occupant behavior," transmits a signal to that effect to the occupant monitoring device 10, and proceeds to step S140.

[0049] On the other hand, if the monitoring allocation unit 212A determines that it has not received information indicating the running state of a vehicle with the ignition turned on ("NO" in step S110), it determines that the vehicle is stopped and executes allocation process B (step S130). In allocation process B (step S130), the monitoring allocation unit 212A allocates the monitoring target of the occupant monitoring device 10 to, for example, "occupant biological information," transmits a signal to that effect to the occupant monitoring device 10, and proceeds to step S140.

[0050] Here, after the execution of the allocation process A (step S120) or the allocation process B (step S130) in the monitoring allocation unit 212A, the processor 210A permits the already activated occupant monitoring device 10 to output information to the information receiving unit 211A. The processor 210A may allow the information receiving unit 211A to receive the occupant monitoring information from the occupant monitoring device 10 after the monitoring allocation unit 212A executes the allocation process A (step S120) or the allocation process B (step S130). Alternatively, the processor 210A may allow the information receiving unit 211A to write the occupant monitoring information from the occupant monitoring device 10 to the memory 220 after executing the allocation process A (step S120) or the allocation process B (step S130) in the monitoring allocation unit 212A.

[0051] The information receiving unit 211A receives occupant monitoring information from the occupant monitoring device 10 to which a monitoring target has been assigned by the monitoring assignment unit 212A. Then, the information receiving unit 211A stores the occupant monitoring information received from the occupant monitoring device 10 in the memory 220 (step S140).

[0052] The occupant condition detection unit 213A compares the occupant monitoring information from the occupant monitoring device 10 stored in the memory 220 with the comparison information during driving or the comparison information during stopped state corresponding to the occupant monitoring information stored in the memory 220, and detects the physical condition of the occupant, etc. (step S150). Specifically, when the occupant state detection unit 213A receives, for example, determination information indicating a driving state from the monitoring allocation unit 212A, it compares occupant monitoring information such as "direction of gaze, direction of face, number of blinks, eye opening, eye movement, posture, body movement" that indicates "occupant behavior" with the driving state comparison information corresponding to this occupant monitoring information, and detects abnormal states such as inattentiveness, drowsiness, fatigue, posture, etc. In addition, when the occupant status detection unit 213A receives information from the monitoring allocation unit 212A indicating that the vehicle is stopped, it compares occupant monitoring information such as "blood flow, pulse rate, respiratory rate" indicating "occupant's biological information" with the comparison information for the stopped state corresponding to this occupant monitoring information, and detects any abnormalities in the occupant's body.

[0053] Then, the processor 210A determines whether or not to continue the occupant state detection process (step S160). If processor 210A determines to continue the occupant state detection process ("YES" in step S160), the process returns to step S110, and the occupant state detection process continues. On the other hand, if the processor 210A determines not to continue the occupant state detection process ("NO" in step S160), the occupant state detection process ends.

[0054] <Actions and Effects> As described above, the processor 210A in the occupant state detection system 1A of this embodiment assigns the monitoring target monitored by the occupant monitoring device 10 to the behavior of the vehicle occupant when the discrimination information indicates a driving state, and assigns the monitoring target to the biometric information of the vehicle occupant when the discrimination information indicates a stopped state. In other words, based on the discrimination information from the discrimination device 100, when the vehicle is in a moving state, the processor 210A assigns the monitoring target to the behavior of the vehicle occupant, and when the vehicle is in a stopped state, assigns the monitoring target to the biometric information of the vehicle occupant. Therefore, in detecting the physical condition of the occupant, the monitoring method is optimized when the vehicle is stopped or running, so that the physical condition of the occupant can be detected accurately at all times. Furthermore, in the occupant state detection system 1A of this embodiment, when the vehicle is stopped and the occupant sleeps, takes a rest, etc., the occupant monitoring device 10 is unable to monitor the occupant's face. Therefore, when the vehicle is stopped, the occupant monitoring device 10 assigns the monitoring target to be monitored to "occupant's biological information" rather than "occupant's behavior." Therefore, the occupant state detecting system 1A can accurately detect the physical state of the occupant even when the vehicle is stopped. Furthermore, when the vehicle is in a traveling state, the occupant state detection system 1A according to this embodiment assigns the monitoring target to be monitored by the occupant monitoring device 10 to "occupant behavior." In other words, when the vehicle is in a driving state, the occupant state detection system 1A compares occupant monitoring information such as the occupant's "direction of gaze, direction of face, number of blinks, eye opening, eye movement, posture, body movement" with the driving state comparison information corresponding to this occupant monitoring information, and can immediately detect dangerous conditions while driving such as the occupant's inattentiveness, drowsiness, fatigue, posture, etc.

[0055] <Third embodiment> An occupant state detection system 1B according to this embodiment will be described with reference to FIGS. Note that components with the same reference numerals as those in the first and second embodiments have similar functions, and therefore detailed descriptions thereof will be omitted.

[0056] <Configuration of occupant state detection system 1B>

[0057] As shown in FIG. 9, an occupant state detecting system 1B according to this embodiment includes an occupant monitoring device 10, a discrimination device 100, and an occupant state detecting device 200B.

[0058] When the occupant state detection device 200B receives information from the discrimination device 100 that the vehicle is in a running state or a stopped state, it assigns a monitoring target to be monitored by the occupant monitoring device 10. The occupant condition detection device 200B receives occupant monitoring information from the occupant monitoring device 10 to which the monitoring target is assigned, and compares the occupant monitoring information with the traveling state comparison information or the stopped state comparison information corresponding to the occupant monitoring information to detect the physical condition of the occupant.

[0059] <Configuration of occupant state detection device 200B> As shown in FIG. 9, the occupant state detection device 200B according to this embodiment includes a processor 210B and a memory 220.

[0060] The processor 210B controls the entire occupant state detection device 200B in accordance with a control program stored in the memory 220. In this embodiment, the processor 210B particularly executes the functions of an information receiving unit 211B, a monitoring allocation unit 212B, an occupant state detection unit 213B, and the like, which will be described later.

[0061] <Configuration of Processor 210B> As shown in FIG. 9, a processor 210B according to this embodiment includes an information receiving unit 211B, a monitoring allocation unit 212B, and an occupant state detection unit 213B.

[0062] The information receiving unit 211B receives passenger monitoring information from the passenger monitoring device 10 to which the monitoring target has been assigned by the monitoring assignment unit 212B, and stores the received passenger monitoring information in the memory 220.

[0063] The monitoring allocation unit 212B receives the discrimination information from the discrimination device 100 and allocates a monitoring target to be monitored by the occupant monitoring device 10. Specifically, as shown in FIG. 10, when the monitoring allocation unit 212B receives discrimination information from the discrimination device 100 indicating that the vehicle is in a moving state, it allocates the monitoring target of the occupant monitoring device 10 to the "occupant in the driver's seat" (allocation process C). That is, when the vehicle is in a traveling state, the monitoring target of the occupant monitoring device 10 is assigned to the occupant in the driver's seat in order to detect abnormal posture, drowsiness, inattentiveness, etc., of the occupant in the driver's seat. On the other hand, when the monitoring allocation unit 212B receives discrimination information from the discrimination device 100 indicating that the vehicle is stopped, it allocates the monitoring target of the occupant monitoring device 10 to "all occupants in the vehicle interior" (allocation process D). That is, when the vehicle is stopped, the occupant monitoring device 10 assigns all occupants in the vehicle interior to be monitored in order to monitor the status of all occupants in the vehicle interior. Moreover, the monitoring allocation unit 212B transmits the discrimination information received from the discrimination device 100 to the occupant state detection unit 213B.

[0064] Based on the discrimination information received from the monitoring allocation unit 212B, the occupant condition detection unit 213B compares the occupant monitoring information from the occupant monitoring device 10 stored in the memory 220 with the comparison information during driving or stopped state corresponding to this occupant monitoring information, and detects the physical condition of the occupant.

[0065] Specifically, when the occupant state detection unit 213B receives determination information indicating that the vehicle is in a driving state from the monitoring allocation unit 212B, it compares the occupant monitoring information, such as the "direction of gaze, direction of face, eye opening, number of blinks, eye movement, posture, and body movement" of the occupant in the driver's seat, with the driving state comparison information corresponding to this occupant monitoring information, and detects whether the occupant in the driver's seat is looking away, drowsy, fatigued, etc.

[0066] In addition, when the occupant state detection unit 213B receives determination information indicating that the vehicle is stopped from the monitoring allocation unit 212B, it compares occupant monitoring information such as the ``posture, body movements'' of occupants throughout the interior with the stopped state comparison information corresponding to this occupant monitoring information, as described above, and detects the state of all occupants in the vehicle, such as sleeping, posture, mobile phone use, television viewing, smoking, eating, drinking, talking, etc.

[0067] <Processing of occupant state detection system 1B> The processing of the occupant state detection system 1B according to this embodiment will be described with reference to FIG.

[0068] The monitoring allocation unit 212B of the processor 210B determines whether or not information indicating the running state of a vehicle with the ignition turned on has been received from the discrimination device 100 (step S210).

[0069] At this time, if the monitoring allocation unit 212B determines that it has received information indicating the traveling state of a vehicle with the ignition turned on ("YES" in step S210), it executes allocation processing C (step S220). In the allocation process C (step S220), the monitoring allocation unit 212B allocates the monitoring target of the occupant monitoring device 10 to the "occupant in the driver's seat" and transmits a signal to that effect to the occupant monitoring device 10.

[0070] On the other hand, if the monitoring allocation unit 212B determines that it has not received information indicating the running state of a vehicle with the ignition turned on ("NO" in step S210), it determines that the vehicle is stopped and executes allocation process D (step S230). In the allocation process D (step S230), the monitoring allocation unit 212B allocates the monitoring target of the occupant monitoring device 10 to "all occupants in the vehicle interior," and transmits a signal to that effect to the occupant monitoring device 10.

[0071] When the monitoring allocation unit 212B executes allocation process C (step S220) or allocation process D (step S230), the information receiving unit 211B receives occupant monitoring information from the occupant monitoring device 10 to which the monitoring target has been allocated by the monitoring allocation unit 212B, and stores the received occupant monitoring information in the memory 220 (step S240).

[0072] The occupant condition detection unit 213B compares the occupant monitoring information from the occupant monitoring device 10 stored in the memory 220 with the comparison information during driving or stopped state corresponding to this occupant monitoring information, and detects the physical condition of the occupant, etc. (step S250). In other words, when the occupant state detection unit 213B receives determination information indicating that the vehicle is in a driving state from the monitoring allocation unit 212B, it compares occupant monitoring information such as the occupant in the driver's seat's "direction of gaze, direction of face, eye opening, number of blinks, eye movement, posture, body movement" with the driving state comparison information corresponding to this occupant monitoring information, and detects the state of sleep, fatigue, absentminded driving, emotions, stress, etc. of the occupant in the driver's seat. In addition, when the occupant state detection unit 213B receives determination information from the monitoring allocation unit 212B indicating that the vehicle is in a stopped state, it compares occupant monitoring information such as the ``posture, body movements'' of occupants throughout the interior with the stopped state comparison information corresponding to this occupant monitoring information, and detects the state of all occupants in the vehicle, such as dozing, posture, mobile phone use, television viewing, smoking, eating, drinking, and conversation.

[0073] Then, processor 210B determines whether or not to continue the occupant state detection process (step S260). If processor 210B determines to continue the occupant state detection process ("YES" in step S260), the process returns to step S210, and the occupant state detection process continues. On the other hand, if the processor 210B determines not to continue the occupant state detection process ("NO" in step S260), the occupant state detection process ends.

[0074] <Actions and Effects> As described above, the processor 210B in the occupant state detection system 1B of this embodiment assigns the monitoring target to be monitored by the occupant monitoring device 10 to the occupant in the driver's seat of the vehicle when the discrimination information indicates a driving state, and assigns the monitoring target to the occupants throughout the interior of the vehicle when the discrimination information indicates a stopped state. That is, when the vehicle is stopped, the processor 210B in the occupant state detection system 1B assigns monitoring targets to all occupants in the vehicle interior because the driver may move around inside the vehicle interior. Therefore, when the vehicle is stopped, the physical states of all occupants in the vehicle can be detected at all times. In addition, when the vehicle is in a driving state, the processor 210B in the occupant state detection system 1B of this embodiment acquires occupant monitoring information such as "direction of gaze, direction of face, eye opening, number of blinks, eye movement, posture, body movement" in order to detect an abnormal state of the driver, and compares the acquired occupant monitoring information with the comparison information during the driving state. Therefore, when the vehicle is in a traveling state, it is possible to immediately detect whether the driver is looking away, dozing, tired, or driving aimlessly.

[0075] <Fourth embodiment> An occupant state detection system 1C according to this embodiment will be described with reference to FIGS.

[0076] <Configuration of occupant state detection system 1C>

[0077] As shown in FIG. 12, an occupant state detecting system 1C according to this embodiment includes an occupant monitoring device 10, a discrimination device 100, and an occupant state detecting device 200C. Note that components with the same reference numerals as those in the first to third embodiments have the same functions, and therefore detailed descriptions thereof will be omitted.

[0078] When the occupant state detection device 200C receives information from the discrimination device 100 that the vehicle is in a running state or a stopped state, the occupant state detection device 200C assigns a monitoring target of the occupant of the vehicle to be monitored by the occupant monitoring device 10. The occupant condition detection device 200C receives occupant monitoring information from the occupant monitoring device 10 to which the occupant monitoring target is assigned, and compares the occupant monitoring information with the traveling state comparison information or the stopped state comparison information corresponding to the occupant monitoring information to detect the physical condition of the occupant.

[0079] <Configuration of occupant state detection device 200C> As shown in FIG. 12, the occupant state detection device 200C according to this embodiment includes a processor 210C and a memory 220.

[0080] The processor 210C controls the entire occupant state detection device 200C in accordance with a control program stored in the memory 220. In this embodiment, the processor 210C particularly executes the functions of an information receiving unit 211C, a monitoring allocation unit 212C, an occupant state detection unit 213C, and the like, which will be described later.

[0081] <Configuration of Processor 210C> As shown in FIG. 12, a processor 210C according to this embodiment includes an information receiving unit 211C, a monitoring allocation unit 212C, and an occupant state detection unit 213C.

[0082] The information receiving unit 211C receives passenger monitoring information from the passenger monitoring device 10 to which the monitoring target has been assigned by the monitoring assignment unit 212C, and stores the received passenger monitoring information in the memory 220.

[0083] The monitoring allocation unit 212C receives the discrimination information from the discrimination device 100, and allocates the vehicle occupants to be monitored by the occupant monitoring device 10 based on the discrimination information. Specifically, for example, as shown in FIG. 13, when the monitoring allocation unit 212C receives discrimination information from the discrimination device 100 indicating that the vehicle is in a traveling state, it allocates the monitoring target of the occupant monitoring device 10 to "occupant behavior and biological information" (allocation process E). In other words, when the vehicle is in a driving state, the occupant monitoring device 10 assigns the monitoring target to "occupant behavior and biological information" in order to detect the occupant state (external state) that can be determined from the occupant's behavior such as drowsiness or distraction, and the occupant state (internal state) that cannot be determined from the occupant's behavior alone such as stress, emotions, or careless driving. Furthermore, when the monitoring allocation unit 212C receives information from the discrimination device 100 that the vehicle is in a stopped state, it allocates the monitoring target of the occupant monitoring device 10 to "occupant behavior" (allocation process F). In other words, when the vehicle is stopped, the monitoring target of the occupant monitoring device 10 is assigned to "occupant behavior" in order to detect the occupant's state (external state) such as drowsiness, distraction, mobile phone use, television viewing, smoking, eating, drinking, conversation, etc. Moreover, the monitoring allocation unit 212C transmits the discrimination information received from the discrimination device 100 to the occupant state detection unit 213C.

[0084] Based on the discrimination information received from the monitoring allocation unit 212C, the occupant condition detection unit 213C compares the occupant monitoring information from the occupant monitoring device 10 stored in the memory 220 with the comparison information during driving or stopped state corresponding to this occupant monitoring information, and detects the physical condition of the occupant. Specifically, when the occupant state detection unit 213C receives determination information indicating a driving state from the monitoring allocation unit 212C, it compares occupant monitoring information such as "direction of gaze, direction of face, eye opening, number of blinks, eye movement, posture, body movement, blood flow, pulse rate, and respiratory rate" with the driving state comparison information corresponding to this occupant monitoring information, and detects the occupant state such as drowsiness, absent-minded driving, emotions, stress, inattentiveness, dozing, using a mobile phone, watching television, smoking, eating and drinking, and talking. In addition, when the occupant status detection unit 213C receives information from the monitoring allocation unit 212C that the vehicle is stopped, it compares occupant monitoring information such as "direction of gaze, direction of face, eye opening, number of blinks, eye movement, posture, body movement" with the comparison information for the stopped state corresponding to this occupant monitoring information, and detects the occupant status such as whether the occupant is looking away, dozing, using a mobile phone, watching television, smoking, eating, drinking, or talking.

[0085] <Processing of occupant state detection system 1C> The processing of the occupant state detection system 1C according to this embodiment will be described with reference to FIG.

[0086] The monitoring allocation unit 212C of the processor 210C determines whether or not information indicating the running state of a vehicle with the ignition turned on has been received from the discrimination device 100 (step S310).

[0087] At this time, if the monitoring allocation unit 212C determines that it has received information indicating the driving state of a vehicle with the ignition turned on ("YES" in step S310), it proceeds to step S320 and executes allocation process E (step S320). As described above, in the allocation process E (step S320), the monitoring allocation unit 212C allocates the monitoring target of the occupant monitoring device 10 to "vehicle behavior and biological information" and transmits a signal to that effect to the occupant monitoring device 10.

[0088] On the other hand, if the monitoring allocation unit 212C of the processor 210C determines that it has not received information indicating the running state of a vehicle with the ignition turned on ("NO" in step S310), it determines that the vehicle is stopped, transitions to step S330, and executes allocation process F (step S330).

[0089] As described above, in the allocation process F (step S330), the monitoring allocation unit 212C allocates the monitoring target of the occupant monitoring device 10 to "occupant behavior" and transmits a signal to that effect to the occupant monitoring device 10.

[0090] When the monitoring allocation unit 212C executes allocation process E (step S320) or allocation process F (step S330), the information receiving unit 211C receives occupant monitoring information from the occupant monitoring device 10 to which the monitoring target has been assigned by the monitoring allocation unit 212C, and stores the received occupant monitoring information in the memory 220 (step S340).

[0091] The occupant condition detection unit 213C compares the occupant monitoring information from the occupant monitoring device 10 stored in the memory 220 with the comparison information during driving or stopped state corresponding to this occupant monitoring information, and detects the physical condition of the occupant, etc. (step S350). In other words, when the occupant state detection unit 213C receives determination information indicating that the vehicle is in a driving state from the monitoring allocation unit 212C, it compares occupant monitoring information such as "direction of gaze, direction of face, eye opening, number of blinks, eye movement, posture, body movement, blood flow, pulse rate, and respiratory rate" with the driving state comparison information, and detects the occupant's state, such as drowsiness, absent-minded driving, emotions, stress, inattentiveness, dozing, mobile phone use, television viewing, smoking, eating and drinking, and conversation. In addition, when the occupant state detection unit 213C receives determination information from the monitoring allocation unit 212C indicating that the vehicle is stopped, it compares occupant monitoring information such as "direction of gaze, direction of face, eye opening, number of blinks, eye movement, posture, body movement" with the comparison information for the stopped state, and detects the occupant's state, such as whether they are looking away, dozing, using a mobile phone, watching television, smoking, eating, drinking, or talking.

[0092] Then, processor 210C determines whether or not to continue the occupant state detection process (step S360). If processor 210C determines to continue the occupant state detection process ("YES" in step S360), the process returns to step S310, and the occupant state detection process continues. On the other hand, if the processor 210C determines not to continue the occupant state detection process ("NO" in step S360), the occupant state detection process ends.

[0093] <Actions and Effects> As described above, the processor 210C in the occupant state detection system 1C of this embodiment assigns the monitoring target monitored by the occupant monitoring device 10 to the behavior and biometric information of the vehicle occupant when the discrimination information indicates a driving state, and assigns the monitoring target to the behavior of the vehicle occupant when the discrimination information indicates a stopped state. That is, when the vehicle is in a traveling state, the occupant state detection system 1C monitors the behavior and biological information of the occupant since it is necessary to immediately detect a dangerous state during driving. Therefore, by monitoring the occupant's behavior and biological information, it is possible to detect the occupant's internal state, such as stress, drowsiness, emotions, distracted driving, fatigue, etc., which cannot be determined from the occupant's behavior (external state) alone. In addition, since the system can detect dangerous physical conditions of occupants while driving, it can provide information to occupants, such as warnings or encouraging them to take a break. Furthermore, the occupant condition detection system 1C of this embodiment detects the physical condition of the vehicle occupant based on the ``occupant behavior'' when the vehicle is stopped, since there is no need to immediately detect dangerous conditions such as occupant stress, drowsiness, emotions, careless driving, fatigue, etc. Therefore, in detecting the physical condition of the occupant, the monitoring method is optimized when the vehicle is stopped or running, so that the physical condition of the occupant can be detected accurately at all times.

[0094] <Modification> In the above-described occupant condition detection systems 1, 1A, 1B, and 1C, the physical condition of the occupant is monitored by an occupant monitoring device 10 equipped with a camera, but it is also possible to further provide an occupant monitoring device equipped with a millimeter-wave radar to monitor the physical condition of the occupant. Millimeter-wave radar can detect objects and measure the distance, horizontal angle, and relative speed of the detected object using short-wavelength radio waves, making it possible to obtain detailed "biometric information" such as heart rate, heart rate variability, respiratory rate, and brain waves. In other words, millimeter-wave radar is suitable for obtaining biometric information for monitoring the health status of occupants, which is difficult to determine from the occupant's external condition (behavior), and by monitoring the occupant using millimeter-wave radar, the occupant's physical condition can be grasped in more detail.

[0095] In addition, in the second to fourth embodiments, information from the occupant monitoring device 10 may be transferred to a server connected to the vehicle, and the server may execute the processing of the monitoring allocation units 212A, 212B, and 212C and the processing of the occupant state detection units 213A, 213B, and 213C. In this way, a large amount of information can be processed quickly, and the physical condition of the occupants can be detected accurately at all times while the vehicle is running and when the vehicle is stopped.

[0096] The processing of the processors 210, 210A, 210B, and 210C in the first to fourth embodiments can be recorded on a recording medium readable by a computer system, and the program recorded on this recording medium can be read and executed by the processors 210, 210A, 210B, and 210C, thereby realizing the occupant state detection systems 1 to 1C of the present invention. The computer system here includes hardware such as an OS and peripheral devices.

[0097] Furthermore, if a WWW (World Wide Web) system is used, the "computer system" also includes the homepage provision environment (or display environment). The above program may be transmitted from a computer system that stores the program in a storage device or the like to another computer system via a transmission medium or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network (communication network) like the Internet or a communication line (communication line) like a telephone line.

[0098] The program may also be for realizing part of the above-mentioned functions. Furthermore, the above-mentioned functions may be realized in combination with a program already recorded in the computer system, that is, a so-called differential file (differential program).

[0099] The above describes an embodiment of the present invention in detail with reference to the drawings, but the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention. [Explanation of symbols]

[0100] 1. Occupant status detection system 1A: Occupant status detection system 1B: Occupant status detection system 10. Passenger monitoring device 100;Discrimination device 200: Occupant status detection device 200A: Occupant status detection device 200B: Occupant status detection device 200C: Occupant status detection device 210;processor 210A; Processor 210B; Processor 210C; processor 211A; Information receiving section 211B; Information receiving section 211C; Information receiving section 212A; Monitoring Allocation Division 212B; Surveillance Allocation Division 212C; Monitoring Allocation Division 213A: Occupant status detection unit 213B: Occupant status detection unit 213C: Occupant status detection unit 220;Memory

Claims

1. an occupant monitoring device that monitors the physical state of a vehicle occupant and acquires, from the body of the occupant via a predetermined device, first monitoring information including at least one of gaze direction, face direction, blink frequency, eye opening, eye movement, posture, and body movement, and second monitoring information including at least one of blood flow, pulse rate, respiratory rate, sleep time, sleep state, number of turns in sleep, and rest time; a discrimination device for discriminating between a running state and a stopped state of the vehicle; an occupant status detection device that detects the status of an occupant of the vehicle based on occupant monitoring information including the first monitoring information and the second monitoring information from the occupant monitoring device; Equipped with the occupant state detection device includes one or more processors and one or more memories communicatively connected to the one or more processors; The one or more memories include: Comparison information during the traveling state is collation data for detecting the state of the occupant of the vehicle during the traveling state, the comparison information including an index indicating a relationship between the first monitoring information and the behavior of the occupant, which includes at least one of inattentiveness, drowsiness, fatigue, and abnormal posture; comparison information during a stopped vehicle, which is collation data for detecting the state of an occupant of the vehicle in the stopped state, and includes an index indicating a relationship between the second monitoring information and at least a physical abnormality of the occupant; is stored, the one or more processors: Switching the monitoring target according to the running state of the vehicle; When it is detected that the vehicle is traveling, the monitoring target is switched to the behavior of the occupant, and an abnormality in the behavior of the occupant is detected by comparing the first monitoring information with the traveling state comparison information; An occupant condition detection system characterized in that, when it detects that the vehicle is stopped, the monitoring target is switched to the occupant's body, and any abnormality in the occupant's body is detected by comparing the second monitoring information with the comparison information when the vehicle is stopped.

2. The occupant state detection system of claim 1, wherein the discrimination device acquires ignition information, and when the ignition is in the OFF state, transmits discrimination information to the one or more processors indicating that the vehicle is in the stopped state, and when the ignition is in the ON state, transmits discrimination information to the one or more processors indicating that the vehicle is in the running state.

3. the occupant monitoring device includes a camera that captures images of the vehicle occupants; The occupant state detection system of claim 1 , further comprising: acquiring and transmitting the occupant monitoring information, the occupant monitoring information including the first monitoring information and the second monitoring information, to the one or more processors.

4. the one or more processors:

4. The occupant status detection system according to claim 3, wherein when the occupant monitoring device detects that the vehicle is moving, it switches the monitoring target to the occupant in the driver's seat of the vehicle, and when it detects that the vehicle is stopped, it switches the monitoring target to the occupants throughout the interior of the vehicle.

5. 4. The occupant state detection system of claim 3, wherein the one or more processors, when detecting that the vehicle is moving, switch the monitoring target of the occupant monitoring device to the behavior and body of the occupant of the vehicle, and when detecting that the vehicle is stopped, switch the monitoring target to the behavior of the occupant of the vehicle.

Images