Driver status detection device, driver status detection method, and program

Abstract

To more accurately detect the driver's condition. [Solution] The driver state detection device of the embodiment includes a recognition unit that recognizes the monitoring direction of the driver of a moving object and the behavior of the moving object; a determination unit that determines whether the monitoring direction of the driver recognized by the recognition unit is appropriate or not; a counting unit that counts the time during which the determination unit determines that the monitoring direction of the driver is inappropriate; and an alarm output unit that outputs alarm information when the count value from the counting unit is equal to or greater than a threshold. The counting unit stops counting while the recognition unit recognizes that the behavior of the moving object is disordered, or resets the count value when disordered behavior of the moving object is recognized.

Description

Technical Field

[0001] The present invention relates to a driver state detection device, a driver state detection method, and a program.

Background Art

[0002] In recent years, efforts have been actively made to provide access to a sustainable transportation system that takes into account people in vulnerable positions among traffic participants. Toward this realization, research and development have focused on further improving traffic safety and convenience through research and development related to preventive safety technologies. In this regard, conventionally, a change pattern of first detection information indicating the driver's line of sight or face direction is acquired, and when the change pattern corresponds to a reference pattern preset according to the driver's lane confirmation operation at the time of lane change, a technique for temporarily switching the normal determination conditions set for side glance determination to determine side glance is known (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, in conventional preventive safety technologies, there are situations where it is necessary for the driver to change the line of sight or face direction to monitor the surroundings even in the behavior of a moving object other than lane change, but the processing for such situations has not been considered. Therefore, there is a problem that the driver's state according to the situation of the moving object may not be appropriately detected.

[0005] One of the objectives of this application is to provide a driver status detection device, a driver status detection method, and a program that can more appropriately detect the driver's status according to the condition of the moving object, in order to solve the above-mentioned problems. Ultimately, this will contribute to the development of a sustainable transportation system. [Means for solving the problem]

[0006] The driver status detection device, driver status detection method, and program according to this invention employ the following configuration. (1) A driver state detection device according to one aspect of the present invention comprises: a recognition unit that recognizes the monitoring direction of the driver of a moving body and the behavior of the moving body; a determination unit that determines whether the monitoring direction of the driver recognized by the recognition unit is appropriate or not; a counting unit that counts the time during which the determination unit determines that the monitoring direction of the driver is inappropriate; and an alarm output unit that outputs alarm information when the count value from the counting unit is equal to or greater than a threshold, wherein the counting unit stops counting while the behavior of the moving body is recognized as irregular by the recognition unit, or resets the count value when irregularity in the behavior of the moving body is recognized.

[0007] (2) In the embodiment of (1) above, the determination unit determines whether the driver's monitoring direction is appropriate based on the driver's monitoring direction and a preset monitoring target area, and expands the monitoring target area if the recognition unit recognizes that the behavior of the moving body is abnormal.

[0008] (3) In the embodiment of (1) above, the counting unit performs counting when it is determined that the monitoring direction is inappropriate, even while the recognition unit recognizes that the behavior of the moving object is disordered, and resets the count when the disordered behavior subsides.

[0009] (4) In the embodiment of (3) above, the alarm output unit suppresses the output of the alarm information when the count value exceeds a threshold value while the behavior is disturbed.

[0010] (5) In the embodiment of (3) above, the counting unit resets the counted value when the recognition unit recognizes that the behavior of the moving body is disordered and determines that the monitoring direction is inappropriate.

[0011] (6) In the embodiment of (5) above, the counting unit resets the count value, including the count that was counted before the recognition unit recognized that the behavior of the moving object was abnormal.

[0012] (7) In the embodiment of (1) above, the disturbance of the behavior includes disturbance of the lateral behavior of the moving body.

[0013] (8) In the embodiment of (1) above, the device further comprises a movement control unit that controls the movement of the moving body, and the recognition unit recognizes that the behavior of the moving body is disordered when the movement control unit is performing control to stabilize the behavior of the moving body.

[0014] (9) In the embodiment of (4) above, the disturbance in behavior includes avoidance actions of the moving body, including lateral movements, by the driver's driving operations in response to the external conditions of the moving body.

[0015] (10): A driver status detection method according to another aspect of the present invention is a driver status detection method in which a computer recognizes the direction in which the driver of a moving object is being monitored and the behavior of the moving object, determines whether the recognized direction in which the driver is being monitored is appropriate, counts the time during which the driver is determined to be inappropriate, outputs alarm information if the counted value is greater than or equal to a threshold, stops the count while the behavior of the moving object is recognized to be irregular, or resets the count value when irregular behavior of the moving object is recognized.

[0016] (11): A program according to another aspect of the present invention is a program that causes a computer to recognize the monitoring direction of the driver of a moving body and the behavior of the moving body, to determine whether the recognized monitoring direction of the driver is appropriate or not, to count the time during which the monitoring direction of the driver is determined to be inappropriate, to output alarm information if the counted value is greater than or equal to a threshold, to stop the count while the behavior of the moving body is recognized to be irregular, or to reset the count value when irregular behavior of the moving body is recognized. [Effects of the Invention]

[0017] According to the embodiments described in (1) to (11) above, the driver's condition can be detected more appropriately depending on the condition of the moving object. [Brief explanation of the drawing]

[0018] [Figure 1] This is a configuration diagram of a vehicle system 1 including a driver status detection device according to an embodiment. [Figure 2] This diagram shows the relationship between the driver's line of sight and the area being monitored. [Figure 3] This diagram illustrates the relationship between the driver's monitoring direction over time, the count value, and the erratic behavior of vehicle M. [Figure 4] This flowchart shows an example of a first process performed by the driving support device 100 in the embodiment. [Figure 5] This flowchart shows an example of a second process performed by the driving support device 100 in the embodiment. [Modes for carrying out the invention]

[0019] Hereinafter, with reference to the drawings, embodiments of the driver state detection device, driver state detection method, and program of the present invention will be described. Hereinafter, an example in which the driver state detection device is applied to a moving body will be described. In addition, a vehicle will be used as an example of the moving body. The moving body includes, in addition to vehicles, for example, ships that can move on the ground (roads) such as hovercrafts, aircraft that can travel on roads, ride-on vehicles having a power unit, micromobility such as electric kick scooters, and the like.

[0020] [Overall Configuration] FIG. 1 is a configuration diagram of a vehicle system 1 including a driver state detection device according to an embodiment. The vehicle (hereinafter referred to as vehicle M) on which the vehicle system 1 is mounted is, for example, a two-wheeled, three-wheeled, four-wheeled vehicle, or micromobility, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using electric power generated by a generator connected to the internal combustion engine, or discharge power of a battery (storage battery) such as a secondary battery or a fuel cell.

[0021] The vehicle system 1 includes, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) 14, a communication device 20, an HMI (Human Machine Interface) 30, a vehicle sensor 40, a navigation device 50, an in-vehicle camera 70, a driving operator 80, a driving support device 100, a traveling driving force output device 200, a brake device 210, and a steering device 220. These devices and equipment are connected to each other by a multiplex communication line such as a CAN (Controller Area Network) communication line, a serial communication line, a wireless communication network, or the like. Note that the configuration shown in FIG. 1 is merely an example, and a part of the configuration may be omitted, or another configuration may be added. A combination of the camera 10, the radar device 12, and the LIDAR 14 is an example of the "detection device DD". The HMI 30 is an example of the "warning unit".

[0022] Camera 10 is a digital camera that uses a solid-state image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). Camera 10 is mounted at any location on the vehicle M on which the vehicle system 1 is installed. When imaging the front, camera 10 is mounted on the top of the front windshield, behind the rearview mirror, or on the front of the vehicle body. When imaging the rear, camera 10 is mounted on the top of the rear windshield or on the tailgate. When imaging the sides, camera 10 is mounted on the left and right door mirrors, etc. Camera 10 periodically and repeatedly images the area around vehicle M. Camera 10 may also be a stereo camera.

[0023] The radar device 12 emits radio waves (radar) such as millimeter waves around the vehicle M and detects radio waves (reflected waves) reflected by surrounding objects to detect at least the position (distance and bearing) of an object. The radar device 12 can be mounted at any location on the vehicle M. The radar device 12 may also detect the position and velocity of an object using the FM-CW (Frequency Modulated Continuous Wave) method.

[0024] The LIDAR 14 illuminates the area around the vehicle M with light and measures the scattered light. The LIDAR 14 detects the distance to the target based on the time from emission to reception. The emitted light is, for example, pulsed laser light. The LIDAR 14 can be mounted at any location on the vehicle M.

[0025] The communication device 20 communicates with other vehicles in the vicinity of vehicle M, terminal devices of users using vehicle M, or various server devices, for example, by utilizing networks such as cellular networks, Wi-Fi networks, Bluetooth®, DSRC (Dedicated Short Range Communication), LAN (Local Area Network), WAN (Wide Area Network), and the Internet.

[0026] The HMI 30 outputs various information to the occupants of the vehicle M (including the driver) and accepts input operations from the occupants. The HMI 30 includes, for example, a display unit 32 and a speaker 34. The display unit 32 is, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro Luminescence) display device. The display unit 32 displays various images (including video) in the embodiment. The display unit 32 may be configured integrally with the input unit as a touch panel. The speaker 34 outputs predetermined sounds (for example, alarm sounds or message sounds). In addition to (or instead of) the display unit 32 and speaker 34, the HMI 30 may also include a microphone, buzzer, touch panel, switches, keys, etc. The switches include switches that execute or terminate predetermined driving controls (for example, lane changes) that can be executed by the driving control unit described later, and switches that approve (permit) or reject driving control recommendations (suggestions) from the system (vehicle system 1). Furthermore, the switch may include switches for operating the turn signals (turn signal switches), etc. A turn signal switch is an example of a "turn signal operation unit."

[0027] The vehicle sensor 40 includes a vehicle speed sensor for detecting the speed of the vehicle M, an acceleration sensor for detecting acceleration, and a yaw rate sensor for detecting yaw rate (for example, the angular velocity of rotation around the vertical axis passing through the center of gravity of the vehicle M). The vehicle sensor 40 may also include a lateral acceleration sensor (lateral G sensor) for detecting the lateral acceleration (lateral G) of the vehicle M, a steering angle sensor for detecting the steering angle of the vehicle M (which may be the angle of the steering wheels or the operating angle of the steering wheel), a steering angular velocity sensor for detecting the steering angular velocity, and a compass sensor for detecting the orientation of the vehicle M.

[0028] The vehicle sensor 40 may also include a position sensor that detects the position of the vehicle M. The position sensor is, for example, a sensor that acquires position information (longitude and latitude information) from a GPS (Global Positioning System) device. Alternatively, the position sensor may be a sensor that acquires position information using, for example, a GNSS (Global Navigation Satellite System) receiver of a navigation device 50. The vehicle sensor 40 may derive the speed of the vehicle M from the difference (i.e., distance) of position information at a predetermined time from the position sensor. The results detected by the vehicle sensor 40 are output to the driver assistance device 100.

[0029] The navigation device 50 includes, for example, a GNSS receiver, a navigation HMI, and a route determination unit. The navigation device 50 may store map information in a storage device such as an HDD (Hard Disk Drive) or flash memory, or it may acquire map information 192 stored in a storage unit 190, which will be described later. The GNSS receiver determines the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M may be determined or supplemented by an INS (Inertial Navigation System) that utilizes the output of the vehicle sensor 40. The navigation HMI includes a display device, speaker, touch panel, keys, etc. The GNSS receiver may be provided on the vehicle sensor 40. The navigation HMI may be partially or completely shared with the HMI 30 described above. The route determination unit determines, for example, a route (hereinafter referred to as a route on a map) from the position of the vehicle M determined by the GNSS receiver (or any input position) to a destination input by the occupant using the navigation HMI, by referring to, for example, map information 192, etc. Furthermore, the navigation device 50 provides route guidance using the navigation HMI based on the determined route on the map. The navigation device 50 may also transmit its current location and destination to the navigation server via the communication device 20 and obtain a route equivalent to the route on the map from the navigation server.

[0030] Here, map information 192 is information in which the road shape is represented by links indicating roads (an example of a travel route) and nodes connected by those links. Map information 192 may also include POI (Point of Interest) information, etc. Map information 192 also includes, for example, the number of lanes (number of travel routes), the type and shape of road markings, information on the center of the lanes, or information on road boundaries. Map information 192 may also include information on whether the road boundary is a boundary (physical boundary) that includes structures that vehicles cannot pass through (including crossing and contact). Physical boundaries include, for example, guardrails, curbs, median strips, fences, etc. Map information 192 may also include road shape information, traffic regulation information, address information (address and postal code), facility information, parking information, telephone number information, etc. Road shape information includes, for example, the curvature of the road (which may be rephrased as radius of curvature; the same applies below), width, road surface gradient, branching and merging points, intersections, T-junctions, etc. Map information 192 may be updated as needed by the communication device 20 communicating with an external device.

[0031] The in-vehicle camera 70 is a digital camera that uses a solid-state image sensor such as a CCD or CMOS. The in-vehicle camera 70 is mounted at any location in the vehicle M in a position and orientation that allows it to capture the head of the driver seated in the driver's seat of the vehicle M from the front. For example, the in-vehicle camera 70 is mounted above the display device located in the center of the instrument panel of the vehicle M. The in-vehicle camera 70 may also capture images of the interior of the vehicle including the occupant (passenger) seated in the passenger seat of the vehicle M. The in-vehicle camera 70 may also capture images of the interior of the vehicle by irradiating infrared light into the interior. The in-vehicle camera 70 captures images of the interior of the vehicle repeatedly and periodically.

[0032] The driver control unit 80 includes, for example, a steering wheel, an accelerator pedal, and a brake pedal. The driver control unit 80 may also include a shift lever, a modified steering wheel, a joystick, or other controls. Each control of the driver control unit 80 is equipped with an operation detection unit that detects, for example, the amount of operation performed by the driver on the control or whether or not an operation has been performed. The operation detection unit detects, for example, the steering angle and steering torque of the steering wheel (for example, the amount of steering due to the driver's driving operation (steering input torque)), the rate of change of the steering torque, the amount of depression of the accelerator pedal and brake pedal, etc. The operation detection unit then outputs the detection results to the driver assistance device 100, or to one or both of the driving force output device 200, the brake device 210, and the steering device 220. The driver control unit 80 may also include a turn signal control unit (for example, a turn signal lever, a turn signal switch). When the turn signal control unit is operated, the turn signal of the vehicle M corresponding to the operation will flash, and the operation details (including, for example, the detection result that the operation was performed by the driver) will be output to the driver assistance device 100.

[0033] The driver assistance device 100 performs various controls to assist the driver of the vehicle M. The driver assistance device 100 includes, for example, a recognition unit 120, a determination unit 140, a counting unit 150, a driving control unit 160, an HMI control unit 180, and a storage unit 190. The recognition unit 120, the determination unit 140, the counting unit 150, the driving control unit 160, and the HMI control unit 180 are each realized by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Furthermore, some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), or SOC (System On Chip), or by the cooperation of software and hardware. The above-mentioned program may be stored in advance in a storage device (a storage device equipped with a non-transient storage medium) such as the HDD or flash memory of the driver assistance device 100, or it may be stored in a removable storage medium such as a DVD, CD-ROM, or memory card, and installed in the storage device of the driver assistance device 100 when the storage medium (non-transient storage medium) is inserted into a drive device or card slot. The driver assistance device 100 is an example of a "driver state detection device". The driving control unit 160 is an example of a "mobility control unit". The HMI 30 and HMI control unit 180 are examples of "alarm output units".

[0034] The storage unit 190 may be implemented using the various storage devices described above, or an EEPROM (Electrically Erasable Programmable Read Only Memory), ROM (Read Only Memory), or RAM (Random Access Memory), etc. The storage unit 190 stores, for example, map information 192, various information in the embodiment, programs, etc. The storage unit 190 may also store various setting information used in the processing in this embodiment.

[0035] The recognition unit 120 includes, for example, a surrounding recognition unit 122, a monitoring direction recognition unit 124, and a behavior recognition unit 126. The surrounding recognition unit 122 recognizes the surrounding conditions of the vehicle M based on, for example, the detection results of the detection device DD (information input from the camera 10, radar device 12, and LIDAR 14). For example, the surrounding recognition unit 122 performs sensor fusion processing on the detection results from some or all of the camera 10, radar device 12, and LIDAR 14 to recognize the position (relative position), size, speed (relative speed), acceleration, and other states of objects present around the vehicle M (within a predetermined distance). Objects recognized by the surrounding recognition unit 122 may include, for example, physical boundaries that demarcate roads (travel paths), as well as other vehicles, pedestrians, bicycles, and other traffic participants (examples of obstacles). The position of an object is recognized as a position on an absolute coordinate system with the vehicle M's representative point (such as the center of gravity or the drive axis center) as the origin, and is used for control. The position of an object may be represented by its center of gravity, a representative point such as a corner, or by the represented region. The "state" of an object may include, for example, the acceleration or jerk of another vehicle, or its "action state" (for example, whether the other vehicle is changing lanes or is about to change lanes).

[0036] Furthermore, the surrounding area recognition unit 122 may recognize, for example, stop lines, red lights, toll booths, other road events, road signs, and markings drawn on the road (e.g., speed limits). The surrounding area recognition unit 122 may also recognize the curvature of the vehicle M's lane (road) based on the detection results of the detection device DD or the map information 192. The surrounding area recognition unit 122 may also recognize the road surface conditions (e.g., whether the road surface is slippery, such as being frozen) based on the detection results of the detection device DD.

[0037] Furthermore, the surrounding recognition unit 122 recognizes, for example, the lane in which the vehicle M is traveling (driving lane) and other surrounding lanes (for example, adjacent lanes). For example, the surrounding recognition unit 122 recognizes road markings from images captured by the camera 10 and recognizes the driving lane and other lanes based on the positional relationship of the road markings as seen from the recognized vehicle M. Alternatively, the surrounding recognition unit 122 may refer to map information 192 based on the position information of the vehicle M obtained from the vehicle sensor 40, etc., to recognize the lane in which the vehicle M is traveling and other lanes.

[0038] The monitoring direction recognition unit 124 recognizes the state of the occupants of vehicle M using images captured by the in-vehicle camera 70. For example, the monitoring direction recognition unit 124 performs known image analysis processing on the images captured by the in-vehicle camera 70 and recognizes the monitoring direction of the driver of vehicle M based on the analysis results. The monitoring direction includes, for example, at least one of the driver's line of sight (the direction the driver is looking) and the direction of the driver's face.

[0039] For example, the monitoring direction recognition unit 124 uses methods such as template matching to detect combinations of reference points (the stationary part of the eye) and moving points (the moving part of the eye) of the driver's eyes from the image. The combination of reference points and moving points may be, for example, the inner corner of the eye and the iris, or the corneal reflection area and the pupil. The corneal reflection area is, for example, the area of ​​infrared light reflection on the cornea when the in-vehicle camera 70 shines infrared light towards the driver. The monitoring direction recognition unit 124 then recognizes the driver's line of sight by performing coordinate transformations from the image plane to real space based on the position of the moving points relative to the reference points. The monitoring direction recognition unit 124 also recognizes the orientation of the driver's face based on positional information of the eyes, nose, mouth, etc. within the face region obtained from the image analysis results (relative positional information of each part, etc.).

[0040] Furthermore, the monitoring direction recognition unit 124 may recognize the shape of the driver's eyes from the analysis results of the images from the in-vehicle camera 70, or recognize whether the driver is wearing sunglasses or glasses through template matching or the like. The monitoring direction recognition unit 124 may also determine whether the driver is monitoring the area around the vehicle M based on the driver's gaze and the direction of their face. In addition, the monitoring direction recognition unit 124 may recognize the driver's posture (movement) and the amount of change (amount of movement) over a predetermined period of time.

[0041] In recognizing each piece of information using the images described above, a pre-trained model, for example, that has been trained in advance by machine learning, may be used. In this case, the monitoring direction recognition unit 124 takes an image as input and outputs the gaze and face direction of a person contained in the image, and inputs the image captured by the in-vehicle camera 70 to acquire the driver's gaze and face direction.

[0042] The behavior recognition unit 126 recognizes the behavior of vehicle M based on the detection results of the vehicle sensor 40, the detection results of the operation detection unit of the driver control unit 80, and the control content executed by the driving control unit 160. The behavior of vehicle M includes the behavior caused by the driver's manual driving and the behavior caused by the driving control executed by the driving control unit 160. The behavior recognition unit 126 may also recognize disturbances in the behavior of vehicle M. A disturbance in behavior is, for example, a disturbance in behavior where the amount of change in the lateral position or orientation of vehicle M over a predetermined time exceeds a threshold. A disturbance in behavior may include, for example, a skid of vehicle M. A skid is a disturbance in which the front or rear wheels of vehicle M drift outward, for example, when vehicle M fails to complete a turn while driving on a curved road, when the driver makes a sudden steering operation, or when steering in a situation where the road surface is frozen. For example, disturbances in behavior may include disturbances in the lateral behavior of the vehicle, and may also include evasive maneuvers, including lateral movements of the vehicle M, caused by the driver's driving operations (manual driving) in response to external conditions of the vehicle M (e.g., approaching obstacles).

[0043] For example, the behavior recognition unit 126 recognizes the lateral position (position in the lane width direction) of the vehicle M relative to the driving lane and the attitude (orientation) of the vehicle M relative to the direction of extension of the driving lane, based on the positional relationship of the vehicle M with respect to the driving lane. For example, the behavior recognition unit 126 may recognize the deviation of the vehicle M's reference point from the center of the lane, and the angle it makes with a line connecting the centers of the lanes in the direction of travel of the vehicle M, as the relative position and attitude of the vehicle M relative to the driving lane. Alternatively, the behavior recognition unit 126 may recognize the position of the vehicle M's reference point relative to any side edge of the driving lane (road marking or road boundary), etc., as the relative position (lateral position) of the vehicle M relative to the driving lane. Furthermore, the behavior recognition unit 126 may recognize the lateral behavior of the vehicle M (for example, whether or not it has moved laterally by a predetermined distance or more in a predetermined time) from the amount of change in the lateral position and orientation (yaw rate) of the vehicle M as described above, and may recognize that the behavior is erratic if the amount of change is greater than a predetermined amount.

[0044] Furthermore, the behavior recognition unit 126 may detect the behavior of the vehicle M from the amount of steering wheel operation (e.g., steering angle, steering torque, steering torque change rate), the amount of depression of the accelerator pedal and brake pedal, etc., obtained by the operation detection unit when the vehicle M is being driven manually, or it may recognize disturbances in the behavior from the amount of change in the behavior over a predetermined period of time. In addition, the behavior recognition unit 126 may compare the amount of steering wheel operation obtained by the operation detection unit with the yaw rate and lateral acceleration (lateral G) of the vehicle M obtained from the vehicle sensor 40, and recognize disturbances in the behavior of the vehicle M, such as skidding, if the actual yaw rate and lateral acceleration are greater than the yaw rate and lateral acceleration set in advance in accordance with the amount of operation (greater than a predetermined allowable range).

[0045] Furthermore, the behavior recognition unit 126 may recognize the behavior of the vehicle M based on the content of the driving control performed by the driving control unit 160. Driving control is a control that drives the vehicle M by controlling at least one of the steering and speed of the vehicle M, without driver operation or by accepting only some of the driver's instructions. Driving control includes, for example, ACC (Adaptive Cruise Control System), LKAS (Lane Keeping Assistance System), ALC (Auto Lane Changing), etc. Driving control may also include control to stop the vehicle M in a safe position such as the shoulder of the road based on the determination result of the determination unit 140, and control to control the steering and speed to avoid contact between the vehicle M and obstacles recognized by the surrounding recognition unit 122. Driving control also includes, for example, VSA (Vehicle Stability Assist). VSA is a system that automatically stabilizes the behavior of the vehicle M when a skid or other similar incident occurs. VSA includes, for example, safety devices such as ABS (Anti-lock Braking System) which reduces skidding due to wheel lock during sudden deceleration or deceleration on low-friction roads, TCS (Traction Control System) which prevents wheelspin during starting and acceleration, and a skid suppression device, or a system that comprehensively controls these safety devices. For example, the behavior recognition unit 126 recognizes the behavior of the vehicle M due to the execution of LKAS or ALC by the driving control unit 160. The behavior recognition unit 126 may also recognize that the behavior of the vehicle M is erratic when VSA is being executed by the driving control unit 160.

[0046] The determination unit 140 includes, for example, a monitoring determination unit 142. The monitoring determination unit 142 determines, for example, whether the driver's monitoring direction is appropriate based on at least one of the driver's gaze direction or face direction recognized by the monitoring direction recognition unit 124 and a preset monitoring target area.

[0047] Figure 2 is a diagram illustrating the relationship between the driver's line of sight and the monitored area. In the example in Figure 2, driver D is seated in the driver's seat ST1 of vehicle M and is performing manual driving of vehicle M by operating the driving controls 80 such as the steering wheel SW. In the example in Figure 2, display units 32-1 and 32-2 included in the HMI 30 are also shown. For example, when vehicle M is traveling in a straight direction (X-axis direction in the figure), the monitoring and determination unit 142 sets a monitored area AR1 that has a predetermined angle to the left and right with respect to the direction of travel V of vehicle M from the position of driver D's head, as shown in Figure 2.

[0048] The monitored area AR1 may be adjusted according to the vehicle M's speed, road shape, recognition results from the recognition unit 120, etc. In this case, for example, the angle (radians) θ1 indicating the size of the arc of the monitored area may be set to decrease in proportion to the speed, or to increase in proportion to the width of the road. Furthermore, when the vehicle M changes lanes from the driving lane to the adjacent lane on the right, the current area of ​​the monitored area AR1 is rotated to the right around the position of the driver D's head, and when the vehicle M changes lanes from the driving lane to the adjacent lane on the left, the current area is rotated to the left around the position of the driver D's head. In this case, the size of the angle θ1 may also be changed. Similarly, when the vehicle M turns right or left, the area of ​​the monitored area AR1 may also be changed. Furthermore, the monitored area AR1 may be expanded when the behavior recognition unit 126 recognizes that the vehicle M's behavior is erratic.

[0049] In this situation, the monitoring and determination unit 142 determines that the monitoring direction of driver D is appropriate if, for example, the line of sight of driver D recognized by the recognition unit 120 is within angle θ1 of the monitored area AR1 (or if the state of being within angle θ1 continues for a predetermined time or longer), and determines that the monitoring direction of driver D is inappropriate if the line of sight is not within angle θ1 (or if the state of not being within angle θ1 continues for a predetermined time or longer). In the example in Figure 2, if driver D's line of sight is line of sight A1, it is determined that the monitoring direction of driver D is appropriate, and if it is line of sight A2, it is determined that the monitoring direction of driver D is inappropriate.

[0050] Furthermore, the monitoring and determination unit 142 may, instead of (or in addition to) the driver D's gaze, compare the direction of driver D's head recognized by the recognition unit 120 with the monitored area AR1 to determine whether the driver D's monitoring direction is appropriate. For example, if the monitoring direction recognition unit 124 recognizes the driver D's gaze, the monitoring and determination unit 142 will make the above determination based at least on the gaze and the monitored area AR1. If the driver D is wearing sunglasses or glasses and the gaze is not recognized, but the direction of the face is recognized, the monitoring and determination unit 142 will make the above determination using the direction of driver D's face.

[0051] Returning to Figure 1, the counting unit 150 counts the time during which the monitoring and determination unit 142 has determined that the driver D's monitoring direction is inappropriate. The counting unit 150 may also stop counting while the vehicle M's behavior is erratic. Furthermore, the counting unit 150 may reset (initialize) at least a portion of the previously counted value (count value) if the vehicle M's behavior becomes erratic. The counting unit 150 may also count the time during which the monitoring and determination unit 142 has determined that the driver D's monitoring direction is appropriate. Details of the functions of the counting unit 150 will be described later.

[0052] The driving control unit 160 controls the driving of the vehicle M (movement of the moving body). For example, the driving control unit 160 performs driving control on the vehicle M based on the recognition results from the recognition unit 120 and the determination results from the determination unit 140. Driving control may be performed in response to an execution instruction from the driver D via the HMI 30, or it may be performed based on the recognition results from the recognition unit 120 without an instruction from the driver D. When performing driving control, the driving control unit 160 generates a future target trajectory for the vehicle M according to the content of the driving control based on the recognition results from the recognition unit 120, and controls at least one of the steering and speed of the vehicle M so that the vehicle M travels along the generated target trajectory.

[0053] For example, the driving control unit 160 includes a VSA control unit 162. When the behavior recognition unit 126 recognizes that the behavior of the vehicle M is erratic, the VSA control unit 162 executes control (e.g., VSA) to automatically stabilize the behavior of the vehicle M. VSA is executed continuously, for example, until the change in the behavior (lateral behavior) of the vehicle M falls below a predetermined value. In addition to the control by the VSA control unit 162, the driving control unit 160 also performs driving control such as ACC, LKAS, and ALC. Furthermore, if the driver's monitoring direction does not improve even after a predetermined time has elapsed since the HMI control unit 180 outputted warning information indicating that the driver's monitoring direction is inappropriate, the driving control unit 160 may perform control to stop the vehicle M in a safe location such as the shoulder of the road.

[0054] The HMI control unit 180 notifies the occupants (including driver D) of predetermined information via the HMI 30 and receives information input by the HMI 30. The predetermined information includes, for example, information related to the driving of vehicle M, such as information regarding the status of vehicle M and information regarding driving control. Information regarding the status of vehicle M includes, for example, the speed of vehicle M, engine speed, and shift position. Information regarding driving control includes, for example, whether or not driving control is being performed by the driving control unit 160, information regarding the status of driving control, information regarding driving control recommendations from the system, and warning information for driver D. The predetermined information may also include information regarding the surrounding conditions recognized by the detection device DD. The predetermined information may also include information unrelated to the driving of vehicle M, such as content (e.g., video) stored on a storage medium such as a television program or DVD. The predetermined information may also include, for example, information regarding the current location and destination of vehicle M, and the remaining fuel level of vehicle M. The HMI control unit 180 may output the information received by the HMI 30 to the communication device 20, recognition unit 120, determination unit 140, navigation device 50, etc.

[0055] Furthermore, the HMI control unit 180 may output to the HMI 30 information such as inquiry information for the occupant, recognition results from the recognition unit 120, and determination results from the determination unit 140. In addition, the HMI control unit 180 may transmit various information to be output to the HMI 30 to a terminal device used by the occupant of the vehicle M via the communication device 20.

[0056] For example, when the count value from the count unit 150 exceeds a threshold, the HMI control unit 180 generates alarm information (images, sounds, etc.) related to the driver D's monitoring and notifies the driver D by outputting the generated alarm information to the HMI 30 (display unit 32-1, display unit 32-2, speaker 34, etc.). The alarm information may include, for example, information to notify that the monitoring direction is inappropriate, or information to notify the correct monitoring direction (e.g., the monitoring target area AR1). The alarm information may also include information regarding the count value counted by the count unit 150, information indicating that the count has been stopped, or information indicating that the count has been reset. This allows the driver D to more accurately understand the control status on the system side.

[0057] For example, the HMI control unit 180 may output alarm information until the monitoring and determination unit 142 determines that the monitoring direction of driver D is appropriate, or it may output alarm information until the time for which the counting unit 150 has determined that the monitoring direction of driver D is appropriate exceeds a predetermined time.

[0058] The driving force output device 200 outputs driving force (torque) to the drive wheels for the vehicle to move. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, and a transmission, and an ECU (Electronic Control Unit) that controls them. The ECU controls the above configuration according to information input from the driving control unit 160 or information input from the accelerator pedal of the driver control unit 80.

[0059] The brake system 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the driving control unit 160 or from the brake pedal of the driver control unit 80, so that brake torque corresponding to the braking operation is output to each wheel. The brake system 210 may be equipped with a backup mechanism that transmits hydraulic pressure generated by the operation of the brake pedal to the cylinder via a master cylinder. The brake system 210 is not limited to the configuration described above, and may also be an electronically controlled hydraulic brake system that controls an actuator according to information input from the driving control unit 160 to transmit hydraulic pressure from the master cylinder to the cylinder.

[0060] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor, for example, applies force to a rack and pinion mechanism to change the direction of the steering wheels. The steering ECU drives the electric motor to change the direction of the steering wheels according to information input from the driving control unit 160 or from the steering wheel of the driver control unit 80.

[0061] [Counting section] Next, the functions of the counting unit 150 will be described in detail. The counting unit 150 counts the time during which the monitoring and determination unit 142 determines that the driver D's monitoring direction is inappropriate. For example, the monitoring and determination unit 142 performs the monitoring and determination repeatedly at predetermined intervals. Therefore, the counting unit 150 can calculate the time (period) by increasing the count value by 1 each time the monitoring and determination unit 142 determines that the driver D's monitoring direction is inappropriate. This count value continues to increase as long as the monitoring and determination unit 142 continues to determine that the driver D's monitoring direction is inappropriate, and may be reset when the monitoring and determination unit 142 determines that the driver D's monitoring direction is appropriate (or a predetermined time has elapsed since the monitoring and determination unit 142 determined that the monitoring direction is appropriate).

[0062] In this case, if the behavior of vehicle M is erratic, driver D is likely to shift their gaze and head direction to look around vehicle M. In such a case, driver D's gaze will no longer be within the monitored area AR1, but it is undesirable for the aforementioned warning information to be output. Therefore, the counting unit 150 stops counting while the behavior recognition unit 126 recognizes that the behavior of vehicle M is erratic.

[0063] Figure 3 illustrates the relationship between the driver's monitoring direction, the count value, and the disturbance in the vehicle M's behavior over time. In the example in Figure 3, the horizontal axis represents time, and the vertical axis represents the driver D's monitoring direction, the on / off status of the count unit 150, and the vehicle behavior (e.g., yaw rate or steering angle). The sign of the vehicle behavior indicates that a plus (+) sign represents one direction to the left or right relative to the reference direction (0), and a minus (-) sign represents the other direction. In the example in Figure 3, each time point T0 to T4 indicates that time has progressed sequentially from T0 to T4. In addition, each plot point shown in Figure 3 represents the driver's monitoring direction (top of Figure 3), the count value (middle of Figure 3), and the steering angle value (bottom of Figure 3) detected at a predetermined cycle.

[0064] For example, in Figure 3, the periods from time T1 to T2 and T3 to T4 are times when the driver D's monitoring direction is outside the monitoring area and the monitoring determination unit 142 has determined that the driver D's monitoring direction is inappropriate (times T0 to T1 also include a period when the driver D's monitoring direction is outside the monitoring area, but for the sake of explanation, this will be omitted). Also, the periods from time T2 to T3 are times when the driver D's monitoring direction is within the monitoring area and the monitoring determination unit 142 has determined that the driver D's monitoring direction is appropriate. For example, the HMI control unit 180 outputs alarm information regarding the monitoring of driver D when the count value counted from time T1 exceeds a threshold.

[0065] Here, as shown in Figure 3, if the behavior recognition unit 126 recognizes that the behavior of vehicle M is erratic at time Ta, which is after time T1 but before the count value exceeds the threshold, the count unit 150 stops counting time. For example, the count unit 150 stops counting time during the period from time Ta to time Tb, when the monitoring determination unit 142 has determined that the driver D's monitoring direction is inappropriate, and the vehicle M's behavior is no longer recognized as erratic (the erratic behavior of vehicle M converges to below a predetermined level). Therefore, between time Ta and Tb, the count value does not increase even if the driver D's monitoring direction is outside the monitored area, so the count value never exceeds the threshold, and no alarm information is output. As a result, even if the vehicle M's behavior is erratic and the driver D is looking around to check the situation, it is not judged as distracted driving, and the output of alarm information can be suppressed.

[0066] Furthermore, after time Tb, when the behavior of vehicle M is no longer recognized as abnormal from time Ta (i.e., when the behavior is recognized as not abnormal), if the driver D's monitoring direction is outside the monitoring area and the monitoring determination unit 142 determines that the driver D's monitoring direction is inappropriate, the count unit 150 continues counting from the count value up to time Ta. Also, if the count value exceeds a threshold, the HMI control unit 180 outputs alarm information. This allows for more appropriate detection of the driver's state according to the vehicle M's condition. In addition, appropriate control (e.g., alarm control or driving control) can be executed according to the detection result.

[0067] In the example in Figure 3, even if the behavior of vehicle M is recognized as erratic, and the monitoring direction of driver D is within the monitoring area, and the monitoring determination unit 142 determines that the monitoring direction of driver D is appropriate (for example, if the time intervals T2 to T3 in Figure 3 exist), the monitoring direction may not necessarily be the correct monitoring direction in a situation where the behavior of vehicle M is erratic, so it is not necessary to reset the count value.

[0068] [Differentiation] In Figure 3 of the embodiment, time Ta may be the time when VSA control (control to stabilize the behavior of the vehicle M) is executed by the VSA control unit 162 of the driving control unit 160, and time Tb may be the time when the VSA control is completed. Therefore, the counting unit 150 may stop counting while VSA control is being executed, or it may reset the count value at the time when the VSA control was started or the time when the VSA control was completed.

[0069] In addition, in this embodiment, instead of stopping the count while the behavior recognition unit 126 recognizes that the behavior of the vehicle M is irregular, as described above, the count unit 150 may reset the count value when irregular behavior of the vehicle M is recognized. In this case, the count unit 150 may reset the count value up to that point at time Ta when irregular behavior of the vehicle M is recognized, or it may reset the count value up to that point (including the count value that had been counted up to time Ta) when the irregular behavior of the vehicle M converges to below a predetermined level (when time Tb arrives).

[0070] Furthermore, the counting unit 150 may, for example, count time even while the behavior recognition unit 126 recognizes that the behavior of vehicle M is erratic, if the monitoring determination unit 142 determines that the driver D's monitoring direction is inappropriate (for example, in the case of times Ta~T2 and T3~Tb in Figure 3), and reset the count value when the erratic behavior of vehicle M subsides (when time Tb arrives). In this case, the HMI control unit 180 suppresses the output of alarm information even if the count value exceeds a threshold while the behavior of vehicle M is recognized as erratic. Suppression may mean not outputting alarm information at all, or it may mean outputting information with a lower alarm level than usual. Outputting information with a lower alarm level may mean, for example, outputting an alarm sound at a lower volume than usual or displaying a simpler message image than usual.

[0071] Furthermore, when resetting the count value, the count unit 150 may reset the count value if the behavior recognition unit 126 recognizes that the behavior of vehicle M is erratic and the monitoring determination unit 142 determines that the driver D's monitoring direction is inappropriate.

[0072] Furthermore, in the above-described embodiment, when the monitoring determination unit 142 determines whether the monitoring direction of driver D is appropriate based on the monitoring direction of driver D and a preset monitoring target area, and the behavior recognition unit 126 recognizes that the behavior of vehicle M is erratic, the monitoring target area may be expanded. Expanding the monitoring target area means, for example, increasing the angle θ1 shown in Figure 2. This makes it less likely that the monitoring direction of driver D will be determined to be inappropriate, and as a result, the time counting by the counter unit 150 can be stopped.

[0073] In this embodiment, instead of determining whether the driver D's monitoring direction is appropriate, the monitoring determination unit 142 may perform a distraction determination to determine whether the driver D is distracted. In this case, the monitoring determination unit 142 determines that the driver D is distracted if the driver D's monitoring direction is not within the monitoring area, and if it is determined that the driver is distracted, the counting unit 150 counts the time.

[0074] [Processing flow] The following describes the processes performed by the driver assistance device 100 of this embodiment. Specifically, the following describes the processes performed by the driver assistance device 100, focusing on the control of alarm output according to the driver D's state. Furthermore, two different processes will be described below.

[0075] [First process] Figure 4 is a flowchart showing an example of a first process performed by the driving support device 100 in the embodiment. Note that the process shown in Figure 4 may be repeatedly performed at predetermined timings or predetermined cycles.

[0076] In the example in Figure 4, the monitoring direction recognition unit 124 recognizes the monitoring direction of the driver D of the vehicle M (at least one of the gaze direction and the direction of the face) (step S100). Next, the behavior recognition unit 126 recognizes the behavior of the vehicle M (step S110). Next, the monitoring determination unit 142 determines whether the driver D's monitoring direction is appropriate or not (step S120). If it is determined that the driver D's monitoring direction is not appropriate, the counting unit 150 determines whether the vehicle M's behavior is erratic based on the recognition result of the behavior recognition unit 126 (step S130). If it is determined that the vehicle M's behavior is erratic, the counting unit 150 stops counting during the period of erratic behavior or resets the count value (step S140). Also, if it is determined that the vehicle M's behavior is not erratic based on the processing in step S130, the counting unit 150 counts the time during which the driver D's monitoring direction is inappropriate (step S150).

[0077] Next, the HMI control unit 180 determines whether the count value counted by the counting unit 150 is above a threshold (step S160). If it is determined to be above the threshold, the HMI control unit 180 generates alarm information and outputs it to the HMI 30 (step S170). If, in the process of step S160, it is determined that the count value is not above the threshold, the process of this flowchart ends. Also, if, in the process of step S120, it is determined that the monitoring direction of driver D is appropriate, the process of this flowchart ends.

[0078] [Second process] Next, the second process will be described. The second process suppresses the alarm output if the vehicle M's behavior is deemed abnormal, even if the count value from the counting unit 150 is above a threshold. Figure 5 is a flowchart showing an example of the second process performed by the driving support device 100 in this embodiment. Note that the process shown in Figure 5 may be repeatedly executed at predetermined timings or predetermined cycles.

[0079] In the example in Figure 5, the monitoring direction recognition unit 124 recognizes the monitoring direction of the driver D of the vehicle M (at least one of the gaze direction and the direction of the face) (step S200). Next, the behavior recognition unit 126 recognizes the behavior of the vehicle M (step S210). Next, the monitoring determination unit 142 determines whether the monitoring direction of the driver D is appropriate or not (step S220). If it is determined that the monitoring direction is not appropriate, the counting unit 150 counts the time (step S230).

[0080] Next, the HMI control unit 180 determines whether the count value counted by the counting unit 150 is above a threshold (step S240). If it is determined to be above the threshold, the HMI control unit 180 determines whether the behavior of vehicle M is abnormal based on the recognition result of the behavior recognition unit 126 (step S250). If it is determined that the behavior of vehicle M is abnormal, the HMI control unit 180 suppresses the output of alarm information (step S260). Also, if it is determined in the process of step S250 that the behavior of vehicle M is not abnormal, the HMI control unit 180 generates alarm information and outputs it to the HMI 30 (step S270). This completes this flowchart. Furthermore, if it is determined in the process of step S220 that the monitoring direction is appropriate, or if it is determined in the process of step S240 that the count value is not above the threshold, the process of this flowchart also completes.

[0081] Furthermore, one of the first and second processes described above may be combined with a part of the other process. In addition, in the second process, the count value may be reset when the disturbance in the vehicle M's behavior converges.

[0082] According to the embodiment described above, the driver state detection device includes a recognition unit 120 that recognizes the driver's monitoring direction and the behavior of the vehicle M (an example of a moving object), a determination unit 140 that determines whether the driver's monitoring direction recognized by the recognition unit 120 is appropriate, a count unit 150 that counts the time during which the determination unit 140 determines that the driver's monitoring direction is inappropriate, and an alarm output unit (HMI 30, HMI control unit 180) that outputs alarm information when the count value from the count unit 150 is above a threshold. The count unit 150 stops counting while the recognition unit 120 recognizes that the behavior of the vehicle M is erratic, or resets the count value when erratic behavior of the vehicle M is recognized, thereby enabling more appropriate detection of the driver's state according to the situation of the vehicle M. Therefore, alarm information can be output to the occupant of the vehicle M at a more appropriate timing, and more appropriate driving control can be performed on the vehicle M.

[0083] For example, according to the embodiment, the count value for outputting a warning to the driver in response to disturbances in the behavior of vehicle M is adjusted, so that, for example, warnings are not output during or immediately after the execution of VSA control. Also, according to the embodiment, for example, when the behavior of vehicle M becomes disturbed, the driver D will check the surroundings in directions other than straight ahead when making corrective steering maneuvers, but by stopping the count or resetting the count, it is possible to prevent the surrounding check during the disturbance from being mistaken for distracted driving, and to avoid the inconvenience of warnings being output immediately after the disturbance subsides. Furthermore, according to the embodiment, even if the monitoring direction is determined to be inappropriate and a count is performed while the behavior is disturbed, by resetting the count when the disturbance subsides, it is possible to avoid the inconvenience of warnings being output immediately after the disturbance subsides.

[0084] Furthermore, according to the embodiment, even if the count continues, the alarm output is suppressed even if the count exceeds a predetermined level while the behavior is erratic, thus preventing the annoyance of alarms being output at unnecessary times. Also, according to the embodiment, by resetting only the count value that was counted while the behavior was erratic, it is possible to prevent the alarm output from being suppressed too much, and to output the alarm at a more appropriate timing according to the driver's condition. Also, according to the embodiment, by resetting (fully resetting) the value that was counted before the behavior became erratic, it is possible to prevent the alarm from being output a short time after the vehicle M's behavior becomes normal.

[0085] Furthermore, in the embodiment, the disturbance in behavior may include disturbance in the lateral behavior of the vehicle M. For example, if the behavior is disturbed by a slip in the longitudinal direction, checking the surroundings is not very necessary, so by adjusting the count for lateral disturbances that require corrective steering, a warning can be output at a more appropriate timing. Also, in the embodiment, the disturbance in behavior may include lateral evasive maneuvers (manual driving) by the driver D in response to external conditions. For example, when performing an evasive maneuver to avoid an obstacle, the driver D will be looking in the direction where the obstacle is located, so it is possible to suppress warnings that such a visual action is distracted.

[0086] The embodiments described above can be expressed as follows. A storage medium that stores computer-readable instructions, A processor connected to the storage medium, The processor executes the computer-readable instructions to: The driver of the moving object is monitored in the direction the moving object is observed, and the behavior of the moving object is recognized. Determine whether the driver's observed direction is appropriate. The time during which the driver's monitoring direction is determined to be inappropriate is counted. If the counted value exceeds the threshold, an alarm message will be output. The count is stopped while the behavior of the moving object is recognized as irregular, or the count value is reset when irregularity in the behavior of the moving object is recognized. Driver status detection device.

[0087] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention. [Explanation of symbols]

[0088] 1...Vehicle system, 10...Camera, 12...Radar device, 14...LIDAR, 20...Communication device, 30...HMI, 40...Vehicle sensor, 50...Navigation device, 70...In-cabin camera, 80...Driver's control panel, 100...Driver's assistance device, 120...Recognition unit, 122...Surroundings recognition unit, 124...Monitoring direction recognition unit, 126...Behavior recognition unit, 140...Determination unit, 142...Monitoring and determination unit, 160...Driving control unit, 162...VSA control unit, 180...HMI control unit, 190...Storage unit, 200...Driving force output device, 210...Brake device, 220...Steering device, M...Vehicle

Claims

1. A recognition unit that recognizes the direction in which the driver of the moving object is being monitored and the behavior of the moving object, A determination unit that determines whether the driver's monitoring direction recognized by the recognition unit is appropriate, A counting unit that counts the time during which the determination unit determines that the driver's monitoring direction is inappropriate, The system includes an alarm output unit that outputs alarm information when the count value from the count unit is equal to or greater than a threshold value, The counting unit stops counting while the recognition unit recognizes that the behavior of the moving object is irregular, or resets the count value when irregularity in the behavior of the moving object is recognized. Driver status detection device.

2. The determination unit, Based on the driver's monitoring direction and the pre-set monitoring target area, it is determined whether the driver's monitoring direction is appropriate. If the recognition unit detects that the behavior of the moving object is abnormal, the monitoring area is expanded. The driver status detection device according to claim 1.

3. The counting unit performs counting even while the recognition unit recognizes that the behavior of the moving object is disordered, if it is determined that the monitoring direction is inappropriate, and resets the count when the disordered behavior subsides. The driver status detection device according to claim 1.

4. The alarm output unit suppresses the output of the alarm information if the count value exceeds a threshold while the behavior is disrupted. The driver status detection device according to claim 3.

5. The counting unit resets the counted value when the recognition unit recognizes that the behavior of the moving object is irregular and determines that the monitoring direction is inappropriate. The driver status detection device according to claim 3.

6. The counting unit resets the count value, including the count that was counted before the recognition unit recognized that the behavior of the moving object was abnormal. The driver status detection device according to claim 5.

7. The disturbance in the aforementioned behavior includes disturbance in the lateral behavior of the moving body. The driver status detection device according to claim 1.

8. The system further comprises a movement control unit that controls the movement of the moving body, The recognition unit recognizes that the behavior of the moving body is erratic when the movement control unit is performing control to stabilize the behavior of the moving body. The driver status detection device according to claim 1.

9. The aforementioned disturbance in behavior includes avoidance maneuvers, including lateral movements, of the moving body by the driver's driving operations in response to the external conditions of the moving body. The driver status detection device according to claim 1.

10. Computers The driver of the moving object is monitored in the direction the moving object is observed, and the behavior of the moving object is recognized. Determine whether the driver's observed direction is appropriate. The time during which the driver's monitoring direction is determined to be inappropriate is counted. If the counted value exceeds the threshold, an alarm message will be output. The count is stopped while the behavior of the moving object is recognized as irregular, or the count value is reset when irregularity in the behavior of the moving object is recognized. Driver status detection method.

11. On the computer, The system recognizes the direction in which the driver of the moving object is being monitored and the behavior of the moving object. The system determines whether the recognized driver's monitoring direction is appropriate. The time during which the driver's monitoring direction is determined to be inappropriate is counted. If the counted value exceeds a threshold, an alarm message will be output. The count is stopped while the behavior of the moving object is recognized as irregular, or the count value is reset when irregularity in the behavior of the moving object is recognized. program.

Images