Vehicle control device, vehicle control method, and program

Abstract

The deceleration control should be appropriately released depending on the driver's condition. [Solution] The vehicle control device includes a determination unit that determines an abnormality in the driver based on the state of the vehicle driver, the behavior of the vehicle, or the output state of an abnormality signal indicating that an abnormality button indicating that an abnormality has occurred in the driver has been operated, and a control unit that performs deceleration control to decelerate the vehicle based on the vehicle state determined by the determination unit, wherein the control unit makes the conditions for releasing the deceleration control stricter when the degree of abnormality is large than when the degree of abnormality is small.

Description

Technical Field

[0001] The present invention relates to a vehicle control device, a vehicle control method, and a program.

Background Art

[0002] In recent years, efforts to provide a sustainable transportation system that takes various situations into consideration have been active. Toward this realization, research and development focusing on further improving traffic safety and convenience through research and development of driving support technologies have been carried out. For example, when a driving support ECU determines that a driver is in an abnormal state because a steering amount correlation value (such as steering torque and steering angle) that changes when a steering wheel SW of a vehicle is operated remains unchanged for a period longer than an abnormal determination threshold time, the driving support ECU decelerates the vehicle, flashes hazard lamps, and invalidates an acceleration request based on a change in an accelerator pedal operation amount (that is, prohibits an accelerator override). After it is determined that the driver is in an abnormal state, the driving support ECU is disclosed to cancel the deceleration of the vehicle and permit an accelerator override when it is determined that there has been a specific driving operation (an operation in which the accelerator pedal changes from an operated state to a non-operated state and then changes back to the operated state within a predetermined threshold time) (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional technology, deceleration may not be canceled according to the state of the driver.

[0005] This invention has been made in consideration of these circumstances, and one of its objectives is to provide a vehicle control device, a vehicle control method, and a program that can appropriately release deceleration control according to the driver's state. Ultimately, this will contribute to the development of a sustainable transportation system. [Means for solving the problem]

[0006] The control device, control method, and program according to this invention employ the following configuration. (1) A vehicle control device according to one aspect of the present invention includes a determination unit that determines an abnormality in the driver based on the state of the vehicle driver, the behavior of the vehicle, or the output state of an abnormality signal indicating that an abnormality button indicating that an abnormality has occurred in the driver has been operated, and a control unit that performs deceleration control to decelerate the vehicle based on the vehicle state determined by the determination unit, wherein the control unit makes the conditions for releasing the deceleration control stricter when the degree of abnormality is large than when the degree of abnormality is small.

[0007] (2) In the embodiment of (1) above, a large degree of abnormality means that the degree of abnormality of the driver's state is greater than in a small degree of abnormality, or that the abnormality of the driver's state continues for a long period of time.

[0008] (3) In the embodiment of (1) above, the control unit determines an abnormality in the driver based on a combination of two or three of the following: the state of the driver determined by analyzing the image captured by the driver, the state of the driver's control of the vehicle, and the dynamic state of the vehicle.

[0009] (4) In the embodiment of (1) above, the control unit releases the deceleration control when the first task is completed if the degree of abnormality is a first predetermined degree, and releases the deceleration control when the second task, which is easier to achieve than the first task, is completed if the degree of abnormality is a second predetermined degree which is less than the first predetermined degree.

[0010] (5) In the embodiment of (1) above, the control unit releases the deceleration control based on one or more of the following states: the state of the driver determined by analyzing the image captured by the driver, the state of the driver's control of the vehicle, the state of the driver's operations on the vehicle other than driving operations, and the state of the driver's operations on the operation unit in the information display device provided in the vehicle or the operation unit provided near the display device.

[0011] (6) In the embodiment of (5) above, the control unit determines the control state of the vehicle based on either or both of the amount and duration of operation of the driver's control element.

[0012] (7) In the embodiment of (5) above, the control unit determines the state of operations on the vehicle other than the driving operation based on the state of operation on the steering wheel or a switch provided near the steering wheel.

[0013] (8) In the embodiment of (5) above, the display device is a display device positioned between the driver's seat and the passenger seat of the vehicle with respect to the width direction of the vehicle.

[0014] (9) In the embodiment of (1) above, when the deceleration control is started, the control unit controls the steering of the vehicle so that the vehicle travels within the lane in which the vehicle is traveling.

[0015] (10): In the embodiment of (1) above, the control unit determines whether the driver is in a first abnormality level, a second abnormality level, or a third abnormality level, the first abnormality level being determined based on (a) an image captured of the driver, the second abnormality level being determined based on (a), (b) an operation of the driver's control element, or (c) the behavior of the vehicle, and the third abnormality level being determined by (a), (b), and (c), or (d) the output state of an abnormality signal indicating that an abnormality button indicating that an abnormality has occurred in the driver has been operated.

[0016] (11): In the embodiment of (10) above, the condition for releasing the deceleration control due to the first degree of abnormality is (e) operation of an operator by the driver, the condition for releasing the deceleration control due to the second degree of abnormality is determined in addition to (e) by (f) an image captured by the driver, and the condition for releasing the deceleration control due to the third degree of abnormality is (g) an operation task that is more difficult to achieve than the operation task in (e), or (h) an operation of a predetermined button provided on the vehicle.

[0017] (12): Another aspect of the present invention relates to a control method in which a computer determines a driver abnormality based on the state of the vehicle driver, the behavior of the vehicle, or the output state of an abnormality signal indicating that an abnormality button has been operated to indicate that an abnormality has occurred in the driver, and executes deceleration control to decelerate the vehicle based on the determined vehicle state, and makes the conditions for releasing the deceleration control stricter when the degree of abnormality is large than when the degree of abnormality is small.

[0018] (13): Another aspect of the present invention is a program which causes a computer to perform a process of determining a driver abnormality based on the state of the vehicle driver, the behavior of the vehicle, or the output state of an abnormality signal indicating that an abnormality button indicating that an abnormality has occurred in the driver has been operated, and to perform a process of deceleration control to decelerate the vehicle based on the determined vehicle state, wherein the conditions for releasing the deceleration control are stricter when the degree of abnormality is large than when the degree of abnormality is small. [Effects of the Invention]

[0019] According to the embodiments of (1)-(13), the vehicle control device can appropriately release deceleration control according to the driver's condition by making the conditions for releasing deceleration control stricter when the degree of abnormality is high than when the degree of abnormality is low. [Brief explanation of the drawing]

[0020] [Figure 1]It is a configuration diagram of a vehicle system 1 using a vehicle control system according to an embodiment. [Figure 2] It is a diagram showing an example of the content of reference information 182. [Figure 3] It is a flowchart showing an example of the flow of processing executed by the driving support device 100. [Figure 4] It is a flowchart showing another example of the flow of processing executed by the driving support device 100.

Mode for Carrying Out the Invention

[0021] [Overall Configuration] FIG. 1 is a configuration diagram of a vehicle system 1 using a vehicle control system according to an embodiment. The vehicle on which the vehicle system 1 is mounted is, for example, a two-wheeled, three-wheeled, four-wheeled or other vehicle, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination thereof. The electric motor operates using electric power generated by a generator connected to the internal combustion engine, or discharge power of a secondary battery or a fuel cell. Although the present embodiment will be described as being applied to a vehicle, it may be applied to other moving bodies instead of the vehicle.

[0022] The vehicle system 1 includes, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, a vehicle sensor 40, a driver monitor camera 42, a navigation device 50, an MPU 60, an operator 80, a driving support device (vehicle control device) 100, a traveling driving force output device 200, a brake device 210, a steering device 220, and an emergency notification SW (switch) 230. These devices and equipment are connected to each other by a multiplex communication line such as a CAN (Controller Area Network) communication line, a serial communication line, a wireless communication network, or the like. The configuration shown in FIG. 1 is merely an example, and a part of the configuration may be omitted, or another configuration may be added. The driving support device 100 is an example of a "control device".

[0023] Camera 10 is a digital camera that utilizes a solid-state image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor). Camera 10 is mounted at any location on the vehicle (hereinafter referred to as vehicle M) on which the vehicle system 1 is installed. When imaging the area in front, camera 10 is mounted on the top of the front windshield, behind the rearview mirror, etc. Camera 10 periodically and repeatedly images the area around vehicle M. Camera 10 may also be a stereo camera.

[0024] The radar device 12 emits radio waves such as millimeter waves around the vehicle M and detects radio waves reflected by objects (reflected waves) to determine at least the position (distance and bearing) of the object. The radar device 12 can be mounted at any location on the vehicle M. The radar device 12 may also detect the position and velocity of the object using the FM-CW (Frequency Modulated Continuous Wave) method.

[0025] LIDAR14 irradiates light (or electromagnetic waves with a wavelength close to light) around vehicle M and measures the scattered light. Based on the time from emission to reception, LIDAR14 detects the distance to the target. The irradiated light is, for example, pulsed laser light. LIDAR14 can be attached to any location on vehicle M.

[0026] The object recognition device 16 performs sensor fusion processing on the detection results from some or all of the camera 10, radar device 12, and LIDAR 14 to recognize the position, type, speed, etc., of an object. The object recognition device 16 outputs the recognition results to the driver assistance device 100. The object recognition device 16 may output the detection results from the camera 10, radar device 12, and LIDAR 14 directly to the driver assistance device 100. The object recognition device 16 may be omitted from the vehicle system 1.

[0027] The communication device 20 communicates with other vehicles in the vicinity of vehicle M, or with various server devices via a wireless base station, for example, by using a cellular network, Wi-Fi network, Bluetooth®, DSRC (Dedicated Short Range Communication), etc.

[0028] The HMI30 presents various information to the occupants of vehicle M and accepts input operations from the occupants. The HMI30 includes various display devices, speakers, buzzers, touch panels, switches, keys, etc. The HMI30 is equipped with a display device. The display device is a display device, also known as a multi-information display, that displays various information in vehicle M, such as a speedometer that shows the vehicle's speed or a tachometer that shows the rotational speed of the internal combustion engine in vehicle M.

[0029] The HMI 30 includes, for example, a display device 32 and operation buttons 34. The display device 32 is a display device that displays maps, vehicle information, etc., and is located on or near the center console between the driver's seat and the passenger seat.

[0030] The operation button 34 is, for example, a selector switch located on the spokes of the steering wheel. Operating the selector switch changes the display content on the multi-information display located in front of the driver's seat.

[0031] The vehicle sensor 40 includes a vehicle speed sensor for detecting the speed of the vehicle M, an acceleration sensor for detecting acceleration, a yaw rate sensor for detecting angular velocity around the vertical axis, and an orientation sensor for detecting the orientation of the vehicle M.

[0032] The driver monitoring camera 42 is a camera that captures images of the driver of vehicle M. The driver monitoring camera 42 is mounted in a position inside the vehicle M that captures images of the driver from the front.

[0033] The navigation device 50 includes, for example, a GNSS (Global Navigation Satellite System) receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 stores first map information 54 in a storage device such as an HDD (Hard Disk Drive) or flash memory. The GNSS receiver 51 determines the position of the vehicle M based on signals received from GNSS satellites. The position of the vehicle M may be determined or supplemented by an INS (Inertial Navigation System) that utilizes the output of the vehicle sensors 40. The navigation HMI 52 includes a display device, speakers, a touch panel, keys, etc. The navigation HMI 52 may be partially or completely shared with the HMI 30 described above. The route determination unit 53 determines, for example, a route (hereinafter referred to as the route on the map) from the position of the vehicle M determined by the GNSS receiver 51 (or any input position) to the destination input by the occupant using the navigation HMI 52, by referring to the first map information 54. The first map information 54 is, for example, information in which the road shape is represented by links indicating roads and nodes connected by those links. The first map information 54 may also include information such as road curvature and POI (Point of Interest) information. The route on the map is output to the MPU 60. The navigation device 50 may provide route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may be implemented, for example, by the functions of a terminal device such as a smartphone or tablet held by an occupant. The navigation device 50 may transmit the current location and destination to the navigation server via the communication device 20 and obtain a route equivalent to the route on the map from the navigation server.

[0034] The MPU 60 includes, for example, a recommended lane determination unit 61 and stores second map information 62 in a storage device such as an HDD or flash memory. The recommended lane determination unit 61 divides the map route provided by the navigation device 50 into multiple blocks (for example, every 100m with respect to the vehicle's direction of travel) and determines a recommended lane for each block by referring to the second map information 62. The recommended lane determination unit 61 makes decisions such as which lane from the left the vehicle should travel in. If there is a branching point on the map route, the recommended lane determination unit 61 determines a recommended lane so that the vehicle M can travel along a reasonable route to proceed to the branching point. For example, if the vehicle M reaches a predetermined distance before a branching road it is traveling on, the recommended lane determination unit 61 determines the lane connecting to the branching road as the recommended lane. The recommended lane determination unit 61 and the second map information 62 may be functional units or information included in other devices such as the driver assistance device 100. The driver assistance system 100 recommends to the driver that vehicle M move to a recommended lane, or automatically moves vehicle M.

[0035] The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 includes, for example, information on the center of lanes or information on lane boundaries. The second map information 62 may also include road information, traffic regulation information, address information (address and postal code), facility information, telephone number information, etc. The second map information 62 may be updated as needed by the communication device 20 communicating with other devices.

[0036] The control element 80 includes, for example, a steering wheel, as well as an accelerator pedal, brake pedal, shift lever, and other control elements. The control element 80 is equipped with a sensor that detects the amount of operation or whether or not an operation is performed, and the detection result is output to the driver assistance device 100, or to some or all of the driving force output device 200, brake device 210, and steering device 220. The steering wheel does not necessarily have to be annular in shape, and may take the form of an irregularly shaped steering wheel, a joystick, buttons, etc.

[0037] The driver assistance device 100 includes, for example, a recognition unit 110, a first state detection unit 120, a second state detection unit 130, a driver assistance unit 140, and a storage unit 180. The recognition unit 110, the first state detection unit 120, the second state detection unit 130, and the driver assistance unit 140 are realized, for example, by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), and SOC (System On Chip), or by the cooperation of software and hardware. The program may be stored in advance in a storage unit 180 (a storage device equipped with a non-transient storage medium) such as the HDD or flash memory of the driver assistance device 100, or it may be stored in a removable storage medium such as a DVD or CD-ROM and installed in the HDD or flash memory of the driver assistance device 100 when the storage medium (non-transient storage medium) is mounted on a drive device. For example, reference information 182, which will be described later, is stored in the storage unit 180. One or both of the first state detection unit 120 and the second state detection unit 130 are examples of "determination units". The driver assistance unit 140 is an example of a "control unit".

[0038] The recognition unit 110 recognizes the position and state of objects around the vehicle M, such as their speed and acceleration, based on information input from the camera 10, radar device 12, and LIDAR 14 via the object recognition device 16. The position of an object is recognized as a position on an absolute coordinate system with a representative point of the vehicle M (such as the center of gravity or the center of the drive axis) as the origin, and is used for control. The position of an object may be represented by a representative point such as the center of gravity or a corner of the object, or it may be represented by a region. The "state" of an object may include the object's acceleration, jerk, or "action state" (for example, whether or not it is changing lanes or attempting to change lanes).

[0039] The recognition unit 110 recognizes, for example, the lane in which the vehicle M is traveling. For example, the recognition unit 110 recognizes the driving lane by comparing the pattern of road markings (for example, an arrangement of solid and dashed lines) obtained from the second map information 62 with the pattern of road markings around the vehicle M recognized from the image captured by the camera 10. The recognition unit 110 may also recognize the driving lane by recognizing not only road markings, but also road boundaries (road boundaries) including road markings, shoulders, curbs, median strips, guardrails, etc. In this recognition, the position of the vehicle M obtained from the navigation device 50 and the processing results by INS may also be taken into consideration. The recognition unit 110 recognizes stop lines, obstacles, red lights, toll booths, and other road events.

[0040] When recognizing a driving lane, the recognition unit 110 recognizes the position and orientation of the vehicle M relative to the driving lane. For example, the recognition unit 110 may recognize the deviation of the vehicle M's reference point from the center of the lane, and the angle it makes with a line connecting the centers of the lanes in the direction of travel, as the relative position and orientation of the vehicle M relative to the driving lane. Alternatively, the recognition unit 110 may recognize the position of the vehicle M's reference point relative to any side edge of the driving lane (road marking or road boundary), etc., as the relative position of the vehicle M relative to the driving lane.

[0041] The first state detection unit 120 analyzes the image captured by the driver monitor camera 42 to determine whether the driver is in an abnormal state. For example, the first state detection unit 120 recognizes the driver's head, eyes, eyelids, etc. from the image and determines the driver's abnormality based on these conditions. For example, if the head is not facing forward but is tilted upward or downward, or in a direction different from the reference direction of forward, or if the gaze direction is different from the reference direction, or if the eyelids are closed, the first state detection unit 120 determines that the driver is in an abnormal state.

[0042] The first state detection unit 120 determines whether the driver is in a normal state. For example, the first state detection unit 120 analyzes the image as described above to determine whether the state of the driver's head, eyes, and eyelids is in a preset normal state.

[0043] The second state detection unit 130 detects the state of the vehicle M and the driver's operations on the control elements 80. The second state detection unit 130 detects the state of the vehicle M based on information acquired from the vehicle sensor 40, for example. The state of the vehicle M may include acceleration / deceleration or swaying. The driver's operations on the control elements may include operations on the accelerator pedal, brake pedal, steering wheel, the degree of these operations, and the duration of these operations.

[0044] The driver assistance unit 140 performs driver assistance control. For example, the driver assistance unit 140 automatically controls the driving force output device 200 and the brake device 210 without relying on the driver's operation to automatically control the speed of the vehicle M. The driver assistance unit 140 performs so-called ACC (Adaptive Cruise Control). The driver assistance unit 140 controls the vehicle M so that it travels at a set speed, or makes the vehicle M follow the vehicle in front at a predetermined distance from the vehicle in front.

[0045] The driver assistance unit 140 controls the steering device 220 to prevent the vehicle M from deviating from the driving lane. For example, the driver assistance unit 140 controls the steering device 220 so that the vehicle M travels in the center or near the center of the driving lane recognized by the recognition unit 110. Hereinafter, this control may be referred to as "lane keeping control". The control unit 150 performs hands-on lane keeping control and hands-off lane keeping control.

[0046] Hands-on lane keeping control is a control system that is performed when the driver is holding the steering wheel (when the steering grip sensor, not shown, detects that the driver is gripping the steering wheel). The conditions under which hands-on lane keeping control can be performed are less stringent than the conditions under which hands-off lane keeping control can be performed.

[0047] Hands-off lane keeping control is a control system that is performed when the driver is not holding the steering wheel (when the steering grip sensor, not shown, does not detect that the driver is holding the steering wheel). Hands-off lane keeping control can be performed, for example, when the following conditions are met: the speed of vehicle M is above a predetermined speed, vehicle M is traveling on a predetermined road (for example, a road or type of road that has been set in advance as being capable of performing hands-off lane keeping control), and the driver is monitoring the road ahead. Hands-off lane keeping control is performed when the driver is monitoring the road ahead, and is not performed or is stopped when the driver is not monitoring the road ahead.

[0048] The conditions under which hands-on lane keeping control and hands-off lane keeping control can be performed, as described above, are examples, and other conditions (for example, that vehicle M is following the vehicle in front) may be included, or some conditions may be omitted. The conditions under which hands-on lane keeping control can be performed are less stringent than the conditions under which hands-off lane keeping control can be performed (and the conditions under which hands-off lane keeping control can be performed are stricter than the conditions under which hands-on lane keeping control can be performed). The driver assistance device 100 recognizes whether the driver is monitoring the road ahead based on images captured by a camera (not shown) that captures images of the driver.

[0049] The driving force output device 200 outputs driving force (torque) to the drive wheels for the vehicle M to move. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, and a transmission, and an ECU that controls them. The ECU controls the above configuration according to information input from the driver assistance device 100 or from the operator 80.

[0050] The braking system 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the driver assistance device 100 or from the control element 80, so that brake torque corresponding to the braking operation is output to each wheel.

[0051] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor, for example, applies force to a rack and pinion mechanism to change the direction of the steering wheels. The steering ECU drives the electric motor to change the direction of the steering wheels according to information input from the driver assistance device 100 or from the control element 80.

[0052] The emergency call switch 230 is installed, for example, in a location within the vehicle M's interior that can be operated by the driver or occupant. The emergency call switch is installed, for example, on the ceiling between the driver's seat and the passenger seat inside the vehicle. The driver or occupant operates the emergency call switch if there is a problem with the driver or the vehicle M. The driver assistance device 100 controls the vehicle M so that it can stop safely based on the signal received in response to the operation.

[0053] [overview] The driver assistance system 100 controls vehicle M to ensure safe driving or stopping when an abnormality occurs or is suspected to occur with the driver. For example, the driver assistance system 100 determines the driver's condition based on the driver's state in vehicle M, the vehicle's behavior, or the vehicle state, which is the output state of an abnormality signal indicating that an abnormality button (e.g., emergency call SW230) has been operated. Based on the determined vehicle state, the driver assistance system 100 performs deceleration control to slow down vehicle M. When the degree of abnormality is high, the driver assistance system 100 makes the conditions for releasing the deceleration control stricter than when the degree of abnormality is low. This control will be described below.

[0054] A high degree of abnormality means that the driver's condition is more abnormal than in a low degree of abnormality, or that the driver's condition is abnormal for a longer period of time.

[0055] The driver assistance device 100 determines driver abnormalities based on any combination of two or three of the following: the driver's state determined by analyzing the images captured by the driver, the driver's control state of the vehicle M, and the dynamic state of the vehicle M. The control state of the vehicle M is, for example, the state of operation of the driver on the control elements (accelerator pedal, brake pedal, steering wheel, etc.). The dynamic state of the vehicle M is a change in the behavior of the vehicle M, such as swaying of the vehicle M or sudden changes in acceleration and deceleration of the vehicle M.

[0056] [Standard Information] The driver assistance device 100 determines driver abnormalities and deceleration control cancellation by referring to the reference information 182 shown in Figure 2. Figure 2 is a diagram showing an example of the contents of the reference information 182. The reference information 182 includes, for example, information indicating the degree of abnormality, the ease with which deceleration control can be canceled when an abnormality occurs, the criteria for determining the abnormality, and the criteria for determining the cancellation of deceleration control.

[0057] The degree of abnormality includes, for example, a first degree of abnormality, a second degree of abnormality, and a third degree of abnormality. The degree of abnormality increases in the order of first degree of abnormality, second degree of abnormality, and third degree of abnormality. A first degree of abnormality is, for example, a state in which the driver is mildly drowsy or has reduced responsiveness. A second degree of abnormality is, for example, a state in which the driver is a life-saving target or is asleep at the wheel.

[0058] The ease of release includes three degrees: first degree, second degree, and third degree. Release becomes increasingly difficult in the order of first degree, second degree, and third degree.

[0059] (Criteria for determining the first degree of abnormality) The criterion for determining the first degree of abnormality is that (1) is met. (1) The analysis results of the images captured by the driver monitor camera 42 indicate that a condition in which the driver is presumed to be abnormal has persisted for X1 seconds.

[0060] (Criteria for determining the second degree of abnormality) The criteria for determining the second degree of abnormality are that (2), (3), or (4) are met. (2) The analysis of the images captured by the driver monitor camera 42 indicates that a condition in which the driver is presumed to be abnormal has persisted for X2 seconds. X2 seconds is a longer time than X1 seconds. (3) The condition is that there is no acceleration or deceleration by the driver and no steering by the driver for Y seconds. (4) The state in which vehicle M is swaying satisfies the set criteria. Swaying means, for example, that vehicle M is traveling at a first position which is a predetermined distance or more in the width direction from a predetermined position such as the center of the road or lane, or that vehicle M is moving between different positions in the width direction of the road or lane within a predetermined time, or in any case that vehicle M is moving while continuously changing the above position in the width direction. The set criteria are met if the swaying state continues for a predetermined time, or if vehicle M moves to a position to the right in the width direction and then to a position to the left, and repeats this action a predetermined number of times (for example, three or four times).

[0061] (Criteria for determining the third degree of abnormality) The criteria for determining the third degree of abnormality are that (5), (3), and (4) are met, or (6) is met. (5) The analysis of the images captured by the driver monitor camera 42 indicates that a condition in which the driver is presumed to be abnormal has persisted for X3 seconds. X3 seconds is a longer time than X2 seconds. (6) The emergency call switch was activated.

[0062] An abnormality score may be used in the above determination. For example, the driver assistance device 100 derives a score according to the contents of (1) to (4) above and determines whether the score falls within the range of scores set for each degree of abnormality. The driver assistance device 100 may also determine the type of degree of abnormality according to the applicable range. For example, a score may be derived from the number of seconds in (1) to (3) or the information in (1) to (3) (driver's awareness, operating manner, degree of swaying), and these scores may be integrated to determine the degree of abnormality.

[0063] (Criteria for determining the removal of the first level of abnormality) The criterion for clearing the first level of abnormality is that (A) is met. (A) A first driving operation has been performed. The first driving operation is a minor driving operation, such as gripping the steering wheel, operating the accelerator pedal to a degree of 1 or greater, or operating the brake pedal to a degree of 2 or greater.

[0064] (Criteria for determining whether the second level of abnormality is resolved) The criteria for clearing the second level of abnormality are that (A) and (B) are met. (B) The analysis results of the images captured by the driver monitor camera 42 indicated that there was no abnormality in the driver, or that the state in which no abnormality was estimated continued for a predetermined time.

[0065] (Criteria for determining whether the third level of abnormality is resolved) The criteria for clearing the third level of abnormality are that (C), (D), or (E) are met. (C) The steering wheel has been operated and the accelerator pedal has been operated. The operation of the steering wheel is the operation of gripping the steering wheel or the operation of applying steering torque to the steering wheel to a predetermined degree or more. The operation of the accelerator pedal is, for example, the accelerator pedal being pressed to a predetermined degree a predetermined number of times (e.g., twice). In (C) above, the control state of the vehicle M is determined based on one or both of the amount and duration of operation of the driver's control devices. For example, the determination may be made based on one or more pieces of information from the operation of the steering wheel and its operation duration, and the operation of the accelerator pedal and its operation duration.

[0066] (D) A predetermined operation has been performed on the operation button 34. The predetermined operation may be a pre-defined operation or an operation specified in the display on the multi-information display. In other words, the driver assistance device 100 "determines the state of operations on the vehicle other than the driving operation based on the state of operation on the steering wheel or a switch provided near the steering wheel."

[0067] (E) A predetermined operation has been performed on the display device 32. The predetermined operation may be a pre-defined operation or an operation specified in the display of the display device 32. The display device 32 is an example of a "display device positioned between the driver's seat and the passenger seat of the vehicle in the width direction of the vehicle."

[0068] The third degree of abnormality is an example of the "first degree of abnormality," and the second degree of abnormality is an example of the "second degree of abnormality." The task for the conditions for releasing the third degree of abnormality is an example of the "first task" of the "criteria for releasing the first degree of abnormality." The task for the conditions for releasing the second degree of abnormality is an example of the "second task" of the "criteria for releasing the second degree of abnormality."

[0069] In the above example, in order to release the abnormality status, the driver assistance device 100 may release the deceleration control based on one or more of the following conditions, in addition to the driver's state determined by analyzing the image captured by the driver: (C) the driver's control status of the vehicle, (D) the driver's status of operations on the vehicle other than driving operations, and (E) the driver's status of operations on the operation unit in the information display device 32 provided in the vehicle M or the operation unit provided near the display device 32. In order to release the first or second abnormality status, the driver assistance device 100 may release the status based on one or more of the following information, in addition to the driver's state determined by analyzing the image: (C), (D), and (E); or in order to release the third abnormality status, (B) the analysis results of the image captured by the driver may be used.

[0070] For example, the conditions for releasing deceleration control should be progressively stricter in the following order: first, second, and third degrees of abnormality.

[0071] In the above determination, a release score may be used. For example, the driver assistance device 100 derives a score according to the contents of (A) to (E) above and determines whether the score falls within the range of scores set for each release condition of the degree of abnormality. The driver assistance device 100 determines whether the release condition for the target degree of abnormality is met. For example, a score is derived from the operation pattern in (A), the driver's state obtained from (B), the operation pattern in (C), and the operation patterns in (D) and (E), and these scores are integrated to make a determination.

[0072] Furthermore, although the above example was explained assuming there were three levels of abnormality, it is not limited to three levels; there could also be two or four levels. In this case as well, the more severe the abnormality, the stricter the conditions for releasing the deceleration control should be.

[0073] [Flowchart (1)] Figure 3 is a flowchart illustrating an example of the processing flow performed by the driver assistance system 100. This process determines the degree of driver abnormality. This flowchart assumes that ACC and lane keeping control are being performed.

[0074] First, the driver assistance device 100 determines whether or not an abnormality of the first degree of abnormality has occurred in the driver (step S100). If an abnormality of the first degree of abnormality has occurred, the driver assistance device 100 executes a first control (step S102). The first control is a control that stops the execution of ACC and executes lane keeping control. The vehicle M decelerates as a result of the first control.

[0075] Next, the driver assistance device 100 determines whether or not an abnormality of the second degree of abnormality has occurred in the driver (step S104). If an abnormality of the second degree of abnormality has occurred, the driver assistance device 100 executes the second control (step S106). The second control executes lane keeping control and, if the accelerator pedal is being operated, disables the operation. The vehicle M decelerates further due to the second control.

[0076] Next, the driver assistance device 100 determines whether or not an abnormality of the third degree of abnormality has occurred in the driver (step S108). If an abnormality of the third degree of abnormality has occurred, the driver assistance device 100 executes the third control (step S106). The third control is a control to bring the vehicle M to a stop, or a control to safely bring the vehicle M to a stop on the shoulder of the road. The third control further decelerates the vehicle M. This completes the processing of one routine in this flowchart.

[0077] As described above, the driver assistance device 100 can appropriately control the vehicle M according to the degree of driver abnormality.

[0078] [Flowchart (2)] Figure 4 is a flowchart showing another example of the processing flow performed by the driver assistance device 100. This process is for releasing control that is being performed due to a driver malfunction.

[0079] First, the driver assistance device 100 determines whether or not an abnormality has occurred in the driver (step S200). If an abnormality has occurred, the driver assistance device 100 sets a release condition according to the degree of the abnormality (step S202). Next, the driver assistance device 100 determines whether or not the release condition according to the degree of the abnormality is met (step S204). The release condition according to the degree of the abnormality is, for example, if the degree of abnormality is the first degree of abnormality, a release condition for the first degree of abnormality is set, and it is determined whether or not this release condition is met.

[0080] If the release conditions are met, the driver assistance device 100 releases the control corresponding to the degree of the abnormality being performed (step S206). For example, if the release conditions for the first degree of abnormality are met while the first control is being performed, the driver assistance device 100 releases the first control. This completes the processing of one routine in this flowchart.

[0081] As described above, the driver assistance device 100 can appropriately control the vehicle M by appropriately releasing the control according to the degree of abnormality of the driver when it is estimated that it is in a state where the control according to the degree of abnormality of the driver can be released. For example, by changing the release conditions according to the degree of abnormality, deceleration control can be easily released when the driver is in a normal state. Also, since the release conditions become stricter as the degree of abnormality increases, deceleration control is reliably continued so that the vehicle M is safely controlled in a state where an abnormality occurs in the driver and deceleration control should be performed. In this way, the driver assistance device 100 can appropriately release deceleration control according to the driver's state.

[0082] In the above embodiment, it was described that driver assistance for vehicle M is performed, but instead (or in addition to this), the above process may be performed when vehicle M is performing autonomous driving.

[0083] According to the embodiments described above, the driver assistance device 100 determines driver abnormalities based on the driver's state in the vehicle M, the behavior of the vehicle M, or the output state of an abnormality signal indicating that an abnormality button has been operated. Based on the determined vehicle state, it performs deceleration control to decelerate the vehicle M. When the degree of abnormality is high, the conditions for releasing the deceleration control are made stricter than when the degree of abnormality is low, thereby appropriately releasing the deceleration control according to the driver's state.

[0084] The embodiments described above can be expressed as follows. A memory device that stores the program, Equipped with a hardware processor, The hardware processor executes the program stored in the memory device, Based on the vehicle status, which is the state of the vehicle driver, the behavior of the vehicle, or the output state of an abnormal signal indicating that an abnormality button has been pressed to indicate that an abnormality has occurred in the driver, the driver abnormality is determined. Based on the determined vehicle state, deceleration control is performed to slow down the vehicle. When the degree of the abnormality is high, the conditions for releasing the deceleration control are made stricter than when the degree of the abnormality is low. A control device configured in such a way.

[0085] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention. [Explanation of Symbols]

[0086] 1. Vehicle System 10 Cameras 32 Display device 34 Operation buttons 42 Driver monitoring cameras 80 Operators 100 Driving support devices 110 Recognition part 120 First state detection unit 130 Second state detection unit 140 Driver Support Department

Claims

1. A determination unit determines a driver abnormality based on the vehicle status, which is the state of the vehicle driver, the behavior of the vehicle, or the output state of an abnormality signal indicating that an abnormality button indicating that an abnormality has occurred in the driver has been operated. The system includes a control unit that performs deceleration control to decelerate the vehicle based on the vehicle state determined by the determination unit, The control unit makes the conditions for releasing the deceleration control stricter when the degree of abnormality is high than when the degree of abnormality is low. Vehicle control device.

2. A "large degree of abnormality" refers to a situation where the degree of abnormality in the driver's state is greater than in a "small degree of abnormality," or where the abnormality in the driver's state persists for a longer period of time. The vehicle control device according to claim 1.

3. The control unit, The driver's state, determined by analyzing the captured image, The driver's control state of the vehicle, Based on the dynamic state of the vehicle and a combination of two or three of the states, the driver's abnormality is determined. The vehicle control device according to claim 1.

4. The control unit, When the degree of abnormality reaches a first predetermined degree, the deceleration control is released when the first task is completed. When the degree of abnormality is less than a first predetermined degree (a second predetermined degree), the deceleration control is released when the second task, which is easier to accomplish than the first task, is completed. The vehicle control device according to claim 1.

5. The control unit, In addition to the driver's state determined by analyzing the captured image, The driver's control state of the vehicle, The state of operations performed on the vehicle other than the driver's driving operations, Based on one or more of the following states, the deceleration control is released: The vehicle control device according to claim 1.

6. The control unit determines the control state of the vehicle based on either the amount of operation of the driver's control element, the operation time, or both. The vehicle control device according to claim 5.

7. The control unit determines the status of operations on the vehicle other than the driving operation based on the status of operations on the steering wheel or switches provided near the steering wheel. The vehicle control device according to claim 5.

8. The display device is a display device positioned between the driver's seat and the passenger seat of the vehicle in the width direction of the vehicle. The vehicle control device according to claim 5.

9. When the deceleration control is initiated, the control unit controls the steering of the vehicle so that the vehicle travels within the lane in which it is traveling. The vehicle control device according to claim 1.

10. The control unit determines whether the driver is in a first abnormality level, a second abnormality level, or a third abnormality level. The first degree of abnormality is determined (a) based on the image captured by the driver, The second degree of abnormality is determined based on (a), (b) the driver's operation of the control element, or (c) the behavior of the vehicle. The third degree of abnormality is determined by (a), (b), and (c), or (d) the output state of an abnormality signal indicating that an abnormality button indicating an abnormality in the driver has been operated. The vehicle control device according to claim 1.

11. The conditions for releasing the deceleration control based on the first degree of abnormality are (e) operation of the control element by the driver, The conditions for releasing the deceleration control based on the second degree of abnormality are determined based on (e) and (f) the image captured by the driver. The conditions for releasing the deceleration control due to the third degree of abnormality are an operation task that is more difficult to achieve than the operation task in (g)(e), or (h) an operation of a predetermined button provided on the vehicle. The vehicle control device according to claim 10.

12. Computers Based on the vehicle status, which is the state of the vehicle driver, the behavior of the vehicle, or the output state of an abnormal signal indicating that an abnormality button has been pressed to indicate that an abnormality has occurred in the driver, the driver abnormality is determined. Based on the determined vehicle state, deceleration control is performed to slow down the vehicle. When the degree of the abnormality is high, the conditions for releasing the deceleration control are made stricter than when the degree of the abnormality is low. Vehicle control method.

13. On the computer, A process for determining a driver abnormality based on the vehicle status, which is the state of the vehicle driver, the behavior of the vehicle, or the output state of an abnormality signal indicating that an abnormality button indicating that an abnormality has occurred in the driver has been operated. The process of performing deceleration control to slow down the vehicle based on the determined vehicle state is executed, When the degree of the abnormality is high, the conditions for releasing the deceleration control are made stricter than when the degree of the abnormality is low. program.

Images