Vehicle control device, vehicle control method, and program

Abstract

To provide a vehicle control device, a vehicle control method, and a program capable of performing the lane change more comfortable for a driver.SOLUTION: A vehicle control device according to an embodiment includes: a detection part which detects a part or all of a degree of arousal of a driver of a vehicle, a direction of a face of the driver and a direction of a line of sight of the driver; and an operation control part for causing the vehicle to change the lane by controlling at least one of the speed or steering of the vehicle. The operation control part changes an aspect of lane change based on a detection result by the detection part.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a vehicle control device, a vehicle control method, and a program. [Background technology]

[0002] There is known a technique for changing lanes while decreasing the maximum lateral acceleration or maximum lateral speed depending on the width of the lane to which the vehicle is to change lanes (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-100534 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technology, lane changing may occur even when the driver is not facing forward, which may cause the driver to become car sick. Furthermore, in the conventional technology, lane changing while taking into account the driver's level of alertness has not been considered.

[0005] One aspect of the present invention has been made in consideration of the above circumstances, and one of its objectives is to provide a vehicle control device, a vehicle control method, and a program that enable the driver to change lanes more comfortably. [Means for solving the problem]

[0006] The vehicle control device, the vehicle control method, and the program according to the present invention employ the following configuration.

[0007] One aspect (1) of the present invention is a vehicle control device that includes a detection unit that detects some or all of the level of alertness of a driver of a vehicle, the direction of the driver's face, and the direction of the driver's gaze, and a driving control unit that causes the vehicle to change lanes by controlling at least one of the speed or steering of the vehicle, and the driving control unit changes the manner of the lane change based on the detection result by the detection unit.

[0008] In an aspect (2), in the vehicle control device of the aspect (1) above, the driving control unit determines, based on the detection result by the detection unit, whether or not the driver has faced the front of the vehicle within a period from the time when the lane change becomes possible until a predetermined time has elapsed, or from the time when the lane change becomes possible until the vehicle has traveled a predetermined distance, and if it is determined that the driver has faced the front of the vehicle within the period, performs the lane change.

[0009] In an aspect (3), in the vehicle control device of the aspect (2) above, when the driving control unit determines that the driver is not facing forward of the vehicle within the period, the driving control unit uses the output unit to output information urging the driver to face forward of the vehicle.

[0010] An aspect (4) is a vehicle control device according to any one of the aspects (1) to (3) above, wherein the driving control unit determines, based on a detection result by the detection unit, whether or not the driver faces the front of the vehicle within a first period from when the lane change becomes possible until a first predetermined time has elapsed, or from when the lane change becomes possible until the vehicle travels a first predetermined distance, and if it is determined that the driver faces the front of the vehicle within the first period, performs the lane change; if it is determined that the driver does not face the front of the vehicle within the first period, activates a turn signal; and, based on the detection result by the detection unit, determines whether or not the driver faces the front of the vehicle within a second period from the first period until a second predetermined time has elapsed, or from the first period until the vehicle travels a second predetermined distance, and if it is determined that the driver does not face the front of the vehicle within the second period, uses an output unit to output information urging the driver to face the front of the vehicle.

[0011] In an embodiment (5), in a vehicle control device of any one of the embodiments (1) to (4) above, the driving control unit restricts the lane change when the level of alertness detected by the detection unit is below a threshold value.

[0012] Another aspect (6) of the present invention is a vehicle control method in which a computer mounted on a vehicle detects some or all of the level of alertness of a driver of the vehicle, the direction of the driver's face, and the direction of the driver's line of sight, controls at least one of the speed or steering of the vehicle to cause the vehicle to change lanes, and changes the manner of the lane change based on the detection results.

[0013] Another aspect (7) of the present invention is a program for causing a computer mounted on a vehicle to detect some or all of the level of alertness of a driver of the vehicle, the direction of the driver's face, and the direction of the driver's gaze, cause the vehicle to change lanes by controlling at least one of the speed or steering of the vehicle, and change the manner of the lane change based on the results of the detection. Effect of the Invention

[0014] According to any of the above aspects, lane changing can be performed more comfortably for the driver. [Brief description of the drawings]

[0015] [Figure 1] 1 is a configuration diagram of a vehicle system 1 that uses a vehicle control device according to an embodiment. [Diagram 2] 2 is a functional configuration diagram of a first control unit 120 and a second control unit 160. FIG. [Diagram 3] 1 is a diagram showing an example of a correspondence relationship between a driving mode, a control state of a host vehicle M, and a task. [Figure 4] 4 is a flowchart showing an example of a flow of a series of processes performed by the automatic driving control device 100 of the embodiment. [Diagram 5] 10 is a flowchart showing an example of a series of processing steps based on a driver's alertness. [Figure 6] 1 is a diagram illustrating an example of a hardware configuration of an automatic driving control device 100 according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of a vehicle control device, a vehicle control method, and a program according to the present invention will be described with reference to the drawings.

[0017] [Overall configuration] 1 is a configuration diagram of a vehicle system 1 that utilizes a vehicle control device according to an embodiment. The vehicle (hereinafter, the subject vehicle M) on which the vehicle system 1 is mounted is, for example, a two-wheeled, three-wheeled, or four-wheeled vehicle, and its drive source is an internal combustion engine such as a diesel engine or a gasoline engine, an electric motor, or a combination of these. The electric motor operates using power generated by a generator connected to the internal combustion engine, or discharged power from a secondary battery or a fuel cell.

[0018] The vehicle system 1 includes, for example, a camera 10, a radar device 12, a LIDAR (Light Detection and Ranging) 14, an object recognition device 16, a communication device 20, an HMI (Human Machine Interface) 30, a vehicle sensor 40, a navigation device 50, an MPU (Map Positioning Unit) 60, a driver monitor camera 70, a driving operator 80, a direction indicator 90, an automatic driving control device 100, a driving force output device 200, a brake device 210, and a steering device 220. These devices and devices are connected to each other by a multiple communication line such as a CAN (Controller Area Network) communication line, a serial communication line, a wireless communication network, or the like. Note that the configuration shown in FIG. 1 is merely an example, and a part of the configuration may be omitted, or another configuration may be added. The automatic driving control device 100 is an example of a "vehicle control device".

[0019] The camera 10 is a digital camera using a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The camera 10 is attached to an arbitrary location of the vehicle in which the vehicle system 1 is mounted. For example, when imaging the area in front of the vehicle M, the camera 10 is attached to the upper part of the front windshield or the back side of the room mirror. When imaging the area behind the vehicle M, the camera 10 is attached to the upper part of the rear windshield. When imaging the area to the right or left of the vehicle M, the camera 10 is attached to the right or left side of the vehicle body or a door mirror. The camera 10 periodically and repeatedly captures images of the surroundings of the vehicle M. The camera 10 may be a stereo camera.

[0020] The radar device 12 emits radio waves such as millimeter waves around the vehicle M and detects radio waves reflected by an object (reflected waves) to detect at least the position (distance and direction) of the object. The radar device 12 is attached to an arbitrary location of the vehicle M. The radar device 12 may detect the position and speed of an object by an FM-CW (Frequency Modulated Continuous Wave) method.

[0021] The LIDAR 14 irradiates light (or electromagnetic waves with a wavelength close to that of light) around the vehicle M and measures the scattered light. The LIDAR 14 detects the distance to the target based on the time between light emission and light reception. The irradiated light is, for example, a pulsed laser light. The LIDAR 14 is attached to an arbitrary location of the vehicle M.

[0022] The object recognition device 16 performs sensor fusion processing on some or all of the detection results from the camera 10, the radar device 12, and the LIDAR 14 to recognize the position, type, speed, etc. of the object. The object recognition device 16 outputs the recognition result to the automatic driving control device 100. The object recognition device 16 may output the detection results from the camera 10, the radar device 12, and the LIDAR 14 directly to the automatic driving control device 100. The object recognition device 16 may be omitted from the vehicle system 1.

[0023] The communication device 20 communicates with other vehicles in the vicinity of the vehicle M, for example, using a cellular network, a Wi-Fi network, Bluetooth (registered trademark), DSRC (Dedicated Short Range Communication), etc., or communicates with various server devices via a wireless base station.

[0024] The HMI 30 presents various information to the occupants (including the driver) of the vehicle M and accepts input operations by the occupants. For example, the HMI 30 may include a display device, a switch, a speaker, a buzzer, a touch panel, and the like. For example, the occupant inputs the destination of the vehicle M to the HMI 30. The HMI 30 is an example of an "output unit."

[0025] The vehicle sensor 40 includes a vehicle speed sensor that detects the speed of the host vehicle M, an acceleration sensor that detects the acceleration, a gyro sensor that detects the angular velocity, and a direction sensor that detects the orientation of the host vehicle M. The gyro sensor may include, for example, a yaw rate sensor that detects the angular velocity around a vertical axis.

[0026] The navigation device 50 includes, for example, a Global Navigation Satellite System (GNSS) receiver 51, a navigation HMI 52, and a route determination unit 53. The navigation device 50 holds first map information 54 in a storage device such as a hard disk drive (HDD) or a flash memory.

[0027] The GNSS receiver 51 receives radio waves from each of a plurality of GNSS satellites (artificial satellites) and identifies the position of the vehicle M based on the received radio wave signals. The GNSS receiver 51 outputs the identified position of the vehicle M to the route determination unit 53, or outputs the position to the automatic driving control device 100 directly or indirectly via the MPU 60. The position of the vehicle M may be identified or complemented by an INS (Inertial Navigation System) that utilizes the output of the vehicle sensor 40.

[0028] The navigation HMI 52 includes a display device, a speaker, a touch panel, keys, etc. The navigation HMI 52 may be partially or entirely common to the above-mentioned HMI 30. For example, instead of or in addition to inputting the destination of the host vehicle M to the HMI 30, the occupant may input the destination of the host vehicle M to the navigation HMI 52.

[0029] The route determination unit 53 determines, for example, a route (hereinafter, a route on a map) from the position of the vehicle M identified by the GNSS receiver 51 (or any input position) to a destination input by the occupant using the HM 30 or the navigation HMI 52, by referring to the first map information 54.

[0030] The first map information 54 is, for example, information that expresses road shapes using links indicating roads and nodes connected by the links. The first map information 54 may also include road curvatures and POI (Point Of Interest) information. The route on the map is output to the MPU 60.

[0031] The navigation device 50 may provide route guidance using the navigation HMI 52 based on the route on the map. The navigation device 50 may be realized by, for example, a function of a terminal device such as a smartphone or tablet terminal owned by a passenger. The navigation device 50 may transmit a current position and a destination to a navigation server via the communication device 20, and obtain a route equivalent to the route on the map from the navigation server.

[0032] The MPU 60 includes, for example, a recommended lane determination unit 61, and holds second map information 62 in a storage device such as an HDD or a flash memory. The recommended lane determination unit 61 is realized by a hardware processor such as a CPU (Central Processing Unit) executing a program (software). The recommended lane determination unit 61 may be realized by hardware (including a circuit unit; circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by cooperation between software and hardware. The program may be stored in advance in a storage device (a storage device having a non-transient storage medium) of the MPU 60, or may be stored in a removable storage medium such as a DVD or CD-ROM, and installed in the storage device of the MPU 60 by mounting the storage medium (non-transient storage medium) in a drive device.

[0033] The recommended lane determination unit 61 divides the route on the map provided by the navigation device 50 into a number of blocks (for example, every 100 m in the vehicle travel direction), and determines a recommended lane for each block by referring to the second map information 62. The recommended lane determination unit 61 determines which lane from the left the vehicle should travel in. When a branch point is present on the route on the map, the recommended lane determination unit 61 determines a recommended lane so that the vehicle M can travel along a reasonable route to proceed to the branch point.

[0034] The second map information 62 is map information with higher accuracy than the first map information 54. The second map information 62 includes, for example, information on the center of lanes or information on boundaries of lanes. The second map information 62 may also include road information, traffic regulation information, address information (address and zip code), facility information, telephone number information, and the like. The second map information 62 may be updated as needed by the communication device 20 communicating with another device.

[0035] The driver monitor camera 70 is, for example, a digital camera using a solid-state imaging element such as a CCD or a CMOS. The driver monitor camera 70 is attached to an arbitrary location of the host vehicle M in a position and orientation that allows an image of an occupant (i.e., a driver) seated in the driver's seat of the host vehicle M to be captured from the front. For example, the driver monitor camera 70 is attached on the instrument panel of the host vehicle M.

[0036] The driving operators 80 include, for example, a steering wheel 82, an accelerator pedal, a brake pedal, a shift lever, and other operators. Sensors that detect the amount of operation or the presence or absence of operation are attached to the driving operators 80. The detection results of the sensors are output to the automatic driving control device 100, or are output to some or all of the driving force output device 200, the brake device 210, and the steering device 220.

[0037] The steering wheel 82 does not necessarily have to be annular, and may be in the form of an irregular steering wheel, a joystick, buttons, etc. A steering grip sensor 84 is attached to the steering wheel 82. The steering grip sensor 84 is a capacitance sensor, etc. The steering grip sensor 84 detects whether the driver is gripping the steering wheel 82 (meaning that the driver is in contact with the steering wheel in a state in which force can be applied), and outputs a signal indicating the detection result to the automatic driving control device 100.

[0038] The direction indicator (also called a blinker or turn lamp) 90 is a lamp that indicates to the surroundings the direction of turning right or left or the direction of a lane change. The direction indicator 90 is equipped with a direction indicator lever 92. The direction indicator lever 92 may be attached, for example, near the steering wheel 82. When the occupant operates the direction indicator lever 92, the direction indicator 90 is activated. "Activation" refers to the action of turning on or blinking the lamp (turn lamp) that functions as the direction indicator 90.

[0039] The automatic driving control device 100 includes, for example, a first control unit 120, a second control unit 160, and a storage unit 180. The first control unit 120 and the second control unit 160 are each realized by, for example, a hardware processor such as a CPU executing a program (software). In addition, some or all of these components may be realized by hardware (including circuitry) such as an LSI, ASIC, FPGA, or GPU, or may be realized by cooperation between software and hardware. The program may be stored in advance in a storage device (storage device having a non-transient storage medium) such as an HDD or flash memory of the automatic driving control device 100, or may be stored in a removable storage medium such as a DVD or CD-ROM, and installed in the HDD or flash memory of the automatic driving control device 100 by mounting the storage medium (non-transient storage medium) in a drive device.

[0040] The storage unit 180 is realized by, for example, a HDD, a flash memory, an EEPROM, a ROM, or a RAM. The storage unit 180 stores, for example, a program that is read and executed by a processor.

[0041] 2 is a functional configuration diagram of the first control unit 120 and the second control unit 160. The first control unit 120 includes, for example, a recognition unit 130, a behavior plan generation unit 140, and a mode control unit 150. A combination of the behavior plan generation unit 140 and the second control unit 160, or a combination of the behavior plan generation unit 140, the mode control unit 150, and the second control unit 160 is an example of a "driving control unit."

[0042] The first control unit 120, for example, realizes a function based on AI (Artificial Intelligence) and a function based on a pre-given model in parallel. For example, the function of "recognizing an intersection" may be realized by executing in parallel the recognition of an intersection by deep learning or the like and the recognition based on pre-given conditions (such as traffic lights and road markings that can be pattern matched), and by scoring both and evaluating them comprehensively. This ensures the reliability of the autonomous driving.

[0043] The recognition unit 130 recognizes the situation or environment around the vehicle M. For example, the recognition unit 130 recognizes objects present around the vehicle M based on information input from the camera 10, the radar device 12, and the LIDAR 14 via the object recognition device 16. The objects recognized by the recognition unit 130 include, for example, bicycles, motorcycles, four-wheeled vehicles, pedestrians, road signs, road markings, dividing lines, utility poles, guardrails, fallen objects, and the like. The recognition unit 130 also recognizes the position, speed, acceleration, and other states of the objects. The position of the object is recognized as a position on a relative coordinate system (i.e., a relative position with respect to the vehicle M) with a representative point of the vehicle M (such as the center of gravity or the center of the drive shaft) as the origin, and is used for control. The position of the object may be represented by a representative point such as the center of gravity or a corner of the object, or may be represented by a represented area. The "state" of the object may include the acceleration or jerk of the object, or the "action state" (for example, whether or not the object is changing lanes or is about to change lanes).

[0044] Furthermore, the recognition unit 130 recognizes, for example, the lane in which the host vehicle M is traveling (hereinafter, the host vehicle lane) and adjacent lanes adjacent to the host vehicle lane. For example, the recognition unit 130 acquires the second map information 62 from the MPU 60, and compares the pattern of road dividing lines (e.g., an arrangement of solid lines and dashed lines) included in the acquired second map information 62 with the pattern of road dividing lines around the host vehicle M recognized from the image from the camera 10, thereby recognizing the space between the dividing lines as the host vehicle lane and adjacent lanes.

[0045] The recognition unit 130 may recognize lanes such as the vehicle's own lane and adjacent lanes by recognizing road boundaries (road boundaries) including not only road dividing lines but also road shoulders, curbs, medians, guardrails, etc. In this recognition, the position of the vehicle M acquired from the navigation device 50 and the processing results by the INS may be taken into consideration. In addition, the recognition unit 130 may recognize stop lines, obstacles, red lights, toll booths, and other road phenomena.

[0046] Furthermore, when recognizing the own lane, the recognition unit 130 recognizes the relative position and attitude of the own vehicle M with respect to the own lane. For example, the recognition unit 130 may recognize the deviation of the reference point of the own vehicle M from the center of the lane and the angle with respect to a line connecting the coordinate points of the lane center in the traveling direction of the own vehicle M as the relative position and attitude of the own vehicle M with respect to the own lane. Alternatively, the recognition unit 130 may recognize the position of the reference point of the own vehicle M with respect to either side end of the own lane (a road dividing line or a road boundary) as the relative position of the own vehicle M with respect to the own lane.

[0047] The action plan generation unit 140 generates a future target trajectory for the host vehicle M to travel, in principle, along the recommended lane determined by the recommended lane determination unit 61, and further, to automatically (without relying on the driver's operation) in a driving state defined by an event described below so as to be able to respond to the surrounding conditions of the host vehicle M.

[0048] The target trajectory includes, for example, a speed element. For example, the target trajectory is expressed as a sequence of points (trajectory points) to be reached by the host vehicle M. The trajectory points are points to be reached by the host vehicle M at each predetermined travel distance (for example, about several meters) along a road, and separately, a target speed and a target acceleration are generated as part of the target trajectory for each predetermined sampling time (for example, about a few tenths of a second). The trajectory points may also be positions to be reached by the host vehicle M at each predetermined sampling time. In this case, the information on the target speed and target acceleration is expressed as the interval between the trajectory points.

[0049] In order to respond to the surrounding conditions of the host vehicle M, the action plan generating unit 140 may generate a target trajectory such that the host vehicle M exceptionally travels on lanes other than the recommended lane (for example, lanes adjacent to the recommended lane). In other words, the priority of lanes other than the recommended lane is relatively lower than the priority of the recommended lane. For example, the priority of the recommended lane is the highest (priority 1), the priority of lanes adjacent to the recommended lane (hereinafter, adjacent lanes) is the second highest (priority 2), and the priority of lanes adjacent to the adjacent lanes is the third highest (priority 3). In this way, the action plan generating unit 140 generates a target trajectory such that the host vehicle M travels on the recommended lane with the highest priority in principle, while generating a target trajectory such that the host vehicle M exceptionally travels on lanes with a lower priority than the recommended lane depending on the surrounding conditions of the host vehicle M.

[0050] When generating the target trajectory, the action plan generating unit 140 determines an event of automatic driving (including some driving assistance) on the route for which the recommended lane has been determined. An event of automatic driving is information that specifies the behavior of the host vehicle M under automatic driving (some driving assistance), that is, the state during driving (or the manner during driving).

[0051] Autonomous driving events include, for example, a constant speed driving event, a low-speed following driving event, a lane change event, and an overtaking event. The constant speed driving event is an event in which the host vehicle M is driven in the same lane at a constant speed. The low-speed following driving event is an event in which the host vehicle M is driven to follow another vehicle (hereinafter referred to as a leading vehicle) that is present within a predetermined distance (for example, within 100 [m]) ahead of the host vehicle M and is closest to the host vehicle M. "Following" may be, for example, a driving state in which the relative distance (inter-vehicle distance) between the host vehicle M and the leading vehicle is maintained constant, or a driving state in which the host vehicle M is driven in the center of the host vehicle's lane in addition to maintaining the relative distance between the host vehicle M and the leading vehicle constant. The lane change event is an event in which the host vehicle M is driven to change lanes from the host vehicle's own lane to an adjacent lane. An overtaking event is an event in which the host vehicle M first changes lanes to an adjacent lane, passes a vehicle ahead in the adjacent lane, and then changes lanes back to the original lane.

[0052] Furthermore, the autonomous driving events include branching events, merging events, lane reduction events, takeover events, and the like. A branching event is an event in which the host vehicle M is traveling on a main line and the destination is on an extension of a branch line (hereinafter, a branch lane) that branches off from the main line, and the host vehicle M is guided to change lanes from the main line to a branch lane at a branch point. A merging event is an event in which the host vehicle M is traveling on a branch line (hereinafter, a merging lane) that merges into the main line, and the destination is on an extension of the main line, and the host vehicle M is guided to change lanes from the merging lane to the main line at a merging point. A lane reduction event is an event in which the host vehicle M is made to change lanes to another lane when traveling on a route in which the number of lanes is reduced midway. A takeover event is an event in which the autonomous driving mode (Mode A described later) is terminated and switched to a driving assistance mode (Modes B, C, and D described later) or a manual driving mode (Mode E described later). For example, there are cases where the dividing line is broken just before the toll gate on the expressway, and it is not possible to recognize the relative position of the host vehicle M. In such a case, a takeover event is determined (planned) for the section just before the toll gate.

[0053] The action plan generation unit 140 sequentially determines these multiple events on the route to the destination, and generates a target trajectory for driving the vehicle M in the conditions specified by each event, taking into account the surrounding conditions of the vehicle M.

[0054] The mode control unit 150 determines the driving mode of the vehicle M to be one of a plurality of driving modes. The driving modes each have a different task assigned to the driver. The mode control unit 150 includes, for example, a driver state determination unit 152, a mode determination unit 154, and an equipment control unit 156. The individual functions of these units will be described later. The combination of the driver monitor camera 70 and the driver state determination unit 152 is an example of a "detection unit."

[0055] FIG. 3 is a diagram showing an example of the correspondence between the driving modes, the control state of the vehicle M, and the tasks. The driving modes of the vehicle M include, for example, five modes, from mode A to mode E. The control state, i.e., the degree of automation (control level) of the driving control of the vehicle M, is highest in mode A, followed by mode B, mode C, mode D, and the lowest in mode E. Conversely, the tasks imposed on the driver are the lightest in mode A, followed by mode B, mode C, mode D, and the most severe in mode E. Note that in modes D and E, the control state is not automatic driving, so the automatic driving control device 100 has a responsibility to end the control related to automatic driving and to transition to driving assistance or manual driving. Below, the contents of each driving mode are illustrated.

[0056] In mode A, the vehicle is in an automatic driving state, and the driver is not required to monitor the front or hold the steering wheel 82 (holding the steering wheel in the figure). However, even in mode A, the driver is required to be in a position where he or she can quickly switch to manual driving in response to a request from a system centered on the automatic driving control device 100. Note that automatic driving here means that both steering and acceleration / deceleration are controlled without the driver's operation. The front means the space in the traveling direction of the host vehicle M that is visible through the front windshield. Mode A is a driving mode that can be executed when conditions are met, such as the host vehicle M traveling at a predetermined speed (for example, about 50 [km / h]) or less on a motor vehicle-only road such as an expressway, and there is a vehicle ahead that is to be followed, and is sometimes called TJP (Traffic Jam Pilot). If this condition is no longer met, the mode control unit 150 changes the driving mode of the host vehicle M to mode B.

[0057] In mode B, the vehicle is in a driving assistance state, and the driver is tasked with monitoring the front of the vehicle M (hereinafter, forward monitoring), but is not tasked with holding the steering wheel 82. In mode C, the vehicle is in a driving assistance state, and the driver is tasked with the tasks of monitoring the front and holding the steering wheel 82. Mode D is a driving mode that requires some degree of driving operation by the driver for at least one of steering and acceleration / deceleration of the vehicle M. For example, in mode D, driving assistance such as ACC (Adaptive Cruise Control) and LKAS (Lane Keeping Assist System) is performed. In mode E, the vehicle is in a manual driving state in which both steering and acceleration / deceleration are required by the driver. In both mode D and mode E, the driver is naturally tasked with the task of monitoring the front of the vehicle M.

[0058] The automatic driving control device 100 (and the driving assistance device (not shown)) executes an automated lane change according to the driving mode. There are two types of automated lane changes: automated lane change at the system request (1) and automated lane change at the driver request (2). Automated lane change (1) includes an automated lane change for overtaking, which is performed when the speed of the vehicle ahead is slower than a reference value compared to the speed of the vehicle itself, and an automated lane change for proceeding toward the destination (an automated lane change due to a change in the recommended lane). Automated lane change (2) is a lane change in which, when conditions related to the speed, the positional relationship with surrounding vehicles, etc. are met and the driver operates the turn signal, the vehicle M is caused to change lanes in the direction of the operation.

[0059] In mode A, the automatic driving control device 100 does not execute either automated lane change (1) or (2). In modes B and C, the automatic driving control device 100 executes either automated lane change (1) or (2). In mode D, the driving assistance device (not shown) does not execute automated lane change (1), but executes automated lane change (2). In mode E, neither automated lane change (1) nor (2) is executed.

[0060] Returning to the description of Fig. 2, when the task related to the determined driving mode is not executed by the driver, the mode control unit 150 changes the driving mode of the host vehicle M to a driving mode with a heavier task.

[0061] For example, in mode A, if the driver is in a position where he / she cannot switch to manual driving in response to a request from the system (for example, if the driver continues to look away from the road outside the permitted area or if a sign of driving difficulty is detected), the mode control unit 150 uses the HMI 30 to prompt the driver to switch to manual driving, and if the driver does not comply, the host vehicle M is pulled over to the roadside and gradually stopped, and the automatic driving is stopped. After the automatic driving is stopped, the host vehicle becomes in a state of mode D or E, and the host vehicle M can be started by the driver's manual operation. The same applies to "stopping automatic driving" below. In mode B, if the driver is not monitoring the road ahead, the mode control unit 150 uses the HMI 30 to prompt the driver to monitor the road ahead, and if the driver does not comply, the host vehicle M is pulled over to the roadside and gradually stopped, and the automatic driving is stopped. In mode C, if the driver is not monitoring the road ahead or is not gripping the steering wheel 82, the mode control unit 150 uses the HMI 30 to prompt the driver to monitor the road ahead and / or grip the steering wheel 82, and if the driver does not comply, the mode control unit 150 controls the vehicle M to move to the shoulder of the road and gradually stop it, thereby stopping automatic driving.

[0062] In order to change the mode, the driver state determination unit 152 determines whether the driver is in a state where he or she can handle the task based on the image from the driver monitor camera 70 and the detection signal from the steering grip sensor 84.

[0063] For example, the driver state determination unit 152 analyzes the image from the driver monitor camera 70 to estimate the driver's posture, and based on the estimated posture, determines whether the driver is in a position that allows him or her to switch to manual driving in response to a request from the system.

[0064] The driver state determination unit 152 analyzes the image from the driver monitor camera 70 to estimate the driver's line of sight or facial direction, and determines whether the driver is monitoring the area ahead of the vehicle M based on the estimated line of sight or facial direction.

[0065] For example, the driver state determination unit 152 detects the positional relationship between the driver's head and eyes, the combination of the reference point and the moving point in the eyes, and the like from the image of the driver monitor camera 70 using a method such as template matching. Then, the driver state determination unit 152 estimates the direction of the face based on the relative position of the eyes with respect to the head. Also, the driver state determination unit 152 estimates the direction of the driver's line of sight based on the position of the moving point with respect to the reference point. For example, if the reference point is the inner corner of the eye, the moving point is the iris. Also, if the reference point is the corneal reflection area, the moving point is the pupil.

[0066] The driver state determination unit 152 determines the wakefulness of the driver by analyzing the image of the driver monitor camera 70. The driver state determination unit 152 determines whether the driver is gripping the steering wheel 82 or not based on the detection signal of the steering grip sensor 84.

[0067] The mode determination unit 154 determines the driving mode of the host vehicle M depending on the determination result of the driver state determination unit 152.

[0068] The device control unit 156 controls the HMI 30, the turn indicator 90, and the like, based on the driving mode of the vehicle M determined by the mode determination unit 154 and the determination result by the driver state determination unit 152. For example, the device control unit 156 may cause the HMI 30 to output information that prompts the driver to perform a task corresponding to each driving mode.

[0069] The second control unit 160 controls the traveling driving force output device 200, the braking device 210, and the steering device 220 so that the host vehicle M passes the target trajectory generated by the action plan generating unit 140 at the scheduled time.

[0070] The second control unit 160 includes, for example, an acquisition unit 162, a speed control unit 164, and a steering control unit 166. The acquisition unit 162 acquires information on the target trajectory (trajectory points) generated by the action plan generation unit 140, and stores it in a memory (not shown). The speed control unit 164 controls the traveling driving force output device 200 or the brake device 210 based on a speed element associated with the target trajectory stored in the memory. The steering control unit 166 controls the steering device 220 according to the curvature of the target trajectory stored in the memory. The processing of the speed control unit 164 and the steering control unit 166 is realized, for example, by a combination of feedforward control and feedback control. As an example, the steering control unit 166 executes a combination of feedforward control according to the curvature of the road ahead of the host vehicle M and feedback control based on the deviation from the target trajectory.

[0071] The driving force output device 200 outputs a driving force (torque) for the vehicle to travel to the driving wheels. The driving force output device 200 includes, for example, a combination of an internal combustion engine, an electric motor, a transmission, and an ECU (Electronic Control Unit) that controls these. The ECU controls the above components according to information input from the second control unit 160 or information input from the driving operation device 80.

[0072] The brake device 210 includes, for example, a brake caliper, a cylinder that transmits hydraulic pressure to the brake caliper, an electric motor that generates hydraulic pressure in the cylinder, and a brake ECU. The brake ECU controls the electric motor according to information input from the second control unit 160 or information input from the driving operation element 80, so that a brake torque corresponding to a braking operation is output to each wheel. The brake device 210 may include a backup mechanism that transmits hydraulic pressure generated by operation of a brake pedal included in the driving operation element 80 to the cylinder via a master cylinder. Note that the brake device 210 is not limited to the configuration described above, and may be an electronically controlled hydraulic brake device that controls an actuator according to information input from the second control unit 160 to transmit hydraulic pressure in the master cylinder to the cylinder.

[0073] The steering device 220 includes, for example, a steering ECU and an electric motor. The electric motor changes the direction of the steered wheels by applying a force to, for example, a rack and pinion mechanism. The steering ECU drives the electric motor in accordance with information input from the second control unit 160 or information input from the driving operator 80 to change the direction of the steered wheels.

[0074] [Overall processing flow] Hereinafter, a flow of a series of processes performed by the automatic driving control device 100 of the embodiment will be described with reference to a flowchart. Fig. 4 is a flowchart showing an example of a flow of a series of processes performed by the automatic driving control device 100 of the embodiment.

[0075] The processing of this flowchart may be repeatedly executed at a predetermined period, for example, under mode A, when the vehicle M reaches a section on the route to the destination where an event involving a lane change has been determined, or when the vehicle M is predicted to reach that section within a predetermined time or distance.

[0076] As described above, events involving lane changes include lane change events, overtaking events, branching events, merging events, lane reduction events, and the like.

[0077] First, the driver state determination unit 152 determines whether or not the event involving a lane change is an essential event before the vehicle reaches the destination (step S100).

[0078] For example, if an event involving a lane change is an event that requires a lane change in accordance with the road structure, such as a branching event, a merging event, or a lane narrowing event, the driver state determination unit 152 determines that the event is an essential event.

[0079] On the other hand, when an event involving a lane change is a lane change event or an overtaking event, the driver state determination unit 152 determines that the event is not an essential event.

[0080] When the driver state determination unit 152 determines that the event involving a lane change is not an essential event, it further determines whether the driver is awake and facing forward (step S102).

[0081] For example, the driver state determination unit 152 calculates the driver's wakefulness level based on the state of the driver's eyelids and the degree of eye opening on the image of the driver monitor camera 70. If the driver's eyelids are drooping more than when the driver is awake, if the degree of eye opening is less than when the driver is awake, if the driver blinks more frequently than when the driver is awake, or if the time the driver keeps his or her eyes closed between blinks is longer than when the driver is awake, the driver's wakefulness level is low. If the calculated wakefulness level is less than a threshold, the driver state determination unit 152 determines that the driver is not awake, and if the calculated wakefulness level is equal to or greater than the threshold, the driver state determination unit 152 determines that the driver is awake.

[0082] Furthermore, the driver state determination unit 152 estimates the driver's line of sight or facial direction from the image of the driver monitor camera 70, and determines whether the driver is facing forward (i.e., whether the driver is performing the task of monitoring the front) based on the estimated line of sight or facial direction.

[0083] When the driver is not awake and / or not facing forward, the driver state determination unit 152 determines whether a first predetermined time has elapsed from a certain starting point (step S104). The starting point is, for example, a time when the host vehicle M reaches a section on the route to the destination where an event involving a lane change is planned, and typically a time when a lane change becomes necessary for overtaking or the like.

[0084] When the first predetermined time has not elapsed from the starting point, the driver state determination unit 152 returns the process to S102, and continues to determine whether or not the driver is awake and facing forward.

[0085] Note that the driver state determination unit 152 may determine whether the host vehicle M has traveled a first predetermined distance from the starting point, instead of determining whether the first predetermined time has elapsed from the starting point.

[0086] If the driver wakes up and faces forward within the period from the starting point until a first predetermined time has elapsed or until the vehicle M has traveled a first predetermined distance from the starting point, the device control unit 156 uses the HMI 30 to request an approval operation for the lane change (step S106).

[0087] The approval operation is an operation in which the driver indicates his / her intention to approve the automatic lane change by the automatic driving control device 100 of the host vehicle M, and is typically an operation of the turn signal lever 92. Note that the approval operation may be an operation of a touch panel or a switch of the HMI 30 instead of or in addition to the operation of the turn signal lever 92.

[0088] For example, the device control unit 156 displays, on the display device of the HMI 30, a message or an image prompting the driver to operate the turn signal lever 92, thereby requesting the driver to perform an approval operation.

[0089] When the driver performs an approval operation, the device control unit 156 activates the direction indicator 90 (step S108).

[0090] Next, the action plan generation unit 140 generates a target trajectory based on an event involving a lane change, and the second control unit 160 controls the steering and speed of the vehicle M based on the target trajectory, thereby causing the vehicle M to change lanes (step S110).

[0091] On the other hand, in the process of S104, if a first predetermined time has elapsed from the starting point or the vehicle M has traveled a first predetermined distance from the starting point, the device control unit 156 activates the turn indicator 90 without an approval operation (step S112). In other words, if the driver does not wake up and / or does not face forward within the period from the starting point until the first predetermined time has elapsed or the vehicle M has traveled the first predetermined distance from the starting point, the device control unit 156 activates the turn indicator 90 without an approval operation.

[0092] Next, the driver state determination unit 152 determines whether or not the driver faces forward (step S114).

[0093] When the driver state determination unit 152 determines that the driver is not facing forward, it determines whether or not a second predetermined time has elapsed since the first predetermined time has elapsed or the vehicle M has traveled a first predetermined distance (hereinafter referred to as a first time point) (step S116). The second predetermined time may be the same as or different from the first predetermined time.

[0094] The driver state determination unit 152 may determine whether the host vehicle M has traveled a second predetermined distance from the primary time point, instead of determining whether the second predetermined time has elapsed from the primary time point. The second predetermined distance may be the same as or different from the first predetermined distance.

[0095] If the driver faces forward within a period from the primary time point until the second predetermined time has elapsed or from the primary time point until the host vehicle M has traveled a second predetermined distance, the process proceeds to S110. That is, the action plan generating unit 140 generates a target trajectory based on an event involving a lane change, and the second control unit 160 controls the steering and speed of the host vehicle M based on the target trajectory, thereby causing the host vehicle M to change lanes.

[0096] On the other hand, if the driver still does not face forward within the period from the primary point in time until the second specified time has elapsed or until the vehicle M has traveled the second specified distance from the primary point in time, the equipment control unit 156 uses the HMI 30 to output information urging the driver to face forward (step S118).

[0097] Next, the driver state determination unit 152 determines whether or not the driver faces forward (step S120).

[0098] If the driver faces forward after being prompted to do so, the process proceeds to S110. That is, the action plan generating unit 140 generates a target trajectory based on an event involving a lane change, and the second control unit 160 controls the steering and speed of the host vehicle M based on the target trajectory, thereby causing the host vehicle M to change lanes.

[0099] On the other hand, if the driver does not face the road after being prompted to do so, the second control unit 160 cancels the lane change (step S122), which ends the process of this flowchart.

[0100] [Processing flow based on alertness] Hereinafter, a series of processing steps based on the driver's alertness will be described with reference to a flowchart. Fig. 5 is a flowchart showing an example of a series of processing steps based on the driver's alertness. The processing steps of this flowchart may be repeatedly executed at a predetermined interval.

[0101] First, the driver state determination unit 152 calculates the driver's alertness level based on the state of the driver's eyelids and the degree to which the eyes are open on the image captured by the driver monitor camera 70 (step S200).

[0102] Next, the action plan generating unit 140 determines whether the wakefulness level of the driver is less than a threshold value (step S202).

[0103] When the driver's wakefulness is below the threshold, that is, when the driver is deemed to be asleep, the behavior plan generating unit 140 sets a limit on the acceleration (lateral acceleration and vertical acceleration) of the vehicle M so as not to wake up the driver (step S204). This limits lane changing and acceleration / deceleration.

[0104] Next, the device control unit 156 lowers the volume of the HMI 30 and the volume of the turn indicators 90 so as not to wake the driver (step S206). This ends the processing of this flowchart.

[0105] According to the embodiment described above, the automatic driving control device 100 detects some or all of the wakefulness, face direction, and line of sight direction of the driver of the vehicle M, and changes the manner of lane changing based on the detection results. This allows the driver to perform lane changing more comfortably.

[0106] [Hardware configuration] FIG. 6 is a diagram showing an example of a hardware configuration of the automatic driving control device 100 of the embodiment. As shown in the figure, the automatic driving control device 100 is configured such that a communication controller 100-1, a CPU 100-2, a RAM 100-3 used as a working memory, a ROM 100-4 for storing a boot program, etc., a storage device 100-5 such as a flash memory or HDD, a drive device 100-6, etc. are connected to each other by an internal bus or a dedicated communication line. The communication controller 100-1 communicates with components other than the automatic driving control device 100. The storage device 100-5 stores a program 100-5a executed by the CPU 100-2. This program is expanded in the RAM 100-3 by a DMA (Direct Memory Access) controller (not shown) or the like, and executed by the CPU 100-2. As a result, a part or all of the first control unit and the second control unit 160 are realized.

[0107] The above-described embodiment can be expressed as follows. A memory for storing a program; a processor; When the processor executes the program, Detecting a part or all of a driver's wakefulness, a face direction of the driver, and a line of sight direction of the driver; causing the vehicle to change lanes by controlling at least one of the speed or steering of the vehicle; changing the manner of the lane change based on the result of the detection; The vehicle control device is configured as follows.

[0108] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0109] 1...vehicle system, 10...camera, 12...radar device, 14...LIDAR, 16...object recognition device, 20...communication device, 30...HMI, 40...vehicle sensor, 50...navigation device, 60...MPU, 70...driver monitor camera, 80...driving operator, 82...steering wheel, 84...steering grip sensor, 90...direction indicator, 92...direction indicator lever, 100...automatic driving control device, 120...first control unit, 130...recognition unit, 140...action plan generation unit, 150...mode control unit, 152...driver state determination unit, 154...mode determination unit, 156...equipment control unit, 160...second control unit, 162...acquisition unit, 164...speed control unit, 166...steering control unit, 180...memory unit, M...own vehicle

Claims

1. A detection unit that detects a part or all of a wakefulness level of a driver of a vehicle, a face direction of the driver, and a line of sight direction of the driver; A driving control unit that controls at least one of the speed and steering of the vehicle to change lanes of the vehicle, The operation control unit is changing the manner of the lane change based on the detection result by the detection unit; Based on a detection result by the detection unit, it is determined whether or not the driver faces forward of the vehicle within a period from a predetermined time point when the vehicle reaches a section on a route to a destination where the lane change is planned until a predetermined time has elapsed, or from the predetermined time point until the vehicle travels a predetermined distance; When it is determined that the driver faces forward of the vehicle within the period, the lane change is performed. Vehicle control device.

2. When it is determined that the driver is not facing forward of the vehicle within the period, the driving control unit uses an output unit to output information prompting the driver to face forward of the vehicle. The vehicle control device according to claim 1.

3. The operation control unit is Based on a detection result by the detection unit, it is determined whether or not the driver faces a front of the vehicle within a first period from the predetermined time point until a first predetermined time has elapsed or until the vehicle has traveled a first predetermined distance from the predetermined time point; When it is determined that the driver faces a front of the vehicle within the first period, the lane change is performed; When it is determined that the driver is not facing forward of the vehicle within the first period, a turn signal is activated, and based on a detection result by the detection unit, it is determined whether or not the driver faces forward of the vehicle within a second period from the first period until a second predetermined time has elapsed or from the first period until the vehicle has traveled a second predetermined distance; outputting, using an output unit, information prompting the driver to face the front of the vehicle when it is determined that the driver is not facing the front of the vehicle within the second period of time; The vehicle control device according to claim 1 or 2.

4. The driving control unit restricts the lane change when the awakening level detected by the detection unit is less than a threshold. The vehicle control device according to any one of claims 1 to 3.

5. A detection unit that detects a part or all of a wakefulness level of a driver of a vehicle, a face direction of the driver, and a line of sight direction of the driver; A driving control unit that controls at least one of the speed and steering of the vehicle to change lanes of the vehicle, The operation control unit is changing the manner of the lane change based on the detection result by the detection unit; Based on a detection result by the detection unit, it is determined whether or not the driver faces forward of the vehicle within a first period from a predetermined time point when the vehicle reaches a section on a route to a destination where the lane change is planned until a first predetermined time has elapsed, or from the predetermined time point until the vehicle travels a first predetermined distance; When it is determined that the driver faces a front of the vehicle within the first period, the lane change is performed; When it is determined that the driver is not facing forward of the vehicle within the first period, a turn signal is activated, and based on a detection result by the detection unit, it is determined whether or not the driver faces forward of the vehicle within a second period from the first period until a second predetermined time has elapsed or from the first period until the vehicle has traveled a second predetermined distance; outputting, using an output unit, information prompting the driver to face the front of the vehicle when it is determined that the driver is not facing the front of the vehicle within the second period of time; Vehicle control device.

6. The vehicle's on-board computer Detecting a part or all of a driver's wakefulness, a face direction of the driver, and a line of sight direction of the driver of the vehicle; causing the vehicle to change lanes by controlling at least one of the speed or steering of the vehicle; changing the manner of the lane change based on the result of the detection; Based on the detection result, it is determined whether or not the driver faces forward of the vehicle within a period from a predetermined time point when the vehicle reaches a section on the route to the destination where the lane change is planned until a predetermined time has elapsed, or until the vehicle travels a predetermined distance from the predetermined time point; When it is determined that the driver faces forward of the vehicle within the period, the lane change is performed. A vehicle control method.

7. The vehicle's on-board computer Detecting a part or all of a driver's wakefulness, a face direction of the driver, and a line of sight direction of the driver of the vehicle; causing the vehicle to change lanes by controlling at least one of the speed or steering of the vehicle; changing the manner of the lane change based on the result of the detection; Based on the detection result, it is determined whether or not the driver faces the front of the vehicle within a first period from a predetermined time point when the vehicle reaches a section on the route to the destination where the lane change is planned until a first predetermined time has elapsed, or from the predetermined time point until the vehicle travels a first predetermined distance; When it is determined that the driver faces a front of the vehicle within the first period, the lane change is performed; When it is determined that the driver is not facing forward of the vehicle within the first period, a turn signal is activated, and based on the detection result, it is determined whether or not the driver faces forward of the vehicle within a second period from the first period until a second predetermined time has elapsed or from the first period until the vehicle has traveled a second predetermined distance; when it is determined that the driver is not facing forward of the vehicle within the second period, outputting information prompting the driver to face forward of the vehicle using an output unit. A vehicle control method.

8. The vehicle's on-board computer Detecting some or all of the driver's wakefulness, the driver's face direction, and the driver's line of sight direction of the vehicle; causing the vehicle to change lanes by controlling at least one of the speed or steering of the vehicle; changing the manner of the lane change based on the result of the detection; Based on the detection result, determining whether or not the driver faces forward of the vehicle within a period from a predetermined time point when the vehicle reaches a section on a route to a destination where the lane change is planned until a predetermined time has elapsed, or within a period from the predetermined time point when the vehicle travels a predetermined distance; performing the lane change when it is determined that the driver faces ahead of the vehicle within the period; A program for executing the above.

9. The vehicle's on-board computer Detecting some or all of the driver's wakefulness, the driver's face direction, and the driver's line of sight direction of the vehicle; causing the vehicle to change lanes by controlling at least one of the speed or steering of the vehicle; changing the manner of the lane change based on the result of the detection; Based on the detection result, determining whether or not the driver faces forward of the vehicle within a first period from a predetermined time point when the vehicle reaches a section on a route to a destination where the lane change is planned until a first predetermined time has elapsed, or from the predetermined time point when the vehicle travels a first predetermined distance; performing the lane change when it is determined that the driver faces ahead of the vehicle within the first period of time; when it is determined that the driver is not facing forward of the vehicle within the first period, actuating a turn signal, and determining, based on the detection result, whether or not the driver faces forward of the vehicle within a second period from the first period until a second predetermined time has elapsed or from the first period until the vehicle has traveled a second predetermined distance; when it is determined that the driver is not facing forward of the vehicle within the second period, outputting information prompting the driver to face forward of the vehicle using an output unit; A program for executing the above.

Images