Vehicle control device, vehicle control method, and vehicle control computer program

Abstract

To provide a vehicle control device capable of reducing the frequency with which driving control levels that require driver intervention are applied.SOLUTION: The vehicle control device is configured as follow: when an object that has a risk of collision with a vehicle 10 is detected based on a sensor signal of any one of multiple sensors (3-1, 3-2, 4) mounted on the vehicle 10, the level of automatic driving control applied to the vehicle 10 is shifted to a second level where it does not require accelerator and steering operations but requires the driver to monitor surroundings of the vehicle 10 from a first level that does not require the driver to operate the accelerator and steering and monitor the surroundings of the vehicle 10. On the other hand, when an object that has a risk of colliding with the vehicle 10 is detected from the sensor signals of two or more sensors, the automatic driving control level is shifted from the first level to a third level that requires driver operation of at least one of the accelerator and steering.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a vehicle control device, a vehicle control method, and a computer program for vehicle control that control automatic driving of a vehicle. [Background technology]

[0002] In a vehicle to which automatic driving control can be applied, the subject of vehicle driving control may be switched from the vehicle control device to the vehicle driver due to the environment around the vehicle or the operation of the vehicle driver. Techniques for appropriately switching the subject of control have been proposed (see Patent Documents 1 to 3).

[0003] The vehicle control device described in Patent Document 1 includes, as control states, a first state and a second state in which the level of automation of control is higher than that of state 1. This vehicle control device uses different methods of operation or determination methods for operations for ending the first state and the second state by driver operations, depending on whether the state is the first state or the second state.

[0004] The driving assistance device disclosed in Patent Document 2 controls the transfer of driving rights to the driver according to the driver's situation awareness level. In this case, the driving assistance device controls the transfer of driving rights so that the transfer of driving rights is performed in a gradual manner, with the transfer of driving rights being changed over a transition period as needed.

[0005] The driving assistance method disclosed in Patent Document 3 determines whether a section on a driving route is a mode switch section in which the driving state of the vehicle is switched from autonomous driving mode to manual driving mode when the road structure makes it difficult to continue autonomous driving. Furthermore, this driving assistance method determines whether a lane change section on the driving route in which the vehicle changes lanes is adjacent to the mode switch section. If the mode switch section and the lane change section are adjacent and the lane change section is congested, preventing autonomous driving from being performed, this driving assistance method continues manual driving mode between the point in the lane change section where the lane change is performed manually and the mode switch section. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 2019-155956 [Patent Document 2] Japanese Patent Application Publication No. 2018-169806 [Patent Document 3] Japanese Patent Application Publication No. 2019-109666 Summary of the Invention [Problem to be solved by the invention]

[0007] Even if it is possible to apply a level of driving control to a vehicle that does not require driver intervention in driving operations, the driver may feel annoyed if that level of driving control is frequently changed to a driving control level that requires driver intervention in driving operations.

[0008] Therefore, an object of the present invention is to provide a vehicle control device that can reduce the frequency with which a driving control level that requires intervention in the driver's driving operation is applied. [Means for solving the problem]

[0009] According to one embodiment, there is provided a vehicle control device that can control a vehicle autonomously according to any one of the following levels: a first level that does not require the driver of the vehicle to operate the accelerator or steering or monitor the surroundings of the vehicle, a second level that requires the driver to monitor the surroundings of the vehicle but does not require the driver to operate the accelerator or steering, and a third level that requires the driver to operate at least one of the accelerator and steering. The vehicle control device includes: a detection unit that detects objects around the vehicle based on sensor signals from each of a plurality of sensors mounted on the vehicle that can detect objects around the vehicle and determines whether the detected objects pose a risk of collision with the vehicle; and a level control unit that, when the first level of autonomous driving control is applied to the vehicle, transitions the level of autonomous driving control applied to the vehicle from the first level to the second level when an object determined to pose a risk of collision with the vehicle based on the sensor signal of one of the plurality of sensors is detected, and transitions the level of autonomous driving control applied to the vehicle from the first level to the third level when an object determined to pose a risk of collision with the vehicle based on the sensor signals of two or more of the plurality of sensors is detected.

[0010] In this vehicle control device, when the second level of autonomous driving control is applied to the vehicle, it is preferable that the level control unit transitions the level of autonomous driving control applied to the vehicle from the second level to the third level when an object is detected that is determined to be at risk of colliding with the vehicle based on sensor signals from two or more of the multiple sensors.

[0011] Furthermore, in this vehicle control device, when the second level of autonomous driving control is applied to the vehicle, it is preferable that the level control unit transitions the level of autonomous driving control applied to the vehicle from the second level to the first level if no object determined to be at risk of colliding with the vehicle is detected from any of the sensor signals of the multiple sensors over a predetermined period of time.

[0012] According to another embodiment, there is provided a vehicle control method capable of performing autonomous driving control of a vehicle according to any one of a first level that does not require the driver of the vehicle to operate the accelerator and steering or monitor the surroundings of the vehicle, a second level that requires the driver to monitor the surroundings of the vehicle but does not require the driver to operate the accelerator and steering, and a third level that requires the driver to operate at least one of the accelerator and steering. This vehicle control method includes detecting objects around the vehicle based on sensor signals from each of a plurality of sensors mounted on the vehicle that can detect objects around the vehicle, determining whether the detected objects pose a risk of collision with the vehicle, and, when the first level of autonomous driving control is applied to the vehicle, transitioning the level of autonomous driving control applied to the vehicle from the first level to the second level when an object determined to pose a risk of collision with the vehicle based on the sensor signal of one of the plurality of sensors is detected, and transitioning the level of autonomous driving control applied to the vehicle from the first level to the third level when an object determined to pose a risk of collision with the vehicle based on the sensor signals of two or more of the plurality of sensors is detected.

[0013] According to yet another embodiment, there is provided a vehicle control computer program capable of autonomous driving control of a vehicle according to any one of a first level that does not require the driver of the vehicle to operate the accelerator and steering or monitor the surroundings of the vehicle, a second level that requires the driver to monitor the surroundings of the vehicle but does not require the driver to operate the accelerator and steering, and a third level that requires the driver to operate at least one of the accelerator and steering. The vehicle control computer program causes a processor mounted on the vehicle to execute the following steps: detect objects around the vehicle based on sensor signals from each of a plurality of sensors mounted on the vehicle that can detect objects around the vehicle, determine whether the detected objects pose a risk of collision with the vehicle, and, when the first level of autonomous driving control is applied to the vehicle, transition the level of autonomous driving control applied to the vehicle from the first level to the second level when an object determined to pose a risk of collision with the vehicle based on the sensor signal of one of the plurality of sensors is detected, and transition the level of autonomous driving control applied to the vehicle from the first level to the third level when objects determined to pose a risk of collision with the vehicle based on the sensor signals of two or more of the plurality of sensors are detected. [Effects of the Invention]

[0014] The vehicle control device according to the present disclosure has the effect of reducing the frequency with which a driving control level that requires intervention in the driver's driving operation is applied. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a schematic configuration diagram of a vehicle control system in which a vehicle control device is implemented. [Figure 2] 1 is a hardware configuration diagram of an electronic control device that is one embodiment of a vehicle control device. [Figure 3] FIG. 2 is a functional block diagram of a processor of an electronic control unit related to vehicle control processing. [Figure 4]10(a) and 10(b) are diagrams showing an example of the relationship between the number of sensors related to the detection of dangerous objects and the transition of the level of autonomous driving control applied to a vehicle. [Figure 5] 4 is an operational flowchart of a vehicle control process. DETAILED DESCRIPTION OF THE INVENTION

[0016] A vehicle control device, a vehicle control method, and a vehicle control computer program executed by the vehicle control device will be described below with reference to the drawings. When a first driving control level is applied to the host vehicle, the vehicle control device detects an object around the host vehicle based on sensor signals obtained from multiple sensors mounted on the host vehicle and determines whether there is a risk of collision between the detected object and the host vehicle. The vehicle control device then controls the level of driving control applied to the host vehicle depending on the number of sensors that generated sensor signals detecting the object determined to be a risk of collision with the host vehicle. In particular, when such an object is detected based on a sensor signal from one sensor, the vehicle control device transitions the level of autonomous driving control applied to the vehicle from the first driving control level to a second driving control level that is lower in autonomous driving than the first driving control level. Furthermore, when such an object is detected based on sensor signals from two or more sensors, the vehicle control device transitions the level of autonomous driving control applied to the vehicle from the first driving control level to a third driving control level that is lower in autonomous driving than the second driving control level.

[0017] In this embodiment, the first driving control level (first level) is a driving control level that does not require the driver to operate the accelerator and steering or monitor the surroundings of the vehicle. For example, the first driving control level may be Level 3 autonomous driving control as defined by the Society of Automotive Engineers (SAE). The second driving control level (second level) is a driving control level that requires the driver to monitor the surroundings of the vehicle but does not require the driver to operate the accelerator and steering, and may be Level 2 autonomous driving control as defined by the SAE. The third driving control level (third level) is a driving control level that requires the driver to operate at least one of the accelerator and steering, and may be Level 0 or Level 1 autonomous driving control as defined by the SAE.

[0018] FIG. 1 is a schematic configuration diagram of a vehicle control system in which a vehicle control device is implemented. FIG. 2 is a hardware configuration diagram of an electronic control device, which is one embodiment of the vehicle control device. The vehicle control system 1 is mounted on a vehicle 10 and controls the vehicle 10. To this end, the vehicle control system 1 includes a GPS receiver 2, two cameras 3-1 and 3-2, a distance measurement sensor 4, a wireless communication terminal 5, a user interface 6, a storage device 7, and an electronic control unit (ECU) 8, which is an example of a vehicle control device. The GPS receiver 2, the cameras 3-1 and 3-2, the distance measurement sensor 4, the wireless communication terminal 5, the user interface 6, the storage device 7, and the ECU 8 are communicatively connected via an in-vehicle network that complies with a standard such as a controller area network. The vehicle 10 is an example of a host vehicle. The vehicle control system 1 may also include a navigation device (not shown) for searching for a planned driving route to a destination.

[0019] The GPS receiver 2 receives GPS signals from GPS satellites at predetermined intervals and determines the vehicle's own position based on the received GPS signals. The GPS receiver 2 then outputs positioning information representing the results of determining the vehicle's own position based on the GPS signals to the ECU 8 at predetermined intervals via the in-vehicle network. Note that the vehicle 10 may also have a receiver that complies with a satellite positioning system other than the GPS receiver 2. In this case, the receiver may determine the vehicle's own position.

[0020] Cameras 3-1 and 3-2 are examples of sensors capable of detecting objects around vehicle 10. Each of cameras 3-1 and 3-2 has a two-dimensional detector configured with an array of photoelectric conversion elements, such as a CCD or C-MOS, that are sensitive to visible light, and an imaging optical system that forms an image of the area to be photographed on the two-dimensional detector. The focal length of the imaging optical system of camera 3-1 is shorter than that of the imaging optical system of camera 3-2. In other words, camera 3-1 can photograph a wider area than camera 3-2, while camera 3-2 can photograph a more distant area in a magnified manner than camera 3-1. Cameras 3-1 and 3-2 are mounted, for example, inside the cabin of vehicle 10 so as to face forward of vehicle 10. Therefore, the photographing areas of cameras 3-1 and 3-2 at least partially overlap. The cameras 3-1 and 3-2 each capture an image of the area ahead of the vehicle 10 at a predetermined capture interval (for example, every 1 / 30 to 1 / 10 seconds) and generate an image of the area ahead. The images captured by the cameras 3-1 and 3-2 are an example of sensor signals and may be color images or gray images. The vehicle 10 may be provided with three or more cameras with different capture directions or focal lengths. For example, in addition to the cameras 3-1 and 3-2, the vehicle 10 may be provided with cameras mounted facing the side or rear of the vehicle 10.

[0021] Each time the cameras 3-1 and 3-2 generate an image, they output the generated image to the ECU 8 via the in-vehicle network.

[0022] The ranging sensor 4 is another example of a sensor capable of detecting other objects around the vehicle 10. The ranging sensor 4 may be, for example, a ranging sensor using LiDAR, radar, or sonar. The ranging sensor 4 generates a ranging signal representing the distance to an object in each direction within a predetermined ranging range around the vehicle 10 at predetermined intervals. The ranging signal is another example of a sensor signal. The ranging sensor 4 is attached, for example, facing the front of the vehicle 10 so that its ranging range at least partially overlaps with the imaging area of ​​the camera 3-1 or the imaging area of ​​the camera 3-2. Alternatively, the ranging sensor 4 may be attached to the vehicle 10 so that its ranging range does not overlap with the imaging areas of the cameras 3-1 and 3-2. Note that the vehicle 10 may be provided with multiple ranging sensors with different ranging directions or ranging ranges. For example, in addition to the ranging sensor 4 whose ranging range is in front of the vehicle 10, another ranging sensor whose ranging range is to the side or rear of the vehicle 10 may be provided. Furthermore, a plurality of different types of distance measurement sensors may be provided in the vehicle 10. For example, the vehicle 10 may be provided with a LiDAR distance measurement sensor and a radar distance measurement sensor.

[0023] Every time the distance measurement sensor 4 generates a distance measurement signal, the distance measurement sensor 4 outputs the generated distance measurement signal to the ECU 8 via the in-vehicle network.

[0024] The wireless communication terminal 5 wirelessly communicates with a wireless base station in accordance with a predetermined mobile communication standard. The wireless communication terminal 5 receives traffic information indicating traffic conditions on the road on which the vehicle 10 is traveling or in the vicinity thereof from another device via the wireless base station. The wireless communication terminal 5 outputs the received traffic information to the ECU 8 via the in-vehicle network. The wireless communication terminal 5 may also receive a high-precision map used for autonomous driving control for a predetermined area around the current location of the vehicle 10 from a map server via the wireless base station, and output the received high-precision map to the storage device 7.

[0025] The user interface 6 is an example of a notification unit and includes, for example, a display device such as a liquid crystal display or a touch panel display. The user interface 6 is installed in the cabin of the vehicle 10, for example, near the instrument panel, facing the driver. The user interface 6 notifies the driver of various information received from the ECU 8 via the in-vehicle network by displaying the information as icons or text information. The user interface 6 may also include one or more light sources provided on the instrument panel, speakers provided in the cabin, or vibration devices provided on the steering wheel or driver's seat. In this case, the user interface 6 notifies the driver of various information received from the ECU 8 via the in-vehicle network by outputting the information as audio signals. Alternatively, the user interface 6 may notify the driver of predetermined information by vibrating a vibration device in response to a signal received from the ECU 8 via the in-vehicle network. Alternatively, the user interface 6 may notify the driver of predetermined information by turning on or blinking a light source in response to a signal received from the ECU 8 via the in-vehicle network.

[0026] The storage device 7 is an example of a storage unit, and includes, for example, a hard disk drive, a nonvolatile semiconductor memory, or an optical recording medium and an access device therefor. The storage device 7 stores a high-precision map, which is an example of map information. The high-precision map includes, for example, information representing road markings such as lane markings and stop lines for each road included in a predetermined area represented on the high-precision map, information representing road signs, and information representing features around the road (for example, soundproof walls, etc.).

[0027] Furthermore, the storage device 7 may have a processor for executing processes such as updating the high precision map and processing related to a request to read out a high precision map from the ECU 8. In this case, for example, every time the vehicle 10 moves a predetermined distance, the storage device 7 transmits a request to acquire a high precision map together with the current position of the vehicle 10 to the map server via the wireless communication terminal 5. The storage device 7 then receives a high precision map of a predetermined area around the current position of the vehicle 10 from the map server via the wireless communication terminal 5. Furthermore, when the storage device 7 receives a request to read out a high precision map from the ECU 8, the storage device 7 extracts an area that includes the current position of the vehicle 10 and is relatively smaller than the predetermined area from the high precision map stored therein, and outputs the extracted area to the ECU 8 via the in-vehicle network.

[0028] The ECU 8 controls the vehicle 10 in accordance with the level of autonomous driving control applied to the vehicle 10 .

[0029] 2, the ECU 8 has a communication interface 21, a memory 22, and a processor 23. The communication interface 21, the memory 22, and the processor 23 may be configured as separate circuits, or may be integrated into a single integrated circuit.

[0030] The communication interface 21 has an interface circuit for connecting the ECU 8 to the in-vehicle network. The communication interface 21 passes the positioning information to the processor 23 every time it receives positioning information from the GPS receiver 2. The communication interface 21 passes the received image to the processor 23 every time it receives an image from the cameras 3-1, 3-2. The communication interface 21 passes the received ranging signal to the processor 23 every time it receives a ranging signal from the ranging sensor 4. The communication interface 21 also passes the high-precision map read from the storage device 7 to the processor 23. The communication interface 21 also outputs information or signals for the user interface 6 received from the processor 23 to the user interface 6 via the in-vehicle network.

[0031] The memory 22 is another example of a storage unit and includes, for example, a volatile semiconductor memory and a non-volatile semiconductor memory. The memory 22 stores various data used in the vehicle control process executed by the processor 23 of the ECU 8. For example, the memory 22 stores a high-precision map, parameters representing the focal length, angle of view, shooting direction, and mounting position of the cameras 3-1 and 3-2, and the ranging range of the ranging sensor 4. The memory 22 also stores a parameter set for identifying an object detection classifier used to detect other vehicles traveling around the vehicle 10. The memory 22 also temporarily stores sensor signals such as images and ranging signals, and the positioning result of the vehicle's own position measured by the GPS receiver 2. The memory 22 also temporarily stores various data generated during the vehicle control process.

[0032] The processor 23 includes one or more central processing units (CPUs) and their peripheral circuits. The processor 23 may further include other arithmetic circuits such as a logic unit, a numerical calculation unit, or a graphics processing unit. The processor 23 executes vehicle control processing for the vehicle 10.

[0033] 3 is a functional block diagram of the processor 23 related to vehicle control processing. The processor 23 has a detection unit 31, a level control unit 32, and a vehicle control unit 33. Each of these units in the processor 23 is, for example, a functional module realized by a computer program running on the processor 23. Alternatively, each of these units in the processor 23 may be a dedicated arithmetic circuit provided in the processor 23.

[0034] While autonomous driving control of the first driving control level or the second driving control level is being applied to the vehicle 10, the detection unit 31 determines, for each sensor, whether there is a risk of the vehicle 10 colliding with an object present around the vehicle 10, based on the sensor signal obtained by that sensor. In this embodiment, the detection unit 31 detects objects present around the vehicle 10 for each image obtained from each of the cameras 3-1 and 3-2, and for each ranging signal obtained from the ranging sensor 4. The detection unit 31 then determines whether there is a risk of the vehicle 10 colliding with the detected object.

[0035] In this embodiment, the object to be detected is an object that poses a risk of collision with the vehicle 10, and is, for example, a moving object such as another vehicle or a pedestrian, or a structure that exists on the road on which the vehicle 10 is traveling, such as a sign announcing road construction or an object that has fallen on the road. The object to be detected may also include a stationary object that exists on or around the road on which the vehicle 10 is traveling and that affects the traveling of the vehicle 10, such as road markings such as lane markings, road signs such as speed signs, guardrails, or curbs on the roadside. Hereinafter, the object to be detected may be simply referred to as an object or a target object.

[0036] For example, the detection unit 31 detects a target object by inputting an image acquired from the camera 3-1 or 3-2 into a classifier for object detection. As such a classifier, the detection unit 31 may use a deep neural network (DNN) with a convolutional neural network (CNN) architecture, such as a Single Shot MultiBox Detector (SSD) or Faster R-CNN. Alternatively, the detection unit 31 may use a classifier based on another machine learning method, such as an AdaBoost classifier. Such a classifier is trained in advance according to a predetermined learning method, such as backpropagation, so as to detect target objects present around the vehicle 10 from images. The classifier outputs information identifying an object region including the object detected in the input image and information indicating the type of the detected object.

[0037] The detection unit 31 also detects lane markings and the lane in which the vehicle 10 is traveling (hereinafter referred to as the own lane). In this embodiment, the detection unit 31 may also detect lane markings by inputting the image acquired from the camera 3-1 or the camera 3-2 into the above-mentioned classifier. The detection unit 31 may then determine the lane corresponding to the area sandwiched between the two lane markings located on both sides of the horizontal center of the image and closest to that center as the own lane. Furthermore, the detection unit 31 estimates the position of the vehicle 10 at the time of generating each image. In this case, the detection unit 31 may estimate the position of the vehicle 10 by comparing the image with a high-precision map and determining the position of the vehicle 10 where each feature depicted in the image most closely matches the corresponding feature depicted on the high-precision map.

[0038] Furthermore, the detection unit 31 detects objects present around the vehicle 10 based on the ranging signal. In this case, too, the detection unit 31 may detect objects around the vehicle 10 by inputting the ranging signal to a classifier that has been trained in advance to detect objects around the vehicle 10 from the ranging signal. Alternatively, the detection unit 31 may detect objects around the vehicle 10 according to another method for detecting objects from the ranging signal.

[0039] The detection unit 31 determines whether a detected object poses a risk of collision with the vehicle 10. To this end, the detection unit 31 tracks each detected object and predicts the trajectory of the object up to a predetermined time ahead. In this process, the detection unit 31 detects the object based on a sensor signal previously generated by a sensor that generated a sensor signal representing the detected object. For example, if an object of interest is detected in the latest image generated by the camera 3-1, the detection unit 31 tracks the object through a series of images generated by the camera 3-1 over a certain period of time in the past. Also, if an object of interest is detected in the latest ranging signal generated by the ranging sensor 4, the detection unit 31 tracks the object through a series of ranging signals generated by the ranging sensor 4 over a certain period of time in the past.

[0040] The detection unit 31 calculates, for each object being tracked, the distance between the object at a predicted position on the trajectory (hereinafter sometimes referred to as a predicted position) at each future point in time and the predicted position of the vehicle 10. Then, when the distance from the predicted position of the vehicle 10 to any of the objects being tracked at any point in time becomes equal to or less than a predetermined threshold, the detection unit 31 determines that there is a risk of collision between the object being tracked and the vehicle 10.

[0041] The following describes tracking of an object detected from an image generated by camera 3-1 and estimation of a predicted trajectory. Detection unit 31 also tracks an object detected from an image generated by camera 3-2 using a method similar to that described below, and estimates a predicted trajectory based on the tracking result.

[0042] The detection unit 31 tracks an object depicted in an object region of interest in the latest image and in a previous image by applying a tracking process based on optical flow, such as the Lucas-Kanade algorithm, to the object region of interest in the latest image and the object region in the previous image. To this end, the detection unit 31 extracts multiple feature points from the object region of interest by applying a feature point extraction filter, such as SIFT or a Harris operator, to the object region of interest. The detection unit 31 then calculates the optical flow by identifying corresponding points in the object region in the previous image for each of the multiple feature points according to the applied tracking method. Alternatively, the detection unit 31 may track the object depicted in the object region by applying another tracking method, which is used to track moving objects detected from images, to the object region of interest in the latest image and the object region in the previous image.

[0043] The detection unit 31 performs viewpoint conversion processing on the object being tracked using information such as the optical axis direction, focal length, and installation height of the camera 3-1, thereby converting the object's in-image coordinates into coordinates on a bird's-eye image based on the position of the camera 3-1 (bird's-eye coordinates).The detection unit 31 then performs prediction processing using a Kalman filter, a particle filter, or the like, for each bird's-eye coordinate obtained from a series of images acquired during tracking, using the position of the vehicle 10 when the series of images was generated.This allows the detection unit 31 to estimate the object's predicted trajectory up to a predetermined time ahead.

[0044] Furthermore, for an object detected from a ranging signal, the detection unit 31 compares a set of azimuth angles including the object of interest, identified in the ranging signal at a given time, with the next obtained ranging signal. The detection unit 31 then determines that the object of interest is included in a set of azimuth angles in the next obtained ranging signal, the set of azimuth angles whose degree of match with the set of azimuth angles is equal to or greater than a predetermined threshold. In this case, the detection unit 31 may calculate the degree of match based on, for example, the sum of the differences in distances for corresponding azimuth angles in the two sets of azimuth angles being compared with each other. The detection unit 31 may track the object detected from the ranging signal by repeating the above process on a series of ranging signals obtained in time series. The detection unit 31 may estimate the predicted trajectory of the object by performing the same prediction process as above on the position of the object at the time each ranging signal is generated, which is estimated from the position of the vehicle 10 at the time each ranging signal is generated and the azimuth and distance from the vehicle 10 to the object in each ranging signal.

[0045] Furthermore, the detection unit 31 may obtain the predicted position of the vehicle 10 at each time point up to a predetermined time in the future according to the latest planned travel path set by the vehicle control unit 33.

[0046] Furthermore, the detection unit 31 may determine that there is a risk of collision between an object detected by any of the sensors and the vehicle 10 if the object is of a type corresponding to a stationary object and is located ahead of the vehicle 10 on the vehicle's own lane. When an object located on the vehicle's own lane is detected based on an image generated by the camera 3-1 or 3-2, the detection unit 31 may determine whether the object is a stationary object based on the type of object output by the classifier. When an object located on the vehicle's own lane is detected based on a ranging signal, the detection unit 31 may determine that the object is a stationary object if the object's position has not changed as a result of tracking the object as described above.

[0047] In this case, the detection unit 31 estimates the distance from the vehicle 10 to a stationary object in the vehicle's lane. If the stationary object is detected based on a ranging signal, the detection unit 31 determines the distance to the stationary object to be the distance measured in the direction in which the stationary object is located, as indicated by the ranging signal. If the stationary object is detected from an image, the detection unit 31 may estimate the distance to the stationary object based on the width of the vehicle's lane at the position of the stationary object in the image, the width of the vehicle's lane at the current position of the vehicle 10 indicated on the high-precision map, and camera parameters such as the focal length.

[0048] The detection unit 31 notifies the level control unit 32 of the number and types of sensors (camera 3-1, camera 3-2, distance measurement sensor 4) that generated sensor signals representing objects determined to pose a risk of collision with the vehicle 10. The detection unit 31 also notifies the vehicle control unit 33 of the vehicle's own lane and the position of the vehicle 10. The detection unit 31 also notifies the vehicle control unit 33 of the estimated distance to a stationary object detected on the vehicle's own lane. Note that, hereinafter, an object determined to pose a risk of collision with the vehicle 10 may be simply referred to as a dangerous object.

[0049] When the first driving control level is applied to the vehicle 10, the level control unit 32 controls the level of automatic driving control applied to the vehicle 10 based on the number of sensors that generated a sensor signal indicating a dangerous object each time it is notified of the number of sensors.

[0050] When a dangerous object is detected based on a sensor signal from any one of the multiple sensors, the level control unit 32 transitions the level of autonomous driving control applied to the vehicle 10 from the first driving control level to the second driving control level. In this embodiment, when a dangerous object is detected based on any one of the images from the camera 3-1, the image from the camera 3-2, and the ranging signal from the ranging sensor 4, the level control unit 32 transitions from the first driving control level to the second driving control level. This transition requires the driver to monitor the surroundings of the vehicle 10, allowing the driver to easily notice the dangerous object. Therefore, when the risk of a collision between the dangerous object and the vehicle 10 increases, the driver can immediately take over driving control of the vehicle 10. Note that when a dangerous object is detected based on only the sensor signal from any one of the multiple sensors, there is a possibility that another object that is easily mistaken for a dangerous object has been mistakenly detected as a dangerous object. If autonomous driving control is terminated due to such a false detection, the driver's convenience is impaired. However, in this embodiment, if a dangerous object is detected from the sensor signal of only one of the sensors, the level of the autonomous driving control is transitioned to the second driving control level, so the driver does not need to operate the steering wheel or accelerator, and therefore convenience for the driver is not significantly impaired.

[0051] Furthermore, when a dangerous object is detected based on sensor signals from two or more of the multiple sensors, the level control unit 32 transitions the applied level of autonomous driving control from the first driving control level to the third driving control level. In this embodiment, when a dangerous object is detected based on two or more of the images from the cameras 3-1 and 3-2, and the ranging signal from the ranging sensor 4, the level control unit 32 transitions from the first driving control level to the third driving control level. When a dangerous object is detected based on sensor signals from two or more sensors, the possibility of the dangerous object being falsely detected is low. Therefore, by transitioning the applied level of autonomous driving control to the third driving control level, the driver can control the vehicle 10 to avoid a collision with the dangerous object.

[0052] In addition, if no dangerous object is detected from the sensor signals of any of the sensors, the level control unit 32 does not change the level of automatic driving control currently applied to the vehicle 10 and continues as is.

[0053] According to a modified example, when a dangerous object is detected based on sensor signals from two or more sensors while the second driving control level is being applied to the vehicle 10, the level control unit 32 may transition the applied level of autonomous driving control to a third driving control level. This allows the driver to control the vehicle 10 to avoid a collision with the dangerous object, similar to when the applied level of autonomous driving control is transitioned from the first driving control level to the third driving control level.

[0054] According to another modification, when the second driving control level is applied to the vehicle 10, the level control unit 32 may transition the applied level of autonomous driving control to the first driving control level if no dangerous object is detected from the sensor signals of any of the sensors for a predetermined period of time. Note that the predetermined period may be, for example, several seconds to several tens of seconds. This allows the ECU 8 to further improve the convenience of the driver while ensuring the safety of the vehicle 10.

[0055] When the level control unit 32 transitions the level of autonomous driving control applied to the vehicle 10, it notifies the driver of the transition of the control level via the user interface 6. For example, when transitioning the level of autonomous driving control applied to the vehicle 10 to a second driving control level, the level control unit 32 notifies the driver of a warning requesting the driver to monitor the surroundings of the vehicle 10 via the user interface 6. At this time, the level control unit 32 displays a message or icon representing the warning on a display device included in the user interface 6, or turns on or blinks a light source corresponding to the warning. Alternatively, the level control unit 32 outputs an audio signal representing the warning to a speaker included in the user interface 6. Alternatively, the level control unit 32 vibrates a vibration device included in the user interface 6 according to a vibration type (vibration period or vibration intensity) corresponding to the warning. Note that if the user interface 6 has two or more of the above-mentioned devices, the level control unit 32 may notify the driver of the warning via each or any of the two or more devices.

[0056] Furthermore, when the level of autonomous driving control applied to the vehicle 10 is transitioned to the third driving control level, the level control unit 32 notifies the driver via the user interface 6 of a warning indicating that driving control of the vehicle 10 will be transferred to the driver. At this time, the level control unit 32 displays a message or icon representing the warning on a display device included in the user interface 6, or turns on or flashes a light source corresponding to the warning. Alternatively, the level control unit 32 outputs an audio signal representing the warning to a speaker included in the user interface 6. Alternatively, the level control unit 32 vibrates a vibration device included in the user interface 6 according to a vibration method corresponding to the warning. Note that if the user interface 6 has two or more of the above-mentioned devices, the level control unit 32 may notify the driver of the warning via each or any of the two or more devices.

[0057] Furthermore, when the level of autonomous driving control applied to the vehicle 10 is transitioned to the first driving control level, the level control unit 32 notifies the driver via the user interface 6 that the obligation to monitor the surroundings of the vehicle 10 is lifted. At this time, the level control unit 32 displays a message or icon indicating the lifting of the obligation to monitor on a display device of the user interface 6, or turns on or flashes a light source corresponding to the message. Alternatively, the level control unit 32 outputs an audio signal representing the message to a speaker of the user interface 6. Alternatively, the level control unit 32 vibrates a vibration device of the user interface 6 according to a vibration method corresponding to the message. Note that if the user interface 6 has two or more of the above-mentioned devices, the level control unit 32 may notify the driver of the message via each or any of the two or more devices.

[0058] 4(a) and 4(b) are diagrams showing an example of the relationship between the number of sensors related to the detection of dangerous objects and the transition of the level of autonomous driving control applied to the vehicle 10.

[0059] In the example shown in FIG. 4(a), a dangerous object 401 is detected from an image 411 captured by the camera 3-2. However, because the dangerous object 401 is far from the vehicle 10, the dangerous object 401 appears in an image 412 captured by the camera 3-1, but is too small on the image 412. Therefore, the dangerous object 401 is not detected in the image 412. Furthermore, the dangerous object 401 is not detected from the distance measurement signal from the distance measurement sensor 4. Therefore, the level control unit 32 transitions the level of autonomous driving control applied to the vehicle 10 from the first driving control level to the second driving control level.

[0060] 4(b), the dangerous object 402 is detected not only from the image 421 captured by the camera 3-2 but also from the image 422 captured by the camera 3-1. Therefore, the level control unit 32 transitions the level of the autonomous driving control applied to the vehicle 10 from the first driving control level to the third driving control level.

[0061] When a predetermined time has elapsed since the level control unit 32 notified the transition of the level of the automatic driving control described above, the level control unit 32 transitions the level of the automatic driving control applied to the vehicle 10 to the notified driving control level.

[0062] Every time the level control unit 32 changes the level of the automatic driving control applied to the vehicle 10, it notifies the vehicle control unit 33 of the changed control level.

[0063] The vehicle control unit 33 controls the vehicle 10 according to the level of autonomous driving control applied to the vehicle 10 .

[0064] For example, when the level of automatic driving control applied to the vehicle 10 is the first driving control level or the second driving control level, the vehicle control unit 33 sets a route along which the vehicle 10 is scheduled to travel within a predetermined time (hereinafter referred to as a planned driving route). Note that the planned driving route is expressed as a set of target positions of the vehicle 10 at each time when the vehicle 10 travels through a predetermined section, for example.

[0065] The vehicle control unit 33 refers to the high-precision map and sets a planned driving route so that the vehicle 10 will travel along the planned driving route to the destination. For example, if the vehicle's lane is aligned with the planned driving route, the vehicle control unit 33 sets the planned driving route along the center of the vehicle's lane. Furthermore, if the planned driving route indicates that the vehicle will head toward a road that branches off from the road it is currently traveling at a point a predetermined distance ahead, the vehicle control unit 33 sets the planned driving route so that the vehicle will change lanes from the vehicle's lane to a lane that allows entry onto the branching road.

[0066] Once the planned travel route is set, the vehicle control unit 33 controls each unit of the vehicle 10 so that the vehicle 10 travels along the planned travel route. For example, the vehicle control unit 33 calculates a target acceleration of the vehicle 10 based on the planned travel route and the current vehicle speed of the vehicle 10 measured by a vehicle speed sensor (not shown), and sets the accelerator opening or braking amount so as to achieve the target acceleration. The vehicle control unit 33 then calculates a fuel injection amount based on the set accelerator opening, and outputs a control signal corresponding to the fuel injection amount to a fuel injection device of the engine of the vehicle 10. Alternatively, the vehicle control unit 33 controls a power supply device to a motor for driving the vehicle 10 so as to supply power corresponding to the set accelerator opening to the motor. Alternatively, the vehicle control unit 33 outputs a control signal corresponding to the set braking amount to the brake of the vehicle 10. Furthermore, the vehicle control unit 33 determines the steering angle of the vehicle 10 for driving the vehicle 10 according to the planned driving route and the current position of the vehicle 10, and outputs a control signal corresponding to the steering angle to an actuator (not shown) that controls the steering wheels of the vehicle 10.

[0067] Furthermore, the driver may not take the required action even after a predetermined period of time has elapsed since the driver was notified that the level of autonomous driving control applied to the vehicle 10 will be transitioned to the second driving control level or the third driving control level. In such a case, the vehicle control unit 33 may control the vehicle 10 to stop the vehicle 10. Whether the driver has taken the required action is determined, for example, based on an in-vehicle image generated by a camera (not shown) for driver monitoring provided in the cabin of the vehicle 10 or a detection signal from a touch sensor (not shown) provided on the steering wheel. For example, the vehicle control unit 33 detects the driver's line of sight or facial orientation from the in-vehicle image, and determines whether the driver is monitoring the area ahead of the vehicle 10 based on the detection result. If the driver is monitoring the area ahead of the vehicle 10, the vehicle control unit 33 determines that the driver is monitoring the area around the vehicle 10, which is necessary for applying the second driving control level. Furthermore, when the vehicle control unit 33 determines that the driver is monitoring the road ahead of the vehicle 10 based on the interior image and receives a detection signal from the touch sensor indicating that the driver is maintaining the steering wheel, it determines that the driver is ready to control the vehicle 10. That is, the vehicle control unit 33 determines that the driver is ready to apply the third driving control level. The vehicle control unit 33 detects the driver's facial organs by inputting the interior image to a classifier that has been trained to detect each facial organ. The vehicle control unit 33 can use, for example, a DNN with a CNN-type architecture as such a classifier. The vehicle control unit 33 then matches the detected organs to a 3D facial model while changing the orientation of the 3D facial model in various ways, thereby detecting the facial orientation of the 3D model that best matches each organ, and sets the detected facial orientation as the driver's facial orientation. The vehicle control unit 33 also performs template matching on the interior image, or inputs the interior image to a classifier to detect pupils and corneal reflections of the light source of the driver monitor camera. The vehicle control unit 33 then detects the driver's line of sight based on the positional relationship between the center of gravity of the pupil and the corneal reflection image.

[0068] 5 is an operational flowchart of vehicle control processing executed by processor 23. While autonomous driving control of the first driving control level is being applied to vehicle 10, processor 23 executes vehicle control processing in accordance with the following operational flowchart at predetermined intervals.

[0069] The detection unit 31 of the processor 23 detects objects around the vehicle 10 based on the sensor signals obtained by each sensor installed in the vehicle 10, and determines whether the detected objects are dangerous objects (step S101).

[0070] The level control unit 32 of the processor 23 determines whether the number of sensors that generated sensor signals indicating the detection of a dangerous object is two or more among the multiple sensors provided in the vehicle 10 (step S102). If the number of sensors that generated sensor signals indicating the detection of a dangerous object is two or more (step S102-Yes), the level control unit 32 transitions the level of autonomous driving control applied to the vehicle 10 from the first driving control level to the third driving control level (step S103).

[0071] On the other hand, if the number of sensors that generated the sensor signals indicating the detection of a dangerous object is less than two (step S102-Yes), the level control unit 32 determines whether the number of sensors that generated the sensor signals indicating the detection of a dangerous object is one (step S104).If the number of sensors that generated the sensor signals indicating the detection of a dangerous object is one (step S104-Yes), the level control unit 32 transitions the level of autonomous driving control applied to the vehicle 10 from the first driving control level to the second driving control level (step S105).

[0072] After step S103 or S105, the level control unit 32 notifies the driver of the transition of the level of the automatic driving control via the user interface 6 (step S106).

[0073] If no dangerous object is detected from the sensor signals of any of the sensors in step S104 (step S104-No), the level control unit 32 does not change the level of autonomous driving control currently applied to the vehicle 10 and continues as is (step S107).

[0074] After step S106 or step S107, the vehicle control unit 33 of the processor 23 controls the vehicle 10 in accordance with the applied level of autonomous driving control (step S108). Then, the processor 23 ends the vehicle control process.

[0075] As described above, this vehicle control device controls the level of autonomous driving control applied to the vehicle depending on the number of sensors, among multiple sensors installed in the vehicle, that generated sensor signals indicating the detection of a dangerous object. In particular, when a dangerous object is detected based on sensor signals from two or more sensors, this vehicle control device reduces the level of autonomous driving control to a level that requires the driver to operate the accelerator or steering. On the other hand, when a dangerous object is detected based on the sensor signal from only one sensor, this vehicle control device only reduces the level of autonomous driving control applied to a level that requires the driver to monitor the surroundings but does not require the driver to operate the accelerator or steering. Therefore, this vehicle control device can reduce the frequency of application of driving control levels that require driver intervention while reducing the risk of vehicle collision, thereby improving driver convenience.

[0076] According to a modified example, when the level control unit 32 transitions the level of autonomous driving control applied to the vehicle 10 from the first driving control level to the second driving control level, the level control unit 32 may request the driver to hold the steering wheel while monitoring the surroundings of the vehicle 10. In this case, the level control unit 32 may notify the driver via the user interface 6 of a warning requesting the driver to monitor the surroundings of the vehicle 10 and to hold the steering wheel.

[0077] According to another modification, the number of sensors mounted on the vehicle that can detect objects around the vehicle may be two. For example, in the above embodiment, either camera 3-1 or camera 3-2 may be omitted, or distance sensor 4 may be omitted. Even in this case, the vehicle control device can achieve the same effects as those of the above embodiment.

[0078] A computer program that realizes the functions of the processor 23 of ECU 8 according to the above embodiment or variant may be provided in a form recorded on a computer-readable portable recording medium such as a semiconductor memory, a magnetic recording medium or an optical recording medium.

[0079] As described above, those skilled in the art can make various modifications to the embodiments within the scope of the present invention. [Explanation of symbols]

[0080] 1. Vehicle control system 10 vehicles 2 GPS receivers 3-1, 3-2 Camera 4. Distance sensor 5. Wireless communication terminal 6 User Interface 7 Storage Devices 8 Electronic Control Unit (ECU) 21 Communication Interface 22 Memory 23 processors 31 Detector 32 Level control section 33 Vehicle control unit

Claims

1. A vehicle control device capable of controlling automatic driving of a vehicle according to any one of a first level that does not require the driver of the vehicle to operate an accelerator and a steering wheel or to monitor the surroundings of the vehicle, a second level that requires the driver to monitor the surroundings of the vehicle but does not require the driver to operate an accelerator and a steering wheel, and a third level that requires the driver to operate at least one of the accelerator and the steering wheel, a detection unit that detects objects around the vehicle based on sensor signals from each of a plurality of sensors mounted on the vehicle that can detect objects around the vehicle, and determines whether or not the detected objects pose a risk of collision with the vehicle; a level control unit that, when the first level of autonomous driving control is applied to the vehicle, transitions the level of autonomous driving control applied to the vehicle from the first level to the second level when an object determined to be at risk of colliding with the vehicle based on the sensor signal of one of the plurality of sensors is detected, and transitions the level of autonomous driving control applied to the vehicle from the first level to the third level when an object determined to be at risk of colliding with the vehicle based on the sensor signals of two or more of the plurality of sensors is detected; A vehicle control device having the above.

2. 2. The vehicle control device according to claim 1, wherein when the second level of autonomous driving control is applied to the vehicle, and an object is detected that is determined to be at risk of colliding with the vehicle based on the sensor signals of two or more sensors among the plurality of sensors, the level control unit transitions the level of autonomous driving control applied to the vehicle from the second level to the third level.

3. 3. The vehicle control device according to claim 1, wherein when the second level of autonomous driving control is applied to the vehicle, if no object determined to be at risk of colliding with the vehicle is detected from the sensor signals of any of the plurality of sensors over a predetermined period of time, the level control unit transitions the level of autonomous driving control applied to the vehicle from the second level to the first level.

4. A vehicle control method capable of controlling automatic driving of a vehicle according to any one of a first level that does not require the driver of the vehicle to operate an accelerator and a steering wheel and to monitor the surroundings of the vehicle, a second level that requires the driver to monitor the surroundings of the vehicle but does not require the driver to operate an accelerator and a steering wheel, and a third level that requires the driver to operate at least one of the accelerator and the steering wheel, detecting an object around the vehicle based on a sensor signal from each of a plurality of sensors mounted on the vehicle that can detect an object around the vehicle, and determining whether or not the detected object poses a risk of collision with the vehicle; When the first level of autonomous driving control is applied to the vehicle, if an object that is determined to be at risk of colliding with the vehicle based on the sensor signal of one of the plurality of sensors is detected, the level of autonomous driving control applied to the vehicle is transitioned from the first level to the second level; When an object determined to be at risk of colliding with the vehicle is detected based on the sensor signals from two or more sensors among the plurality of sensors, a level of autonomous driving control applied to the vehicle is transitioned from the first level to the third level. A vehicle control method comprising:

5. A computer program for vehicle control capable of controlling automatic driving of a vehicle according to any one of a first level that does not require the driver of the vehicle to operate an accelerator and a steering wheel or to monitor the surroundings of the vehicle, a second level that requires the driver to monitor the surroundings of the vehicle but does not require the driver to operate an accelerator and a steering wheel, and a third level that requires the driver to operate at least one of the accelerator and the steering wheel, detecting an object around the vehicle based on a sensor signal from each of a plurality of sensors mounted on the vehicle that can detect an object around the vehicle, and determining whether or not the detected object poses a risk of collision with the vehicle; When the first level of autonomous driving control is applied to the vehicle, if an object that is determined to be at risk of colliding with the vehicle based on the sensor signal of one of the plurality of sensors is detected, the level of autonomous driving control applied to the vehicle is transitioned from the first level to the second level; When an object determined to be at risk of colliding with the vehicle is detected based on the sensor signals from two or more sensors among the plurality of sensors, a level of autonomous driving control applied to the vehicle is transitioned from the first level to the third level. A computer program for vehicle control that causes a processor mounted on the vehicle to execute the above.

Images