Vehicle control device, vehicle control method, and vehicle control computer program

Abstract

The present disclosure relates to a vehicle control device, a vehicle control method, and a computer program for vehicle control. The vehicle control device, when an object that poses a risk of collision with a vehicle (10) is detected from a sensor signal of any one of a plurality of sensors (3-1, 3-2, 4) mounted on the vehicle (10), causes a level of automatic driving control applied to the vehicle (10) to migrate from a first level where an accelerator and a steering wheel do not need to be operated by a driver and the surroundings of the vehicle (10) do not need to be monitored to a second level where the surroundings of the vehicle (10) need to be monitored by the driver but the accelerator and the steering wheel do not need to be operated, and, on the other hand, when an object that poses a risk of collision with the vehicle (10) is detected from sensor signals of two or more sensors, causes the level of the automatic driving control to migrate from the first level to a third level where at least one of the accelerator and the steering wheel needs to be operated by the driver.

Description

Technical Field

[0001] This invention relates to a vehicle control device, a vehicle control method, and a computer program for vehicle control that enables automatic driving control of vehicles. Background Technology

[0002] In vehicles capable of autonomous driving control, the driving control entity sometimes switches from the vehicle's control unit to the driver due to the surrounding environment or the driver's operation. Techniques for appropriately performing such a switch of control entity have been proposed (see Japanese Patent Application Publication Nos. 2019-155956, 2018-169806, and 2019-109666).

[0003] In the vehicle control device disclosed in Japanese Patent Application Publication No. 2019-155956, the control states include a first state and a second state where the level of automation of the control is higher than that of the first state. Furthermore, the vehicle control device makes the operation method or operation determination method different for each operation used to end the first state and the second state through the driver's operation.

[0004] The driving support device disclosed in Japanese Patent Application Publication No. 2018-169806 controls the transfer of driving rights to the driver based on the driver's condition level. At this time, the driving support device controls the transfer of rights in a phased manner, changing the transfer process and incurring migration time, as needed.

[0005] In the driving support method disclosed in Japanese Patent Application Publication No. 2019-109666, when an area along the driving path is difficult to continue automatic driving due to road conditions, it is determined whether it is a mode switching area where the vehicle's driving state should be switched from automatic driving mode to manual driving mode. Furthermore, in this driving support method, it is determined whether the lane-changing area where the vehicle changes lanes along the driving path is continuous with the mode switching area. Moreover, in this driving support method, if the mode switching area and the lane-changing area are continuous, and the lane-changing area is congested and lane-changing cannot be performed automatically, manual driving mode is continued between the location where the lane-changing is performed manually and the mode switching area. Summary of the Invention

[0006] Even when a level of driving control that does not require driver intervention can be applied to a vehicle, drivers can sometimes find it inconvenient when that level is frequently changed to a level that requires driver intervention.

[0007] Therefore, the object of the present invention is to provide a vehicle control device that can reduce the frequency of applying driving control levels that require driver intervention.

[0008] According to one embodiment, a vehicle control device is provided capable of performing automated driving control of a vehicle according to any one of a first level, a second level, and a third level. The first level does not require the driver to operate the accelerator and steering wheel or monitor the vehicle's surroundings. The second level requires the driver to monitor the vehicle's surroundings but does not require operation of the accelerator and steering wheel. The third level requires the driver to operate at least one of the accelerator and steering wheel. The vehicle control device includes: a detection unit that detects objects around the vehicle based on sensor signals from multiple sensors mounted on the vehicle and capable of detecting objects around the vehicle, and determines whether there is a risk of collision with the detected object; and a level control unit that, when the vehicle is under first-level automated driving control, shifts the level of automated driving control applied to the vehicle from the first level to the second level when an object deemed to pose a risk of collision with the vehicle is detected based on sensor signals from one of the multiple sensors, and shifts the level of automated driving control applied to the vehicle from the first level to the third level when objects deemed to pose a risk of collision with the vehicle are detected based on sensor signals from two or more of the multiple sensors.

[0009] In this vehicle control device, preferably when Level 2 autonomous driving control is applied in the vehicle, when an object that is determined to pose a risk of collision with the vehicle is detected based on sensor signals from two or more of the multiple sensors, the level control unit shifts the level of autonomous driving control applied to the vehicle from Level 2 to Level 3.

[0010] Furthermore, in this vehicle control device, preferably when Level 2 autonomous driving control is applied in the vehicle, if no object deemed to pose a risk of collision with the vehicle is detected by the sensor signals of any of the multiple sensors during a predetermined period, the level control unit shifts the level of autonomous driving control applied to the vehicle from Level 2 to Level 1.

[0011] According to other embodiments, a vehicle control method is provided, capable of autonomous driving control of a vehicle according to any one of a first level, a second level, and a third level, wherein the first level does not require the driver to operate the accelerator and steering wheel or monitor the vehicle's surroundings; the second level requires the driver to monitor the vehicle's surroundings but does not require operation of the accelerator and steering wheel; and the third level requires the driver to operate at least one of the accelerator and steering wheel. The vehicle control method includes: detecting objects around the vehicle based on sensor signals from multiple sensors mounted on the vehicle and capable of detecting objects around the vehicle, and determining whether there is a risk of collision with the detected object; when the vehicle is under first-level autonomous driving control, if an object deemed to pose a risk of collision with the vehicle is detected based on a sensor signal from one of the multiple sensors, shifting the level of autonomous driving control applied to the vehicle from the first level to the second level; and if an object deemed to pose a risk of collision with the vehicle is detected based on sensor signals from two or more of the multiple sensors, shifting the level of autonomous driving control applied to the vehicle from the first level to the third level.

[0012] According to further embodiments, a vehicle control computer program is provided, capable of performing autonomous driving control of a vehicle according to any one of a first level, a second level, and a third level. The first level does not require the driver to operate the accelerator and steering wheel or monitor the vehicle's surroundings. The second level requires the driver to monitor the vehicle's surroundings but does not require operation of the accelerator and steering wheel. The third level requires the driver to operate at least one of the accelerator and steering wheel. This vehicle control computer program is used to cause a processor mounted on the vehicle to execute: detecting objects around the vehicle based on sensor signals from multiple sensors mounted on the vehicle capable of detecting objects around the vehicle, and determining whether there is a risk of collision with the detected object; when the vehicle is under first-level autonomous driving control, if an object deemed to pose a risk of collision with the vehicle is detected based on a sensor signal from one of the multiple sensors, shifting the level of autonomous driving control applied to the vehicle from the first level to the second level; and if an object deemed to pose a risk of collision with the vehicle is detected based on sensor signals from two or more of the multiple sensors, shifting the level of autonomous driving control applied to the vehicle from the first level to the third level.

[0013] The vehicle control device disclosed herein has the effect of reducing the frequency of application of driving control levels that require driver intervention. Attached Figure Description

[0014] Figure 1 It is a schematic diagram of the structure of a vehicle control system with vehicle control devices installed.

[0015] Figure 2 This is a hardware structure diagram of an electronic control device as one embodiment of a vehicle control device.

[0016] Figure 3 It is a functional block diagram of the processor of the electronic control device related to vehicle control processing.

[0017] Figure 4A This is a graph illustrating an example of the relationship between the number of sensors associated with the detection of hazardous objects and the shift in the level of autonomous driving control applied to a vehicle.

[0018] Figure 4B This is a graph illustrating an example of the relationship between the number of sensors associated with the detection of hazardous objects and the shift in the level of autonomous driving control applied to a vehicle.

[0019] Figure 5 It is a flowchart of the vehicle control and processing actions. Detailed Implementation

[0020] Hereinafter, with reference to the accompanying drawings, a vehicle control device, a vehicle control method implemented in the vehicle control device, and a computer program for vehicle control will be described. When applying a first driving control level in the vehicle, the vehicle control device detects objects around the vehicle based on sensor signals obtained from various sensors of a plurality of sensors mounted on the vehicle, and determines whether there is a risk of collision with the vehicle upon detection. Furthermore, the vehicle control device controls the level of driving control applied to the vehicle based on the number of sensors that generate sensor signals for objects detected and determined to pose a risk of collision with the vehicle. Specifically, when such an object is detected based on a sensor signal from one sensor, the vehicle control device shifts the level of automatic driving control applied to the vehicle from the first driving control level to a second driving control level, which is lower than the first driving control level. Additionally, when such an object is detected based on sensor signals from two or more sensors, the vehicle control device shifts the level of automatic driving control applied to the vehicle from the first driving control level to a third driving control level, which is lower than the second driving control level.

[0021] In this embodiment, the first driving control level (Level 1) is a driving control level that does not require the driver to operate the accelerator and steering wheel, or monitor the vehicle's surroundings. For example, the first driving control level can be Level 3 automated driving control as defined by the Society of Automotive Engineers (SAE). Furthermore, the second driving control level (Level 2) is a driving control level that requires the driver to monitor the vehicle's surroundings but does not require operation of the accelerator and steering wheel; for example, it can be Level 2 automated driving control as defined by SAE. Moreover, the third driving control level (Level 3) is a driving control level that requires the driver to operate at least one of the accelerator and steering wheel; for example, it can be Level 0 or Level 1 automated driving control as defined by SAE.

[0022] Figure 1 This is a schematic diagram of the vehicle control system equipped with vehicle control devices. Additionally, Figure 2 This is a hardware structure diagram of an electronic control device as one embodiment of a vehicle control device. The vehicle control system 1 is mounted on and controls the vehicle 10. For this purpose, the vehicle control system 1 includes a GPS receiver 2, two cameras 3-1 and 3-2, a distance sensor 4, a wireless communication terminal 5, a user interface 6, a storage device 7, and an electronic control unit (ECU) 8, which is an example of a vehicle control device. The GPS receiver 2, cameras 3-1 and 3-2, distance sensor 4, wireless communication terminal 5, user interface 6, storage device 7, and ECU 8 are connected and can communicate via an in-vehicle network according to a standard controller area network (CLAN). Furthermore, the vehicle 10 is an example of this vehicle. Additionally, the vehicle control system 1 may also include a navigation device (not shown) for searching a predetermined driving route to the destination.

[0023] GPS receiver 2 receives GPS signals from GPS satellites at predetermined intervals and determines the vehicle 10's own position based on the received GPS signals. Furthermore, GPS receiver 2 outputs positioning information, indicating the vehicle 10's own position based on the GPS signals, to ECU 8 via the in-vehicle network at predetermined intervals. Alternatively, vehicle 10 may have a receiver based on a satellite positioning system other than GPS receiver 2. In this case, that receiver can determine the vehicle 10's own position.

[0024] Cameras 3-1 and 3-2 are examples of sensors capable of detecting objects around vehicle 10. Cameras 3-1 and 3-2 each have a two-dimensional detector composed of an array of photoelectric conversion elements sensitive to visible light, such as CCD or C-MOS, and an imaging optical system that images the area to be photographed onto the two-dimensional detector. Furthermore, the focal length of the imaging optical system of camera 3-1 is shorter than that of the imaging optical system of camera 3-2. That is, camera 3-1 can capture an area wider than camera 3-2, while camera 3-2 can magnify and capture an area farther away than camera 3-1. Cameras 3-1 and 3-2 are mounted, for example, facing forward of vehicle 10, inside the vehicle interior. Therefore, the shooting areas of camera 3-1 and camera 3-2 at least partially overlap. Moreover, cameras 3-1 and 3-2 each capture the area in front of vehicle 10 at a predetermined shooting cycle (e.g., 1 / 30 second to 1 / 10 second), generating an image of that area. The images obtained by cameras 3-1 and 3-2 are an example of sensor signals, and can be either color or grayscale images. Furthermore, three or more cameras with different shooting directions or focal lengths can be installed on the vehicle 10. For example, in addition to cameras 3-1 and 3-2, cameras mounted to the side or rear of the vehicle 10 can also be installed on the vehicle 10.

[0025] Each of the cameras 3-1 and 3-2 outputs the generated image to the ECU8 via the in-vehicle network whenever it generates an image.

[0026] The ranging sensor 4 is another example of a sensor capable of detecting other objects around the vehicle 10. The ranging sensor 4 can be, for example, a ranging sensor utilizing LiDAR, radar, or sonar. The ranging sensor 4 generates a ranging signal representing the distance up to an object present in each direction within a predetermined ranging range around the vehicle 10, according to a predetermined period. The ranging signal is another example of a sensor signal. The ranging sensor 4 is mounted facing forward of the vehicle 10, for example, such that its ranging range at least partially overlaps with the shooting area of ​​camera 3-1 or camera 3-2. Alternatively, the ranging sensor 4 can be mounted on the vehicle 10 in a manner that prevents its ranging range from overlapping with the shooting areas of camera 3-1 and camera 3-2. Furthermore, multiple ranging sensors with different ranging directions or ranging ranges can be provided on the vehicle 10. For example, other ranging sensors with ranging ranges covering the sides or rear of the vehicle 10 can be provided independently of the ranging sensor 4, which covers the front of the vehicle 10. Furthermore, multiple ranging sensors of different types can be installed on the vehicle 10. For example, a ranging sensor using LiDAR and a ranging sensor using radar can also be installed on the vehicle 10.

[0027] Whenever a ranging sensor 4 generates a ranging signal, it outputs the generated ranging signal to the ECU 8 via the in-vehicle network.

[0028] The wireless communication terminal 5 communicates wirelessly with the wireless base station according to a predetermined mobile communication standard. Furthermore, the wireless communication terminal 5 receives traffic information from other devices via the wireless base station, indicating the traffic conditions of the road or its surroundings where the vehicle 10 is traveling. The wireless communication terminal 5 outputs the received traffic information to the ECU 8 via the in-vehicle network. Additionally, the wireless communication terminal 5 can also receive a high-precision map of a predetermined area surrounding the current location of the vehicle 10, used in autonomous driving control, from a map server via the wireless base station, and output the received high-precision map to the storage device 7.

[0029] User interface 6 is an example of a notification unit, such as a display device with an LCD screen or a touch panel display. User interface 6 is located in the vehicle interior of vehicle 10, for example, near the dashboard and facing the driver. Furthermore, user interface 6 notifies the driver of various information received from ECU 8 via the in-vehicle network by displaying it as icons or text information. User interface 6 may also include one or more light sources installed on the dashboard, speakers installed in the vehicle interior, or vibration devices installed on the steering wheel or driver's seat. In this case, user interface 6 notifies the driver of various information received from ECU 8 via the in-vehicle network as sound signals. Alternatively, user interface 6 may also notify the driver of predetermined information by vibrating the vibration device using signals received from ECU 8 via the in-vehicle network. Furthermore, user interface 6 may also notify the driver of predetermined information by illuminating or flashing a light source using signals received from ECU 8 via the in-vehicle network.

[0030] Storage device 7 is an example of a storage unit, such as a hard disk drive, a non-volatile semiconductor memory, or an optical recording medium and its access device. Furthermore, storage device 7 stores a high-precision map, as an example of map information. The high-precision map includes, for example, information about each road included in a predetermined area represented by the high-precision map, information about road markings such as lane markings or stop lines, information about road signs, and information about surrounding structures (such as sound barriers).

[0031] Furthermore, the storage device 7 may also include a processor for performing high-precision map update processing and processing related to high-precision map readout requests from the ECU 8. In this case, for example, whenever the vehicle 10 moves a predetermined distance, the storage device 7 sends the current location of the vehicle 10 and a request to obtain a high-precision map to the map server via the wireless communication terminal 5. Moreover, the storage device 7 receives a high-precision map of a predetermined area surrounding the current location of the vehicle 10 from the map server via the wireless communication terminal 5. Additionally, when the storage device 7 receives a high-precision map readout request from the ECU 8, it cuts out a range from the stored high-precision map that includes the current location of the vehicle 10 and is relatively narrower than the aforementioned predetermined area, and outputs it to the ECU 8 via the in-vehicle network.

[0032] ECU8 controls vehicle 10 according to the level of autonomous driving control applied to vehicle 10.

[0033] like Figure 2As shown, ECU8 has a communication interface 21, a memory 22, and a processor 23. The communication interface 21, the memory 22, and the processor 23 can be configured as individual circuits or integrated into a single integrated circuit.

[0034] The communication interface 21 has an interface circuit for connecting the ECU 8 to the in-vehicle network. Furthermore, whenever the communication interface 21 receives positioning information from the GPS receiver 2, it sends the positioning information to the processor 23. Additionally, whenever the communication interface 21 receives images from cameras 3-1 and 3-2, it sends the received images to the processor 23. Furthermore, whenever the communication interface 21 receives a ranging signal from the ranging sensor 4, it sends the received ranging signal to the processor 23. Moreover, the communication interface 21 sends a high-precision map read from the storage device 7 to the processor 23. Finally, the communication interface 21 outputs information or signals received from the processor 23 and intended for the user interface 6 to the user interface 6 via the in-vehicle network.

[0035] Memory 22 is another example of a storage unit, such as a volatile semiconductor memory and a non-volatile semiconductor memory. Furthermore, memory 22 stores various data used in the vehicle control processing executed by the processor 23 of ECU 8. For example, memory 22 stores high-precision maps, parameters indicating the focal length, field of view, shooting direction, and mounting position of cameras 3-1 and 3-2, and the ranging range of the ranging sensor 4. Additionally, memory 22 stores a set of parameters for determining an object detection detector used to detect other vehicles traveling around vehicle 10. Furthermore, memory 22 temporarily stores sensor signals such as images and ranging signals, as well as the positioning results of its own position received by GPS receiver 2. Moreover, memory 22 temporarily stores various data generated during the vehicle control processing.

[0036] The processor 23 has one or more CPUs (Central Processing Units) and their peripheral circuitry. The processor 23 may also include other arithmetic circuitry such as logic units, numerical processing units, or graphics processing units. Furthermore, the processor 23 performs vehicle control processing for the vehicle 10.

[0037] Figure 3 This is a functional block diagram of the processor 23 related to vehicle control processing. The processor 23 includes a detection unit 31, a level control unit 32, and a vehicle control unit 33. These units of the processor 23 are, for example, functional modules implemented by a computer program operating on the processor 23. Alternatively, these units of the processor 23 may also be dedicated arithmetic circuits provided on the processor 23.

[0038] During the application of automated driving control at either the first or second driving control level to the vehicle 10, the detection unit 31 determines, based on the sensor signals obtained from each sensor, whether there is a risk of collision between the vehicle 10 and objects present in its vicinity. In this embodiment, the detection unit 31 detects objects present in the vicinity of the vehicle 10 based on each image obtained from cameras 3-1 and 3-2, and each ranging signal obtained from the ranging sensor 4. Furthermore, the detection unit 31 determines whether there is a risk of collision between the detected objects and the vehicle 10.

[0039] In this embodiment, the object to be detected is an object that poses a risk of collision with vehicle 10. Examples include moving objects such as other vehicles or pedestrians, billboards announcing roadworks, or road debris—structures existing on the road where vehicle 10 travels. Additionally, the object to be detected may also include road markings such as lane markings, speedometer markings, guardrails, or curbs—stationary objects existing on or around the road where vehicle 10 travels that affect its movement. Hereinafter, the object to be detected will sometimes be simply referred to as an object or target object.

[0040] For example, the detection unit 31 detects objects by inputting images acquired from camera 3-1 or camera 3-2 into an object detection recognizer. As such a recognizer, the detection unit 31 can use a deep neural network (DNN) with a convolutional neural network (CNN) architecture, such as Single Shot MultiBox Detector (SSD) or Faster R-CNN. Alternatively, the detection unit 31 can also use a recognizer based on other machine learning methods, such as the AdaBoost recognizer. Such a recognizer is pre-learned using a predetermined learning method, such as backpropagation, to detect objects existing around the vehicle 10 from the image. The recognizer outputs information on the input image that identifies the object region including the detected object, as well as information indicating the type of the detected object.

[0041] In addition, the detection unit 31 detects lane markings and the lane in which the vehicle 10 is traveling (hereinafter referred to as the "current lane"). In this embodiment, the detection unit 31 detects lane markings by inputting images obtained from camera 3-1 or camera 3-2 into the aforementioned recognizer. Furthermore, the detection unit 31 can designate the lane corresponding to the area enclosed by the two lane markings located on either side of the horizontal center of the image and closest to that center as the current lane. Then, the detection unit 31 infers the position of the vehicle 10 when each image was generated. At this time, the detection unit 31 determines the position of the vehicle 10 by comparing the image and the high-precision map, finding the position of the vehicle 10 that best matches the ground features shown in the image and the corresponding ground features shown in the high-precision map.

[0042] Furthermore, the detection unit 31 detects objects present around the vehicle 10 based on the ranging signal. Alternatively, even in this case, the detection unit 31 can detect objects around the vehicle 10 by inputting the ranging signal into a recognizer that has been pre-learned to detect objects around the vehicle 10 from the ranging signal. Or, the detection unit 31 may detect objects around the vehicle 10 using other methods for detecting objects from the ranging signal.

[0043] The detection unit 31 determines whether there is a risk of collision between the detected object and the vehicle 10. To this end, the detection unit 31 tracks each detected object and predicts the trajectory of that object after a predetermined time. At this time, the detection unit 31 detects the object based on sensor signals previously generated by the sensor that generated the sensor signal indicating the detected object. For example, when detecting an object of interest from the latest image generated by the camera 3-1, the detection unit 31 tracks the object from a series of images generated by the camera 3-1 over a certain period of time. Similarly, when detecting an object of interest from the latest ranging signal generated by the ranging sensor 4, the detection unit 31 tracks the object from a series of ranging signals generated by the ranging sensor 4 over a certain period of time.

[0044] The detection unit 31 calculates the distance between the object and the predicted position of the vehicle 10 at each future time point on the predicted trajectory (hereinafter sometimes referred to as the predicted position) for each object being tracked. Furthermore, if the distance between any object being tracked and the predicted position of the vehicle 10 at any time point is below a predetermined threshold, the detection unit 31 determines that there is a risk of collision between the object being tracked and the vehicle 10.

[0045] The following describes the tracking of objects detected from the image generated by camera 3-1 and the prediction of their trajectories. The detection unit 31 also tracks objects detected from the image generated by camera 3-2 using the same method described below, and predicts their trajectories based on the tracking results.

[0046] The detection unit 31 tracks the object represented in the object region by applying an optical flow field-based tracking process, such as the Lucas-Kanade method, to the object region of interest in the latest image and the object region in the past image. Therefore, the detection unit 31 can extract multiple feature points from the object region by applying a feature point extraction filter, such as SIFT or Harris Operator, to the object region of interest. Furthermore, the detection unit 31 calculates the optical flow field by determining the corresponding points in the object region of the past image for each of the multiple feature points, according to the applied tracking method. Alternatively, the detection unit 31 can also track the object represented in the object region by applying other tracking methods used in tracking moving objects detected from images to the object region of interest in the latest image and the object region in the past image.

[0047] The detection unit 31 uses information such as the optical axis direction, focal length, and set height of the camera 3-1 to perform viewpoint transformation processing on the object being tracked, thereby transforming the object's in-image coordinates into coordinates on a bird's-eye view image based on the position of the camera 3-1 (bird's-eye view coordinates). Furthermore, for each bird's-eye view coordinate obtained from a series of images acquired during tracking, the detection unit 31 performs prediction processing using a Kalman filter or particle filter, etc., based on the position of the vehicle 10 when this series of images was generated. Thus, the detection unit 31 can predict the object's trajectory after a predetermined time.

[0048] Furthermore, regarding objects detected from the ranging signals, the detection unit 31 compares a portion of the ranging signals at a certain point in time that includes the azimuth angles of the object of interest, with the subsequently obtained ranging signals. Moreover, the detection unit 31 determines that the object of interest is included in a portion of the subsequently obtained ranging signals whose azimuth angles have a consistency level with that portion exceeding a predetermined threshold. In this case, the detection unit 31 calculates the consistency level, for example, by summing the differences in distances between the corresponding azimuth angles in the two sets of azimuth angles being compared. The detection unit 31 tracks objects detected from the ranging signals by repeatedly performing the above processing on a series of ranging signals obtained sequentially. The detection unit 31 also performs the same prediction processing as described above on the position of the vehicle 10 at the time each ranging signal is generated, and the azimuth and distance from the vehicle 10 to the object in each ranging signal, thus predicting the predicted trajectory of the object.

[0049] Then, the detection unit 31 calculates the predicted position of the vehicle 10 at each time point after the predetermined time according to the latest predetermined driving trajectory set by the vehicle control unit 33.

[0050] Furthermore, the detection unit 31 can determine a risk of collision between the object and the vehicle 10 if the object detected by any sensor is of a type equivalent to a stationary object and is located in front of the vehicle 10 in the same lane. Additionally, when detecting an object in the same lane based on an image generated by camera 3-1 or camera 3-2, the detection unit 31 determines whether the object is a stationary object based on the type of object output by the recognizer. Furthermore, when detecting an object in the same lane based on a ranging signal, if the object's position remains unchanged as a result of tracking it as described above, the detection unit 31 determines that the object is a stationary object.

[0051] In this case, the detection unit 31 estimates the distance from the vehicle 10 to the stationary object in the current lane. When the stationary object is detected based on a ranging signal, the detection unit 31 sets the distance measured using the ranging signal regarding the location of the stationary object as the distance to the stationary object. Alternatively, when a stationary object is detected from an image, the detection unit 31 estimates the distance to the stationary object based on the width of the current lane at the location of the stationary object in the image, the width of the current lane at the current position of the vehicle 10 shown in the high-precision map, and camera parameters such as focal length.

[0052] The detection unit 31 notifies the grade control unit 32 of the number and type of sensors (camera 3-1, camera 3-2, range sensor 4) that generate sensor signals indicating an object deemed to pose a risk of collision with vehicle 10. Additionally, the detection unit 31 notifies the vehicle control unit 33 of the current lane and the position of vehicle 10. Furthermore, the detection unit 31 notifies the vehicle control unit 33 of the estimated distance up to a stationary object detected in the current lane. Hereinafter, objects deemed to pose a risk of collision with vehicle 10 will sometimes be simply referred to as hazardous objects.

[0053] When the first driving control level is applied to the vehicle 10, the level control unit 32 controls the level of automatic driving control applied to the vehicle 10 based on the number of sensors that generate sensor signals indicating the presence of a dangerous object whenever it is notified of such a number.

[0054] When the level control unit 32 detects a hazardous object based on the sensor signal from any one of the multiple sensors, it shifts the level of autonomous driving control applied to the vehicle 10 from the first driving control level to the second driving control level. In this embodiment, the level control unit 32 shifts from the first driving control level to the second driving control level when a hazardous object is detected based on any one of the images obtained by camera 3-1, images obtained by camera 3-2, and distance measurement signals obtained by distance measurement sensor 4. This shift requires the driver to monitor the surroundings of the vehicle 10, making it easier for the driver to detect hazardous objects. Therefore, when the risk of a collision between a hazardous object and the vehicle 10 increases, the driver can immediately take over driving control of the vehicle 10. Furthermore, if a hazardous object is detected only from the sensor signal of any one of the multiple sensors, there is a possibility that other objects easily misidentified as hazardous objects may be falsely detected as such. If autonomous driving control is terminated due to such false detection, the driver's convenience is compromised. However, in this embodiment, if a dangerous object is detected from the sensor signal of only one sensor, the autonomous driving control level shifts to the second driving control level, so the driver does not need to operate the steering wheel or accelerator. Therefore, the driver's convenience is not seriously compromised.

[0055] Furthermore, when the level control unit 32 detects a hazardous object based on sensor signals from two or more of the multiple sensors, it shifts the applied autonomous driving control level from the first driving control level to the third driving control level. In this embodiment, when a hazardous object is detected based on two or more of the images obtained from camera 3-1, camera 3-2, and the ranging signal obtained from the ranging sensor 4, the level control unit 32 shifts from the first driving control level to the third driving control level. When a hazardous object is detected based on sensor signals from two or more sensors, the possibility of false detection of the hazardous object is low. Therefore, by shifting the applied autonomous driving control level to the third driving control level, the driver can control the vehicle 10 to avoid collisions with the hazardous object.

[0056] Furthermore, if no dangerous object is detected from the sensor signals of any of the sensors, the level control unit 32 continues as is for the vehicle 10 without changing the level of the currently applied autonomous driving control.

[0057] According to a variation, the level control unit 32 can also, when applying a second driving control level to the vehicle 10, shift the applied automatic driving control level to a third driving control level if a dangerous object is detected based on sensor signals from two or more sensors. Thus, similar to when the applied automatic driving control level shifts from a first driving control level to a third driving control level, the driver can control the vehicle 10 to avoid collisions with dangerous objects.

[0058] According to other variations, the level control unit 32 can also, when applying the second driving control level in the vehicle 10, switch the applied automatic driving control level to the first driving control level if no dangerous object is detected from the sensor signals of any sensor within a predetermined period. Furthermore, the predetermined period can be, for example, a few seconds to tens of seconds. Thus, the ECU 8 can further improve driver convenience while ensuring the safety of the vehicle 10.

[0059] When the level of autonomous driving control applied to vehicle 10 is changed, the level control unit 32 notifies the driver of the change via the user interface 6. For example, when changing the level of autonomous driving control applied to vehicle 10 to the second driving control level, the level control unit 32 notifies the driver of a warning requiring monitoring of the surroundings of vehicle 10 via the user interface 6. At this time, the level control unit 32 displays a message or icon indicating the warning on a display device of the user interface 6, or illuminates or flashes a light source corresponding to the warning. Alternatively, the level control unit 32 causes the speaker of the user interface 6 to output an audible signal indicating the warning. Or, the level control unit 32 causes a vibration device of the user interface 6 to vibrate according to a vibration mode (vibration period or vibration intensity) corresponding to the warning. Furthermore, when the user interface 6 has two or more of the above-mentioned devices, the level control unit 32 may notify the driver of the warning via each of the two or more devices or any one of the devices.

[0060] Furthermore, when the level of autonomous driving control applied to vehicle 10 is shifted to the third driving control level, the level control unit 32 notifies the driver via the user interface 6 of a warning indicating that driving control of vehicle 10 is being transferred to the driver. At this time, the level control unit 32 displays a message or icon indicating the warning on a display device of the user interface 6, or illuminates or flashes a light source corresponding to the warning. Alternatively, the level control unit 32 causes the speaker of the user interface 6 to output an audible signal indicating the warning. Or, the level control unit 32 causes a vibration device of the user interface 6 to vibrate according to a vibration mode corresponding to the warning. Furthermore, if the user interface 6 has two or more of the aforementioned devices, the level control unit 32 may notify the driver of the warning via each of the two or more devices or any one of the devices.

[0061] Furthermore, when the level of autonomous driving control applied to vehicle 10 is shifted to the first driving control level, the level control unit 32 notifies the driver via the user interface 6 to release the monitoring obligation around vehicle 10. At this time, the level control unit 32 displays a message or icon indicating the release of the monitoring obligation on the display device of the user interface 6, or illuminates or flashes a light source corresponding to the message. Alternatively, the level control unit 32 causes the speaker of the user interface 6 to output an audio signal indicating the message. Or, the level control unit 32 causes a vibration device of the user interface 6 to vibrate according to a vibration mode corresponding to the message. Furthermore, if the user interface 6 has two or more of the aforementioned devices, the level control unit 32 can also notify the driver of the message via each of the two or more devices or any one of the devices.

[0062] Figure 4A as well as Figure 4B These are graphs illustrating an example of the relationship between the number of sensors associated with the detection of hazardous objects and the migration of the level of autonomous driving control applied to vehicle 10.

[0063] exist Figure 4A In the example shown, a dangerous object 401 is detected in image 411 obtained by camera 3-2. However, the dangerous object 401 is far from the vehicle 10, so although it is captured in image 412 obtained by camera 3-1, it is too small in image 412. Therefore, the dangerous object 401 is not detected in image 412. Furthermore, the dangerous object 401 is also not detected in the ranging signal obtained by ranging sensor 4. Therefore, the level control unit 32 shifts the level of automatic driving control applied to the vehicle 10 from the first driving control level to the second driving control level.

[0064] exist Figure 4B In the example shown, a dangerous object 402 is detected not only from image 421 obtained through camera 3-2, but also from image 422 obtained through camera 3-1. Therefore, the level control unit 32 shifts the level of automatic driving control applied to vehicle 10 from the first driving control level to the third driving control level.

[0065] After a predetermined time has elapsed since the notification of the aforementioned automatic driving control level change, the level control unit 32 causes the automatic driving control level applied to the vehicle 10 to change to the notification driving control level.

[0066] Whenever the level of automatic driving control applied to vehicle 10 is changed, the level control unit 32 notifies the vehicle control unit 33 of the changed control level.

[0067] The vehicle control unit 33 controls the vehicle 10 according to the level of automatic driving control applied to the vehicle 10.

[0068] For example, when the level of autonomous driving control applied to the vehicle 10 is Level 1 or Level 2, the vehicle control unit 33 sets a predetermined path (hereinafter referred to as the predetermined driving path) for the vehicle 10 to travel until a predetermined time thereafter. Furthermore, the predetermined driving path is, for example, represented as a set of the target positions of the vehicle 10 at various times when the vehicle 10 is traveling within a predetermined interval.

[0069] The vehicle control unit 33 refers to a high-precision map to set a predetermined driving path so that the vehicle 10 travels along a predetermined route to its destination. For example, if the vehicle is traveling in the current lane along the predetermined driving path, the vehicle control unit 33 sets the predetermined driving path along the center of the current lane. Furthermore, if the predetermined driving path indicates a point where the vehicle will turn from the currently traveling road to a branch road at a predetermined distance ahead, the vehicle control unit 33 sets the predetermined driving path by performing a lane change from the current lane to a lane that allows access to the branch road.

[0070] After setting a predetermined driving path, the vehicle control unit 33 controls various parts of the vehicle 10 to drive the vehicle 10 along that predetermined driving path. For example, the vehicle control unit 33 calculates a target acceleration for the vehicle 10 based on the predetermined driving path and the current vehicle speed of the vehicle 10 measured by a vehicle speed sensor (not shown), and sets the accelerator opening or braking amount to achieve that target acceleration. Furthermore, the vehicle control unit 33 calculates the fuel injection amount based on the set accelerator opening and outputs a control signal corresponding to that fuel injection amount to the fuel injection device of the engine of the vehicle 10. Alternatively, the vehicle control unit 33 controls the power supply device to the motor that drives the vehicle 10 to supply power corresponding to the set accelerator opening. Or, the vehicle control unit 33 outputs a control signal corresponding to the set braking amount to the brakes of the vehicle 10. Then, the vehicle control unit 33 calculates the steering angle of the vehicle 10 to make the vehicle 10 travel along the predetermined driving path based on the predetermined driving path and the current position of the vehicle 10, and outputs the control signal corresponding to the steering angle to the actuator (not shown) that controls the steering wheel of the vehicle 10.

[0071] Furthermore, even after a predetermined period has elapsed since the driver was notified to switch the level of autonomous driving control applied to vehicle 10 to the second or third driving control level, the driver may sometimes fail to perform the required action. In such cases, the vehicle control unit 33 can also control vehicle 10 to stop. Additionally, for example, it determines whether the driver has performed the required action based on in-vehicle images generated by a driver monitoring camera (not shown) installed in the vehicle 10's interior, or by detection signals generated by a touch sensor (not shown) installed on the steering wheel. For example, the vehicle control unit 33 detects the driver's gaze direction or face orientation from the in-vehicle image and determines whether the driver is monitoring the front of vehicle 10 based on the detection result. Furthermore, if the driver is monitoring the front of vehicle 10, the vehicle control unit 33 determines that the driver is performing the necessary monitoring of the vehicle's surroundings for the second driving control level application. Additionally, when the vehicle control unit 33 determines that the driver is monitoring the front of vehicle 10 based on the in-vehicle image and receives a detection signal from the touch sensor indicating that the driver is holding the steering wheel, it determines that the driver has completed preparations for driving control of vehicle 10. That is, the vehicle control unit 33 determines that the driver has completed the preparations required for the application of the third driving control level. Furthermore, the vehicle control unit 33 detects the driver's facial features by inputting the in-vehicle image into a pre-learned recognizer that detects facial features. For example, the vehicle control unit 33 can use a DNN with a CNN-type architecture as such a recognizer. Moreover, the vehicle control unit 33 matches the detected facial features to the 3D model while making various changes to its orientation, thereby detecting the orientation of the face in the 3D model that best matches each feature, and using the detected face orientation as the driver's face orientation. Additionally, the vehicle control unit 33 detects the pupils and corneal reflection images from the light source of the driver's monitoring camera by performing template matching on the in-vehicle image or by inputting the in-vehicle image into the recognizer. Furthermore, the vehicle control unit 33 detects the driver's gaze direction based on the positional relationship between the center of gravity of the pupils and the corneal reflection image.

[0072] Figure 5 This is a flowchart of the vehicle control processing executed by the processor 23. During the application of Level 1 autonomous driving control to the vehicle 10, the processor 23 executes the vehicle control processing according to the following flowchart at a predetermined cycle.

[0073] The detection unit 31 of the processor 23 detects objects around the vehicle 10 based on the sensor signals obtained by each sensor installed in the vehicle 10, and determines whether the detected object is a dangerous object (step S101).

[0074] The level control unit 32 of the processor 23 determines whether there are two or more sensors among the multiple sensors installed in the vehicle 10 that generate sensor signals indicating that a dangerous object has been detected (step S102). If there are two or more sensors that generate sensor signals indicating that a dangerous object has been detected (step S102 - "Yes"), the level control unit 32 shifts the level of autonomous driving control applied to the vehicle 10 from the first driving control level to the third driving control level (step S103).

[0075] On the other hand, if the number of sensors generating the sensor signal indicating that a dangerous object has been detected is less than two (step S102 - "Yes"), the level control unit 32 determines whether the number of sensors generating the sensor signal indicating that a dangerous object has been detected is one (step S104). If the number of sensors generating the sensor signal indicating that a dangerous object has been detected is one (step S104 - "Yes"), the level control unit 32 shifts the level of the automatic driving control applied to the vehicle 10 from the first driving control level to the second driving control level (step S105).

[0076] After step S103 or S105, the level control unit 32 notifies the driver of the change in the level of automatic driving control via the user interface 6 (step S106).

[0077] In step S104, if no dangerous object is detected from the sensor signal of any sensor (step S104 - "No"), the level control unit 32 continues as is for the vehicle 10 without changing the level of the currently applied automatic driving control (step S107).

[0078] After step S106 or step S107, the vehicle control unit 33 of the processor 23 controls the vehicle 10 according to the applied level of autonomous driving control (step S108). Then, the processor 23 terminates the vehicle control process.

[0079] As explained above, this vehicle control device controls the level of automated driving control applied to the vehicle based on the number of sensors among the multiple sensors mounted on the vehicle that generate sensor signals indicating the detection of a hazard. Specifically, when a hazard is detected based on sensor signals from two or more sensors, the vehicle control device lowers the level of automated driving control to a level requiring driver intervention (accelerator or steering wheel). Conversely, when a hazard is detected based on sensor signals from only one sensor, the vehicle control device lowers the applied level of automated driving control to a level that requires driver monitoring of the surroundings but not driver intervention (accelerator or steering wheel). Therefore, this vehicle control device can reduce the risk of a vehicle collision while decreasing the frequency of applying driving control levels requiring driver intervention, thus improving driver convenience.

[0080] According to a variation, the level control unit 32 can also request the driver to monitor the surroundings of the vehicle 10 and keep the steering wheel when the level of automatic driving control applied to the vehicle 10 is shifted from the first driving control level to the second driving control level. In this case, the level control unit 32 can notify the driver of the warning requesting to monitor the surroundings of the vehicle 10 and keep the steering wheel via the user interface 6.

[0081] According to other variations, the number of sensors mounted on the vehicle capable of detecting objects around the vehicle can also be two. For example, in the above embodiment, either camera 3-1 or camera 3-2 can be omitted, or the ranging sensor 4 can also be omitted. Even in this case, the vehicle control device can achieve the same effect as the above embodiment.

[0082] The computer program that implements the function of the processor 23 of the ECU8 as described in the above embodiments or variations may also be provided in the form of a computer-readable and portable recording medium such as a semiconductor memory, a magnetic recording medium or an optical recording medium.

[0083] As described above, those skilled in the art can make various modifications within the scope of this invention and in accordance with the implementation method.

Claims

1. A vehicle control device capable of automatically controlling a vehicle according to any one of a first level, a second level, and a third level, wherein the first level does not require the driver to operate the accelerator and steering wheel or monitor the vehicle's surroundings; the second level requires the driver to monitor the vehicle's surroundings but does not require operation of the accelerator and steering wheel; and the third level requires the driver to operate at least one of the accelerator and steering wheel, wherein... The vehicle control device has: The detection unit detects objects around the vehicle based on sensor signals from the sensors mounted on the vehicle, and determines whether there is a risk of collision between the detected objects and the vehicle. as well as When the vehicle is using Level 1 autonomous driving control, if an object deemed to pose a risk of collision with the vehicle is detected based on the sensor signal of one of the plurality of sensors, the level of autonomous driving control applied to the vehicle is shifted from Level 1 to Level 2. On the other hand, if an object deemed to pose a risk of collision with the vehicle is detected based on the sensor signals of two or more of the plurality of sensors, the level of autonomous driving control applied to the vehicle is shifted from Level 1 to Level 3.

2. The vehicle control device according to claim 1, wherein, When the vehicle is using the Level 2 autonomous driving control, if an object deemed to pose a risk of collision with the vehicle is detected based on sensor signals from two or more of the plurality of sensors, the level control unit shifts the level of autonomous driving control applied to the vehicle from the Level 2 to the Level 3.

3. The vehicle control device according to claim 1 or 2, wherein, When the vehicle is using the second level of automated driving control, if no object deemed to pose a risk of collision with the vehicle is detected based on the sensor signals of any of the plurality of sensors during a predetermined period, the level control unit shifts the level of automated driving control applied to the vehicle from the second level to the first level.

4. A vehicle control method capable of automatically controlling a vehicle according to any one of a first level, a second level, and a third level, wherein the first level does not require the driver to operate the accelerator and steering wheel or monitor the vehicle's surroundings; the second level requires the driver to monitor the vehicle's surroundings but does not require operation of the accelerator and steering wheel; and the third level requires the driver to operate at least one of the accelerator and steering wheel, wherein... The vehicle control method includes: Regarding each of the multiple sensors mounted on the vehicle that are capable of detecting objects around the vehicle, the sensors detect objects around the vehicle based on sensor signals from the sensors, and determine whether there is a risk of collision between the detected objects and the vehicle. When the vehicle is under Level 1 autonomous driving control, if an object deemed to pose a risk of collision with the vehicle is detected based on a sensor signal from one of the plurality of sensors, the level of autonomous driving control applied to the vehicle is shifted from Level 1 to Level 2; and When an object deemed to pose a risk of collision with the vehicle is detected based on sensor signals from two or more of the plurality of sensors, the level of autonomous driving control applied to the vehicle is shifted from Level 1 to Level 3.

5. A vehicle control computer program capable of performing automated driving control of a vehicle according to any one of a first level, a second level, and a third level, wherein the first level does not require the driver to operate the accelerator and steering wheel or monitor the vehicle's surroundings; the second level requires the driver to monitor the vehicle's surroundings but does not require operation of the accelerator and steering wheel; and the third level requires the driver to operate at least one of the accelerator and steering wheel, wherein... The vehicle control computer program is used to cause the processor mounted on the vehicle to execute: Regarding each of the multiple sensors mounted on the vehicle that are capable of detecting objects around the vehicle, the sensors detect objects around the vehicle based on sensor signals from the sensors, and determine whether there is a risk of collision between the detected objects and the vehicle. When the vehicle is using the first level of automated driving control, if an object deemed to pose a risk of collision with the vehicle is detected based on the sensor signal of one of the plurality of sensors, the level of automated driving control applied to the vehicle is shifted from the first level to the second level. as well as When an object deemed to pose a risk of collision with the vehicle is detected based on sensor signals from two or more of the plurality of sensors, the level of autonomous driving control applied to the vehicle is shifted from Level 1 to Level 3.

Images