Driving control device, driving control method, and computer program for driving control

Abstract

To provide a running control device capable of appropriately switching from automatic driving control of running of a vehicle to manual driving control when a two-wheeled vehicle is detected in the vicinity of the vehicle.SOLUTION: A running control device for a vehicle that is capable of traveling on a road via automatic driving control detects a two-wheeled vehicle, including a bicycle, positioned in the vicinity of the vehicle from surroundings data representing the surroundings of the vehicle generated by a surrounding sensor mounted on the vehicle, and determines whether or not the road of interest on which the vehicle was traveling when the surroundings data in which the two-wheeled vehicle was detected is a bicycle traveling road where traveling of bicycles is permitted. If it is determined that the road of interest is a bicycle traveling road, the running control device provides a driver with an operation change request for requesting a change of operation from automatic driving control to manual driving control which requires driving operation by the driver of the vehicle in control of the vehicle. If it is determined that the road of interest is not a bicycle traveling road, the running control device does not provide the driver with an operation change request.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present disclosure relates to a driving control device, a driving control method, and a computer program for driving control that control the driving of a vehicle.

Background Art

[0002] On the road, a vehicle with two wheels (hereinafter referred to as a "two-wheeled vehicle") and a vehicle other than a two-wheeled vehicle may travel mixedly. Since a bicycle among two-wheeled vehicles has a lower vehicle speed than other vehicles, the speed difference between a bicycle and the own vehicle is larger than the speed difference between other vehicles including two-wheeled vehicles other than a bicycle and the own vehicle. Therefore, when a bicycle is traveling around the vehicle, careful attention is required in the driving control of the vehicle.

[0003] Patent Document 1 describes a vehicle control device that transitions from a second driving state having a predetermined automation rate to a first driving state having an automation rate lower than that of the second driving state when a specific object such as a person or a bicycle is recognized in a specific area such as a highway.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Since a bicycle and a non-bicycle two-wheeled vehicle have a similar shape, it is not easy to surely distinguish between a bicycle and a non-bicycle two-wheeled vehicle when detecting a moving object around the vehicle. When a two-wheeled vehicle is detected around the vehicle and the driving of the vehicle is uniformly changed from automatic driving control to manual control, even if the detected two-wheeled vehicle is other than a bicycle, the driving by automatic control will not be performed, and the usability will decrease.

[0006] The purpose of this disclosure is to provide a driving control device that can appropriately switch from automatic driving control to manual driving control of a vehicle when a motorcycle is detected in the vicinity of the vehicle. [Means for solving the problem]

[0007] The driving control device according to this disclosure is a driving control device for a vehicle capable of driving on a road by automatic driving control, and comprises: a detection unit that detects two-wheeled vehicles, including bicycles, located around the vehicle from surrounding data representing the area around the vehicle generated by surrounding sensors mounted on the vehicle; a determination unit that determines whether the target road on which the vehicle was traveling when the surrounding data in which the two-wheeled vehicle was detected was generated is a bicycle road where bicycle travel is permitted; and a notification unit that, if it is determined that the target road is a bicycle road, notifies the driver of a driver change request requesting a change of control from automatic driving control to manual driving control requiring the driver to operate the vehicle, and does not notify the driver of a driver change request if it is determined that the target road is not a bicycle road.

[0008] In the driving control device according to this disclosure, it is preferable that the notification unit does not notify the driver of a request for a driver change, even if the target road is a bicycle path, if a motorcycle located behind the vehicle is detected from the surrounding data and the distance from the vehicle to the motorcycle is increasing over time.

[0009] The driving control device according to this disclosure further comprises an identification unit that identifies whether or not a two-wheeled vehicle detected from surrounding data is a bicycle, and it is preferable that the notification unit does not notify the driver of a request for a change of driver if the two-wheeled vehicle is identified as not being a bicycle, even if the target road is a bicycle road.

[0010] The driving control method relating to this disclosure is a driving control method for a vehicle capable of driving on a road by automatic driving control, and includes detecting two-wheeled vehicles, including bicycles, located around the vehicle from surrounding data representing the area around the vehicle generated by surrounding sensors mounted on the vehicle, determining whether the target road on which the vehicle was traveling when the surrounding data in which the two-wheeled vehicles were detected was generated is a bicycle road where bicycle travel is permitted, notifying the driver of a change of driver request if it is determined that the target road is a bicycle road, and notifying the driver of a change of driver request if it is determined that the target road is not a bicycle road.

[0011] The computer program for driving control according to this disclosure is a computer program for driving control that is executed on a processor installed in a vehicle capable of driving on a road by automatic driving control, and it detects two-wheeled vehicles, including bicycles, located around a moving vehicle from surrounding data representing the area around the vehicle generated by surrounding sensors installed in the vehicle, determines whether the target road on which the vehicle was traveling when the surrounding data in which the two-wheeled vehicle was detected was generated is a bicycle road where bicycle travel is permitted, and if it is determined that the target road is a bicycle road, it notifies the driver of a driver change request requesting a change of control from automatic driving control to manual driving control requiring the driver to operate the vehicle, and if it is determined that the target road is not a bicycle road, it does not notify the driver of a driver change request.

[0012] The driving control device described herein can appropriately switch from automatic driving control to manual driving control when a motorcycle is detected in the vicinity of the vehicle. [Brief explanation of the drawing]

[0013] [Figure 1] This is a schematic diagram of a vehicle equipped with a vehicle control system. [Figure 2] This is a schematic diagram of the ECU hardware. [Figure 3] This is a functional block diagram of the processor in the ECU. [Figure 4] This is a schematic diagram illustrating an example of the surrounding conditions of a vehicle. [Figure 5] This is a schematic diagram illustrating an example of a surrounding image that will be generated. [Figure 6] This is a flowchart of the vehicle control process. [Modes for carrying out the invention]

[0014] The following describes in detail a vehicle control system that can appropriately switch from automatic driving control to manual driving control when a motorcycle is detected in the vicinity of the vehicle, with reference to the drawings. The vehicle control system is installed in vehicles that are capable of driving on roads using automatic driving control. The vehicle control system detects motorcycles located in the vicinity of the vehicle from the surrounding data representing the vehicle's surroundings, which is generated by surrounding sensors installed on the vehicle. The vehicle control system also determines whether the road the vehicle was traveling on when the surrounding data in which the motorcycle was detected was generated is a bicycle path where bicycle travel is permitted. If the vehicle control system determines that the road is a bicycle path, it notifies the driver of a driver change request, which requires a switch from automatic driving control to manual driving control requiring the driver's operation. If the vehicle control system determines that the road is not a bicycle path, it does not notify the driver of a driver change request.

[0015] Figure 1 is a schematic diagram of a vehicle on which a driving control device is installed.

[0016] Vehicle 1 includes a surrounding camera 2, a meter display 3, a GNSS receiver 4 (Global Navigation Satellite System), a storage device 5, and an ECU 6 (Electronic Control Unit). The ECU 6 is an example of a driving control device. The surrounding camera 2, the meter display 3, the GNSS receiver 4, the storage device 5, and the ECU 6 are communicably connected via an in-vehicle network compliant with a standard such as a controller area network.

[0017] The surrounding camera 2 is an example of a surrounding sensor for generating surrounding data representing the surrounding situation of the vehicle 1. The surrounding camera 2 includes a two-dimensional detector composed of an array of photoelectric conversion elements sensitive to visible light, such as a CCD or a C-MOS, and an imaging optical system for imaging an image of an area to be photographed on the two-dimensional detector. The surrounding camera 2 includes a front surrounding camera 2-1 and a rear surrounding camera 2-2. The front surrounding camera 2-1 is arranged, for example, at the upper front part inside the vehicle cabin facing forward, and the rear surrounding camera 2-2 is arranged, for example, at the upper rear part inside the vehicle cabin facing backward. The surrounding camera 2 photographs the surrounding situation of the vehicle 1 through the front windshield or the rear windshield at a predetermined photographing period (for example, 1 / 30 second to 1 / 10 second), and outputs a surrounding image representing the surrounding situation as surrounding data.

[0018] The meter display 3 is an example of an output device and has, for example, a liquid crystal display. The meter display 3 visually displays a message representing a driving change request for requesting a driving change from automatic control to manual control to the driver by light according to a signal received from the ECU 6 via the in-vehicle network.

[0019] The vehicle 1 may be provided with a speaker (not shown) capable of outputting an audio message to the driver by emitting sound, or a seat vibrator (not shown) capable of outputting a tactile message to the driver by vibrating the seat, as an output device.

[0020] [[ID=The GNSS receiver 4 receives GNSS signals from GNSS satellites at a predetermined period, and measures the self-position of the vehicle 1 based on the received GNSS signals. The GNSS receiver 4 outputs, at a predetermined period, a positioning signal representing the measurement result of the self-position of the vehicle 1 based on the GNSS signals to the ECU 6 via the in-vehicle network.

[0021] The storage device 5 is an example of a storage unit, and has, for example, a hard disk device or a non-volatile semiconductor memory. The storage device 5 stores map data including information on features such as lane dividing lines and information on traffic rules such as the types of vehicles permitted to travel on the road, in association with positions.

[0022] The ECU 6 automatically controls at least one of the acceleration / deceleration and steering of the vehicle 1. Further, the ECU 6 detects a two-wheeled vehicle located around the vehicle 1 from the surrounding image generated by the surrounding camera 2, and determines whether the target road on which the vehicle 1 was traveling when the surrounding image in which the two-wheeled vehicle was detected was generated is a bicycle lane. When it is determined that the target road is a bicycle lane, the ECU 6 notifies the driver of a driving change request. When it is determined that the target road is not a bicycle lane, the ECU 6 does not notify the driver of a driving change request.

[0023] Figure 2 is a hardware schematic diagram of the ECU 6. The ECU 6 includes a communication interface 61, a memory 62, and a processor 63.

[0024] The communication interface 61 is an example of a communication unit, and has a communication interface circuit for connecting the ECU 6 to the in-vehicle network. The communication interface 61 supplies the received data to the processor 63. Further, the communication interface 61 outputs the data supplied from the processor 63 to the outside.

[0025] Memory 62 is another example of a storage unit and includes both volatile and non-volatile semiconductor memory. Memory 62 stores various data used in processing by the processor 63, such as a set of parameters (number of layers, layer configuration, kernel, weight coefficients, etc.) for defining a neural network that operates as a classifier for detecting motorcycles from surrounding images. Memory 62 also stores driver change request messages that are sent to the driver to request a driver change. Furthermore, Memory 62 stores various application programs, such as a driving control program that executes driving control processing.

[0026] The processor 63 is an example of a control unit and has one or more processors and their peripheral circuits. The processor 63 may further have other arithmetic circuits such as a logic unit, a numerical unit, or a graphics processing unit.

[0027] Figure 3 is a functional block diagram of the processor 63 located in the ECU 6.

[0028] The processor 63 of the ECU6 has, as a functional block, a driving control unit 631, a detection unit 632, a determination unit 633, an identification unit 634, and a notification unit 635. Each of these parts of the processor 63 is a functional module implemented by a computer program stored in memory 62 and executed on the processor 63. The computer program that realizes the functions of each part of the processor 63 may be provided in the form of being recorded on a computer-readable portable recording medium such as semiconductor memory, a magnetic recording medium, or an optical recording medium. Alternatively, each of these parts of the processor 63 may be implemented in the ECU6 as an independent integrated circuit, a microprocessor, or firmware.

[0029] The driving control unit 631 performs automatic control of at least one of the following: acceleration, deceleration, and steering of the vehicle 1 traveling on the road.

[0030] The driving control unit 631 acquires map information representing the terrain, such as lanes, around the current position of the vehicle 1, which is determined by the positioning signal received from the GNSS receiver 4, from the storage device 5. The driving control unit 631 also detects moving objects located around the vehicle 1 by inputting the surrounding image generated by the surrounding camera 2 mounted on the vehicle 1 into a classifier that has been pre-trained to detect moving objects such as vehicles from the image. For each type of object to be detected in a predetermined area of ​​the surrounding image, the classifier outputs a confidence level indicating the likelihood that the area is classified as that type. The classifier then detects an object of that type in an area where the confidence level is higher than a predetermined threshold.

[0031] The classifier can be, for example, a convolutional neural network (CNN) having multiple convolutional layers connected in series from the input side to the output side. By pre-training the CNN using images containing moving objects such as vehicles as training data and following a predetermined learning method such as backpropagation, the CNN can function as a classifier that detects moving objects from surrounding images.

[0032] The classifier may be a CNN trained to detect features such as lane markings, road signs, and road markings in addition to moving objects from the surrounding image. The driving control unit 631 detects moving objects and features located around the vehicle 1 by inputting the surrounding image to such a classifier.

[0033] The driving control unit 631 tracks moving objects in the past over a predetermined period and estimates their future positions. The driving control unit 631 creates a driving path along lanes represented in the terrain around the vehicle 1, such that the distance between the estimated future position of the moving object around the vehicle 1 and the current path is greater than a certain amount. The driving control unit 631 then outputs control signals to the vehicle's driving mechanism (not shown) so that the vehicle 1 travels along the driving path. The driving mechanism includes, for example, an engine or motor to accelerate the vehicle 1, brakes to decelerate the vehicle 1, and a steering mechanism to steer the vehicle 1.

[0034] The detection unit 632 detects two-wheeled vehicles, including bicycles, located around the automatically controlled vehicle 1 from the surrounding image generated by the surrounding camera 2.

[0035] The detection unit 632 detects motorcycles located around vehicle 1 by inputting the surrounding image into a classifier that has been pre-trained to detect motorcycles. The classifier can be, for example, a CNN trained to detect motorcycles from the input image. Alternatively, a CNN trained to detect moving objects, including motorcycles, from the input image may be used as the classifier, and the detection of moving objects by the driving control unit 631 and the detection of motorcycles by the detection unit 632 may be performed together.

[0036] The determination unit 633 determines whether the target road on which vehicle 1 was traveling when the surrounding image in which the motorcycle was detected was generated is a bicycle path where bicycle travel is permitted.

[0037] The determination unit 633 obtains information from the storage device 5, which stores map data, that represents the traffic rules applicable to the target road, including the current position of vehicle 1, which is identified by the positioning signal received from the GNSS receiver 4. Then, based on the information representing the traffic rules obtained, the determination unit 633 determines whether or not the target road is a bicycle path.

[0038] The determination unit 633 may determine whether or not the target road is a bicycle lane based on features detected from the surrounding image. For example, if road markings or signs indicating a bicycle lane are detected from the surrounding image, the determination unit 633 can determine that the target road is a bicycle lane.

[0039] The identification unit 634 identifies whether the two-wheeled vehicle detected from the surrounding image is a bicycle or not.

[0040] The identification unit 634 compares, for example, the confidence level output from the classifier when a motorcycle is detected from the surrounding image with a confidence level threshold pre-stored in the memory 62. The identification unit 634 then identifies the detected motorcycle as a bicycle if the confidence level is greater than the confidence level threshold, and identifies the detected motorcycle as not a bicycle if the confidence level is not greater than the confidence level threshold.

[0041] If the notification unit 635 determines that the target road is a bicycle path, it notifies the driver of the request for a change of driver. If the notification unit 635 determines that the target road is not a bicycle path, it does not notify the driver of the request for a change of driver.

[0042] The notification unit 635 notifies the driver of vehicle 1 of the driver change request by, for example, displaying a driver change request image stored as a driver change request message in memory 62 on the meter display 3. The driver change request image includes text such as, "Automatic control of driving will end. Please take over driving." The driver change request image may also be an image requesting the driver to perform an action required of the driver of vehicle 1 for the driver change (for example, holding the steering wheel). The notification unit 635 may also notify the driver of the driver change request by playing a driver change request voice stored as a driver change request message in memory 62 through the speaker. Alternatively, the notification unit 635 may notify the driver of the driver change request by vibrating the seat vibrator with a driver change request vibration pattern stored as a driver change request message in memory 62.

[0043] If the identification unit 634 identifies a two-wheeled vehicle detected from the surrounding image as not being a bicycle, the notification unit 635 does not need to notify the driver of a request to switch drivers, even if the target road is determined to be a bicycle path. By operating in this manner, the ECU 6 can more appropriately switch from automatic driving control to manual driving control when a two-wheeled vehicle is detected in the vicinity of the vehicle.

[0044] Furthermore, if the notification of driver change requests based on the identification result of the identification unit 634 is not suppressed, the ECU 6 does not need to be equipped with the identification unit 634.

[0045] Figure 4 is a schematic diagram illustrating an example of the surrounding conditions of a vehicle.

[0046] In an example of the surrounding conditions of a vehicle, vehicle 1 is traveling in lane L1 under automatic control on road RD, which has lane L1 divided by lane markings LL1 and LL2, and lane L2 divided by lane markings LL2 and LL3. The width of lane L1 is L1w. In lane L1, behind vehicle 1, in order of proximity to vehicle 1, are motorcycle B1 and surrounding vehicle V2. The distance from vehicle 1 to motorcycle B1 is D1. In lane L2, behind vehicle 1, is surrounding vehicle V1.

[0047] Figure 5 is a schematic diagram illustrating an example of a generated peripheral image.

[0048] In the example of the surrounding conditions of a vehicle, the rear surrounding camera 2-2 of vehicle 1 generates a surrounding image P1 that represents the situation behind vehicle 1.

[0049] The detection unit 632 detects the motorcycle B1, surrounding vehicles V1-V2, and lane identification lines LL1-3 by inputting the surrounding image P1 into the classifier. In Figure 5, the region R1 corresponding to the motorcycle B1 in the surrounding image P1 is shown with a double line.

[0050] The detection unit 632 detects a two-wheeled vehicle B1 located around vehicle 1 from the surrounding image P1 generated in an example of the surrounding conditions of the vehicle. If the determination unit 633 determines that road RD is a bicycle path, the notification unit 635 notifies the driver of a request for a change of driver.

[0051] Figure 6 is a flowchart of the driving control process. While the vehicle 1 is driving under the automatic control of the driving control unit 631, the ECU 6 repeatedly executes the process shown in Figure 6 at predetermined time intervals (for example, every second).

[0052] First, the detection unit 632 of the ECU6 processor 63 detects two-wheeled vehicles, including bicycles, located around the vehicle 1 from the surrounding image representing the area around the vehicle 1 generated by the surrounding camera 2 (step S1).

[0053] Next, the detection unit 632 determines whether or not a motorcycle has been detected in the surrounding image (step S2). If it is determined that no motorcycle has been detected in the surrounding image (step S2:N), the detection unit 632 terminates the driving control process. In this case, the ECU 6 continues the automatic control of the vehicle's movement by the driving control unit 631.

[0054] If it is determined that a motorcycle has been detected from the surrounding image (step S2:Y), the determination unit 633 of the ECU6 processor 63 determines whether the target road on which vehicle 1 was traveling when the surrounding image in which the motorcycle was detected was generated is a bicycle path (step S3). If it is determined that the target road is not a bicycle path (step S3:N), the determination unit 633 terminates the driving control process. In this case, the ECU6 continues the automatic control of the vehicle's driving by the driving control unit 631.

[0055] If the target road is determined to be a bicycle path (step S3:Y), the notification unit 635 notifies the driver of a request for a change of driver (step S4) and terminates the driving control process. In this case, the ECU 6 performs control actions such as transitioning from automatic to manual driving control and ensuring safety by stopping the vehicle, in response to the driver's action in response to the request for a change of driver.

[0056] By performing the driving control process in this manner, the ECU6 can appropriately switch the vehicle's driving control from automatic to manual when a motorcycle is detected in the vicinity of the vehicle.

[0057] In a modified version, even if the notification unit 635 detects a motorcycle located around vehicle 1 from the surrounding image and determines that the target road is a bicycle path, it will not notify the driver of a request for a change of driver if the motorcycle is located behind vehicle 1 and the distance from vehicle 1 to the motorcycle is increasing over time. Even if the motorcycle located behind is a bicycle, if the distance from vehicle 1 to that motorcycle is increasing over time, it is considered that the motorcycle will not have any particular effect on the driving of vehicle 1. By operating in this way, the ECU 6 can suppress notifications of unnecessary driver change requests and control the driving of the vehicle more appropriately.

[0058] For example, if a motorcycle is detected from the rear surrounding image, which represents the surrounding conditions behind the vehicle 1 and is generated by the rear surrounding camera 2, the notification unit 635 can determine that a motorcycle located behind the vehicle 1 has been detected from the surrounding image.

[0059] Referring again to Figure 5, the notification unit 635 first identifies the width L1w of lane L1, which is represented at the base of region R1 in the surrounding image P1. If the intersection of the lane markings LL1 and LL2 that demarcate lane L1 in the surrounding image P1 and the extension of the base of region R1 is obscured by surrounding vehicles, the notification unit 635 can find the intersection of the lane marking model generated using the unobscured portions of the lane markings LL1 and LL2 and the extension of the base of region R1, and then identify the width L1w of lane L1 from the position of the intersection. The notification unit 635 can estimate the distance from the surrounding camera 2 of vehicle 1 to the base of region R1 using the width of lane L1 included in the map data stored in the storage device 5, the width L1w of lane L1 represented in the surrounding image P1, and the focal length of the optical system of the surrounding camera 2 stored in memory 62.

[0060] The notification unit 635 can determine whether the distance to the motorcycle detected from surrounding images generated at different times is increasing over time.

[0061] Vehicle 1 may have a LiDAR (Light Detection and Ranging) sensor or a RADAR (Radio Detection and Ranging) sensor as a surrounding sensor. The LiDAR sensor or RADAR sensor outputs a distance image as surrounding data, in which each pixel has a value corresponding to the distance to the object represented by that pixel, based on the surrounding conditions of Vehicle 1.

[0062] Those skilled in the art will understand that various changes, substitutions, and modifications can be made to this invention without departing from its spirit and scope. [Explanation of symbols]

[0063] 1 vehicle 6 ECU 631 Driving control unit 632 Detection unit 633 Judgment section 634 Identification Unit 635 Notification Department

Claims

1. A driving control device for a vehicle capable of driving on a road using automatic driving control, A detection unit that detects two-wheeled vehicles, including bicycles, located around the vehicle from surrounding data representing the area around the vehicle, generated by surrounding sensors mounted on the vehicle, A determination unit that determines whether the target road on which the vehicle was traveling when the surrounding data in which the motorcycle was detected was generated is a bicycle path where bicycle travel is permitted, If the target road is determined to be a bicycle path, the notification unit notifies the driver of a driver change request, which requires a change of control of the vehicle from automatic driving control to manual driving control requiring the driver to operate the vehicle. If the target road is determined not to be a bicycle path, the notification unit does not notify the driver of the driver change request. A driving control device equipped with the following features.

2. The driving control device according to claim 1, wherein the notification unit detects a motorcycle located behind the vehicle from the surrounding data, and the distance from the vehicle to the motorcycle increases over time, and the notification unit does not notify the driver of the request for a change of driver even if the target road is a bicycle path.

3. The system further includes an identification unit that identifies whether or not the two-wheeled vehicle detected from the surrounding data is a bicycle. If the notification unit identifies the two-wheeled vehicle as not being a bicycle, it will not notify the driver of the request for a change of driver, even if the target road is a designated bicycle path. The driving control device according to claim 1 or 2.

4. A method for controlling the driving of a vehicle capable of driving on a road using automatic driving control, From the surrounding data representing the area around the vehicle, generated by the surrounding sensors mounted on the vehicle, two-wheeled vehicles, including bicycles, located around the vehicle are detected. When the surrounding data in which the motorcycle was detected was generated, it is determined whether the target road on which the vehicle was traveling is a bicycle path where bicycle travel is permitted. If the target road is determined to be a bicycle path, the driver is notified of a driver change request, which requires a change of control from automatic driving control to manual driving control requiring the driver to operate the vehicle. If the target road is determined not to be a bicycle path, the driver is not notified of the driver change request. A driving control method that includes the following.

5. A computer program for driving control that is executed on a processor installed in a vehicle capable of driving on a road using autonomous driving control, From the surrounding data representing the area around the vehicle, generated by the surrounding sensors mounted on the vehicle, two-wheeled vehicles, including bicycles, located around the vehicle are detected. When the surrounding data in which the motorcycle was detected was generated, it is determined whether the target road on which the vehicle was traveling is a bicycle path where bicycle travel is permitted. If the target road is determined to be a bicycle path, the driver is notified of a driver change request, which requires a change of control from automatic driving control to manual driving control requiring the driver to operate the vehicle. If the target road is determined not to be a bicycle path, the driver is not notified of the driver change request. A computer program for driving control that causes the processor to perform the following action.

Images