Cruise control device and cruise control method

Abstract

To provide a travel control device capable of controlling travel of a vehicle in such a manner that a driver does not feel anxiety.SOLUTION: A travel control device comprises: an object detection unit for detecting one or more objects from a periphery image representing the periphery of a vehicle which can travel under automatic driving control satisfying a predetermined safety reference; a sight line detection unit for detecting a sight line direction of a driver from a face image representing a face zone of the driver of the vehicle; an identification unit for identifying an object located in the sight line direction of the driver from among the one or more objects; a condition storage unit by which, in a case where a danger avoiding operation performed by the driver in order to avoid danger is detected during the travel of the vehicle, the identified object and a situation condition representing a situation that the identified object was detected are stored in a storage unit while being associated with each other; and a reference change unit by which in a case where the object stored in the storage unit is detected and the situation that the object was detected satisfies the situation condition during the travel of the vehicle under the automatic driving control, a predetermined safety reference is changed in such a manner the driver can more feel safety.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to a driving control device and a driving control method for automatically controlling the driving of a vehicle. [Background technology]

[0002] There is known a driving control device that automatically controls the driving of a vehicle based on a surrounding image generated by a camera mounted on the vehicle. The driving control device detects objects around the vehicle from the surrounding image and controls the driving of the vehicle so as not to collide with the surrounding objects.

[0003] Patent Document 1 describes a data processing system that determines a user's driving behavior and preferences based on driving statistics information collected when an autonomous vehicle is driven in a manual driving mode, and generates a user's driving profile based on the user's driving behavior and preferences. The driving profile generated by the technology of Patent Document 1 for driving scenes such as route selection and lane changing is used for driving control of the autonomous vehicle in similar driving scenes. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2018-101400 A Summary of the Invention [Problem to be solved by the invention]

[0005] Even if a driver's tendency to take a certain driving action in a certain driving scene is confirmed, it is not necessarily clear whether the driver took that driving action while paying attention to the driving scene. For example, when a driving action of slowing down at an intersection without traffic lights is confirmed, it is possible that the driver is slowing down while paying attention to pedestrians near the intersection, not the intersection itself. In such a case, for example, when driving in a place that is not an intersection but where there are pedestrians who may cross the road, if the driver does not slow down, the driver will feel uneasy, and this is not necessarily an appropriate driving control.

[0006] An object of the present disclosure is to provide a driving control device that can control the driving of a vehicle so that the driver does not feel uneasy. [Means for solving the problem]

[0007] The driving control device disclosed herein includes an object detection unit that detects one or more objects from a surrounding image showing the surroundings of a vehicle that can be driven under autonomous driving control that satisfies a specified safety standard; a gaze detection unit that detects the gaze direction of the driver from a face image showing the facial area of ​​the driver of the vehicle; an identification unit that identifies an object in the gaze direction of the driver among the one or more objects; a condition memory unit that, when a danger avoidance operation by the driver to avoid danger is detected while the vehicle is driving, associates the identified object with a situation condition that represents the situation when the identified object was detected and stores it in a memory unit; and a standard change unit that changes the specified safety standard so that the driver feels safer when an object stored in the memory unit is detected while the vehicle is driving under autonomous driving control and the situation when the object was detected satisfies the situation condition.

[0008] In a cruise control device according to the present disclosure, it is preferable that the situation condition includes at least the distance of the identified object from the vehicle.

[0009] The driving control device according to the present disclosure preferably further includes a condition learning unit that creates a learning situation condition representing a situation in which a danger avoidance maneuver is detected when an object is detected based on a plurality of situation conditions stored at different times in association with one of the objects stored in the memory unit, and the criterion changing unit preferably determines that the situation at the time the object is detected satisfies the situation condition if the situation at the time the object is detected satisfies the learning situation condition.

[0010] In the driving control device disclosed herein, it is preferable that the condition memory unit further stores the rate at which a danger avoidance maneuver is detected when the situation at the time an object is detected while the vehicle is traveling satisfies the situation condition, and the condition learning unit creates learning situation conditions such that situation conditions with a higher rate at which a danger avoidance maneuver is detected are given priority over situation conditions with a lower rate at which a danger avoidance maneuver is detected.

[0011] The driving control method disclosed herein includes detecting one or more objects from a surrounding image showing the surroundings of a vehicle capable of driving under autonomous driving control that satisfies a predetermined safety standard, detecting the line of sight of the driver from a face image showing the facial area of ​​the driver of the vehicle, identifying an object in the line of sight of the driver among the one or more objects, and if a danger avoidance operation by the driver to avoid danger is detected while the vehicle is driving, storing the identified object in a memory unit in association with a situation condition that represents the situation when the identified object was detected, and if the object stored in the memory unit is detected while the vehicle is driving under autonomous driving control and the situation when the object was detected satisfies the situation condition, changing the predetermined safety standard so that the driver feels safer.

[0012] According to the driving control device of the present disclosure, it is possible to control the driving of the vehicle so that the driver does not feel uneasy. [Brief description of the drawings]

[0013] [Figure 1] 1 is a schematic configuration diagram of a vehicle in which a driving control device is implemented; [Diagram 2] FIG. 2 is a hardware schematic diagram of a driving control device. [Diagram 3] FIG. 2 is a functional block diagram of a processor included in the driving control device. [Figure 4] FIG. 13 is a diagram illustrating the identification of an object in the line of sight. [Diagram 5] FIG. 13 is a diagram illustrating an example of a status table. [Figure 6] 4 is a flowchart of a first driving control process. [Figure 7] 11 is a flowchart of a second driving control process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, a driving control device capable of controlling the driving of a vehicle so that the driver does not feel uneasy will be described in detail with reference to the drawings. The driving control device detects one or more objects from a surrounding image showing the surroundings of a vehicle that can be driven by automatic driving control that satisfies a predetermined safety standard such as a vehicle speed and a distance from a surrounding object, and detects the line of sight of the driver from a face image showing the face area of ​​the driver of the vehicle. The driving control device specifies an object in the line of sight of the driver among the one or more objects. Furthermore, when the driving control device detects a risk avoidance operation by the driver to avoid a risk while the vehicle is driving, the driving control device associates the specified object with a situation condition that represents the situation when the specified object was detected and stores it in a storage unit. Then, when the object stored in the storage unit is detected while the vehicle is driving by automatic driving control and the situation when the object was detected satisfies the situation condition, the driving control device changes the predetermined safety standard so that the driver feels safer.

[0015] FIG. 1 is a schematic configuration diagram of a vehicle in which a driving control device is implemented.

[0016] The vehicle 1 has a surrounding camera 2, a driver monitor camera 3, and a driving control device 4. The surrounding camera 2, the driver monitor camera 3, and the driving control device 4 are communicatively connected via an in-vehicle network that complies with a standard such as a controller area network.

[0017] The surrounding camera 2 is an example of a surrounding image capturing unit for generating a surrounding image showing the surroundings of the vehicle. The surrounding camera 2 has a two-dimensional detector configured with an array of photoelectric conversion elements sensitive to visible light, such as a CCD or C-MOS, and an imaging optical system for forming an image of the area to be captured on the two-dimensional detector. The surrounding camera 2 is disposed, for example, at the front upper part of the vehicle interior facing forward, captures the surroundings of the vehicle 1 through the windshield at a predetermined capture period (for example, 1 / 30 to 1 / 10 seconds), and outputs a surrounding image showing the surroundings.

[0018] The driver monitor camera 3 is an example of a driver photographing unit for generating a face image showing the facial area of ​​the driver of the vehicle. The driver monitor camera 3 has a two-dimensional detector composed of an array of photoelectric conversion elements sensitive to infrared light, such as CCD or C-MOS, and an imaging optical system for forming an image of the area to be photographed on the two-dimensional detector. The driver monitor camera 3 also has a light source that emits infrared light. The driver monitor camera 3 is attached, for example, at the front of the vehicle interior, facing the face of the driver seated in the driver's seat. The driver monitor camera 3 irradiates the driver with infrared light at a predetermined photographing period (for example, 1 / 30 to 1 / 10 seconds) and outputs an image showing the driver's face.

[0019] The driving control device 4 is an ECU (Electronic Control Unit) having a communication interface, a memory, and a processor. The driving control device 4 outputs control signals to driving mechanisms (not shown) of the vehicle 1, such as the engine, brakes, and steering, so as to satisfy a predetermined safety standard, and executes automatic driving control of the vehicle 1. The driving control device 4 also detects an object in the driver's line of sight from the surrounding image received from the surrounding camera 2 and the face image received from the driver monitor camera 3 via the communication interface, and stores the situation when the object is detected. Then, the driving control device 4 changes the safety standard in the automatic driving control of the vehicle 1 based on the situation when the object is detected from the surrounding image.

[0020] 2 is a schematic diagram of the hardware of the driving control device 4. The driving control device 4 includes a communication interface 41, a memory 42, and a processor 43.

[0021] The communication interface 41 is an example of a communication unit, and has a communication interface circuit for connecting the driving control device 4 to an in-vehicle network. The communication interface 41 supplies the received data to the processor 43. In addition, the communication interface 41 outputs the data supplied from the processor 43 to the outside.

[0022] The memory 42 is an example of a storage unit, and includes a volatile semiconductor memory and a non-volatile semiconductor memory. The memory 42 stores various data used in processing by the processor 43, such as a parameter group for defining a neural network that operates as an object identifier for detecting an object from a surrounding image, a safety standard used for automatic driving control, and a situation table in which a detected object is associated with a situation when the object was detected. The memory 42 also stores various application programs, such as a driving control program for executing a driving control process.

[0023] The processor 43 is an example of a control unit, and includes one or more processors and their peripheral circuits. The processor 43 may further include other arithmetic circuits, such as a logic arithmetic unit, a numerical arithmetic unit, or a graphics processing unit.

[0024] FIG. 3 is a functional block diagram of the processor 43 included in the driving control device 4. As shown in FIG.

[0025] The processor 43 of the driving control device 4 has, as functional blocks, an object detection unit 431, a gaze detection unit 432, a specification unit 433, a condition storage unit 434, a condition learning unit 435, and a criterion change unit 436. Each of these units in the processor 43 is a functional module implemented by a program executed on the processor 43. Alternatively, each of these units in the processor 43 may be implemented in the driving control device 4 as an independent integrated circuit, microprocessor, or firmware.

[0026] The object detection unit 431 detects objects around the vehicle 1 by inputting the surrounding image received from the surrounding camera 2 via the communication interface to an object identifier that has been trained in advance to detect objects.

[0027] The object classifier can be, for example, a convolutional neural network (CNN) having multiple convolution layers connected in series from the input side to the output side. Images containing objects are input to the CNN in advance as training data, and the CNN operates as an object classifier that detects objects from images by learning the images.

[0028] The gaze detection unit 432 detects the gaze direction of the driver from a facial image received via the communication interface from the driver monitor camera 3. The gaze direction is represented by the horizontal angle between the traveling direction of the vehicle 1 and the driver's gaze.

[0029] The gaze detection unit 432 detects the positions of the pupil and the corneal reflection of the driver's eye included in the face image by inputting the positions of the pupil and the corneal reflection of the light source into a gaze identifier that has been trained in advance to detect the positions of the pupil and the corneal reflection of the light source. Then, the gaze detection unit 432 detects the gaze direction based on the positional relationship between the pupil and the corneal reflection.

[0030] The gaze classifier can be, for example, a convolutional neural network (CNN) having multiple convolution layers connected in series from the input side to the output side. By inputting face images including pupils and corneal reflections as training data to the CNN in advance and performing learning, the CNN operates as a gaze classifier that identifies the positions of the pupils and corneal reflections.

[0031] The identification unit 433 identifies an object in the line of sight of the driver from among one or more objects detected from the peripheral image.

[0032] FIG. 4 is a diagram for explaining the identification of an object in the line of sight.

[0033] In the example of Fig. 4, objects O1 (pedestrian), O2 (pedestrian), O3 (other vehicle) and lane markings L1, L2, L3 are detected from a peripheral image showing the area ahead of the vehicle 1. Also, a line of sight direction DR1 is detected from a face image. The line of sight direction DR1 is a direction that forms an angle α with the traveling direction DR0. The identification unit 433 identifies an object O2 (pedestrian) that is located at a position corresponding to the line of sight direction DR1 from the position of the vehicle 1 as an object in the line of sight of the driver.

[0034] When the condition memory unit 434 detects a danger avoidance operation by the driver while the vehicle 1 is traveling, it stores in the memory 42 a situation condition that represents the situation when the identified object was detected, in association with the object identified by the identification unit 433.

[0035] The danger avoidance operation is an operation performed by the driver on the vehicle 1 to avoid danger. The danger avoidance operation may be, for example, the driver depressing the brake pedal to decelerate or stop the vehicle 1 while the vehicle 1 is traveling under manual driving control, or the driver turning the steering wheel to avoid approaching an object. The danger avoidance operation may be the driver depressing the brake pedal, turning the steering wheel, or pressing a danger avoidance intention display button to indicate the driver's intention to avoid danger while the vehicle 1 is traveling under automatic driving control. The condition storage unit 434 detects the danger avoidance operation by receiving an operation signal via the communication interface 41 from an operation unit such as the brake pedal, steering wheel, or danger avoidance intention display button connected to the in-vehicle network.

[0036] The situation conditions include at least the distance of the object from the vehicle, for example, the distance from the vehicle to the object in the traveling direction (vertical distance) or the distance in the horizontal direction (horizontal distance). The situation conditions may also include the direction in which the object is facing, the road environment (presence or absence of a step between the sidewalk and the lane, the number of lanes, etc.), the speed and acceleration of the vehicle, the arrival time of the vehicle to the object, etc.

[0037] In the example of Figure 4, the condition memory unit 434 associates the vertical distance D1 and horizontal distance D2 to the identified object O2 (pedestrian), and information indicating whether the direction DR2 in which the object O2 (pedestrian) is facing is toward the vehicle, with the object O2 (pedestrian), and stores them in the memory 42.

[0038] FIG. 5 is a diagram illustrating an example of the situation table.

[0039] In the situation table 421, detected objects and situations when the objects were detected are stored in association with each other. For example, at time T1, a pedestrian is detected in the driver's line of sight, and the driver's risk avoidance operation is detected, and values ​​representing the vertical distance, horizontal distance, and vehicle direction, which are the situation conditions at this time, are stored. At time T2, which is different from time T1, the vertical distance when the pedestrian is detected in the driver's line of sight satisfies the situation condition of the vertical distance at time T1, and values ​​representing the vertical distance, horizontal distance, and vehicle direction, which are the situation conditions at this time, are stored. At time T3, a vehicle is detected in the driver's line of sight, and values ​​representing the vertical distance, horizontal distance, the number of lanes on the road on which the vehicle 1 is traveling, and the vehicle speed of the vehicle 1, which are the situation conditions at this time, are stored.

[0040] It should be noted that the situation table 421 is not limited to a data management format in the form of a table as shown in FIG. 5, as long as it associates an object with a situation.

[0041] The condition learning unit 435 creates a learning situation condition that represents a situation in which a danger avoidance operation is detected when an object is detected, based on a plurality of situation conditions that are associated with one of the objects stored in the memory 42 and stored at different times.

[0042] At this time, it is preferable that the condition learning unit 435 creates the learning situation conditions so that a situation condition in which a danger avoidance operation is detected at a high rate is given priority over a situation condition in which a danger avoidance operation is detected at a low rate.

[0043] In the example of situation table 421 shown in Fig. 5, at times T1, T4, and T5 when the pedestrian's own vehicle orientation is YES, a danger avoidance operation is detected at times T1 and T5, but not at time T4 (detection rate is 67%). On the other hand, when the pedestrian's own vehicle orientation is NO (time T2), the danger avoidance operation is not detected (detection rate is 0%). Therefore, condition learning unit 435 creates learning situation conditions such that for the own vehicle orientation when a pedestrian is detected as an object, a situation condition of YES, which has a high detection rate for a danger avoidance operation, is prioritized over a situation condition of NO, which has a low detection rate for a danger avoidance operation.

[0044] In the example of the situation table 421, at times T1 and T2 when the vertical distance to the pedestrian is 20 (m), a danger avoidance operation is detected at time T1, and is not detected at time T2 (detection rate is 50%). At time T4 when the vertical distance to the pedestrian is 30 (m), a danger avoidance operation is not detected (detection rate is 0%). At time T5 when the vertical distance to the pedestrian is 15 (m), a danger avoidance operation is detected (detection rate is 100%). Therefore, the condition learning unit 435 creates a learning situation condition such that a situation condition with a high detection rate of a danger avoidance operation (e.g., 15 (m)) is prioritized over a situation condition with a low detection rate of a danger avoidance operation (e.g., 30 (m)) for the vertical distance to the pedestrian. The learning situation condition is created as a conditional expression, for example, "vertical distance to the pedestrian is 20 (m) or less".

[0045] The condition learning unit 435 may create a learning situation condition by learning a classifier that classifies whether or not a situation when an object stored in the situation table 421 is detected is a situation in which a danger avoidance operation by the driver is detected.

[0046] The classifier may be a support vector machine (SVM). The condition learning unit 435 inputs the objects and situation conditions stored in the memory 42 in association with each other to the SVM for learning, thereby creating a learning situation condition.

[0047] The classifier may be a neural network. The condition learning unit 435 creates a learning situation condition by inputting the objects and situation conditions stored in the memory 42 in association with each other to the neural network and having the neural network learn the objects and situation conditions.

[0048] When an object stored in memory 42 is detected while vehicle 1 is traveling under automatic driving control and the situation when the object is detected satisfies a situation condition, standard change unit 436 changes the predetermined safety standard so that the driver feels safer.

[0049] For example, if a pedestrian is detected while the vehicle 1 is traveling under automatic driving control, the vertical distance to the pedestrian is 20 (m), the horizontal distance is 4 (m), and the pedestrian's orientation to the vehicle is Yes, the situation when the object is detected satisfies the situation condition at the time T1 when the danger avoidance operation is detected. At this time, the criterion change unit 436 changes the predetermined driving criterion so that the driver feels safer.

[0050] When the safety criterion is the vehicle speed, the criterion change unit 436 changes the vehicle speed to a lower speed so that the driver feels safer. When the safety criterion is the distance to a surrounding object, the criterion change unit 436 changes the distance to a larger distance so that the driver feels safer.

[0051] If the situation when the object is detected satisfies the learning situation condition created by the condition learning unit 435, the criterion change unit 436 determines that the situation when the object is detected satisfies the situation condition. For example, assume that a learning situation condition is created in which "the vertical distance to the pedestrian is 20 (m) or less, and the horizontal distance to the pedestrian is 4 (m) or less, and the pedestrian's own vehicle orientation is Yes." If the vertical distance to the detected pedestrian is 18 (m), the horizontal distance to the pedestrian is 3.5 (m), and the pedestrian's own vehicle orientation is Yes, the situation when the object is detected satisfies the learning situation condition, and therefore the criterion change unit 436 may determine that the situation when the object is detected satisfies the situation condition.

[0052] 6 is a flowchart of the first driving control process. The driving control device 4 repeatedly executes the first driving control process at predetermined time intervals (for example, 1 / 10 second intervals) while the vehicle 1 is driving.

[0053] First, the object detection unit 431 of the driving control device 4 detects one or more objects from the peripheral image generated by the peripheral camera 2 (step S11). In addition, the line of sight detection unit 432 of the driving control device 4 detects the line of sight direction of the driver of the vehicle 1 from the face image generated by the driver monitor camera 3 (step S12).

[0054] Next, the identification unit 433 of the driving control device 4 identifies an object in the line of sight of the driver from among the one or more detected objects (step S13).

[0055] The condition storage unit 434 of the driving control device 4 judges whether or not a danger avoidance operation by the driver is detected (step S14). If a danger avoidance operation is detected (step S14: Y), the condition storage unit 434 associates the specified object with a situation condition that represents the situation when the specified object was detected, and stores them in the memory 42 (step S15), and ends the first driving control process.

[0056] If a danger avoidance operation is not detected (step S14: N), the condition storage unit 434 ends the first driving control process.

[0057] 7 is a flowchart of the second driving control process. The driving control device 4 repeatedly executes the second driving control process at a predetermined time interval (for example, every 1 / 10 seconds) while the vehicle 1 is driving under automatic driving control.

[0058] First, the object detection unit 431 of the driving control device 4 detects one or more objects from the peripheral image generated by the peripheral camera 2 (step S21).

[0059] Next, the criterion change unit 436 of the driving control device 4 judges whether or not the detected object is an object stored in the situation table 421 of the memory 42 (step S22). If it is judged that the detected object is not an object stored in the situation table 421 (step S22: N), the criterion change unit 436 ends the second driving control process.

[0060] Furthermore, the criterion change unit 436 of the driving control device judges whether or not the situation when the object is detected satisfies the situation condition (step S23). If it is judged that the situation when the object is detected does not satisfy the situation condition (step S23: N), the criterion change unit 436 ends the second driving control process.

[0061] If it is determined that the situation when the object is detected satisfies the situation condition (step S23: Y), the standard change unit 436 changes the predetermined safety standard so that the driver feels safer (step S24), and ends the second driving control process.

[0062] By executing the first and second driving control processes in this manner, the driving control device 4 can control the driving of the vehicle so that the driver does not feel uneasy.

[0063] It should be understood that those skilled in the art can make various changes, substitutions, and alterations thereto without departing from the spirit and scope of the present disclosure. [Explanation of symbols]

[0064] 1 vehicle 4. Driving control device 431 Object Detection Unit 432 Line of Sight Detection Unit 433 Specific part 434 Condition memory section 435 Conditional Learning Section 436 Standard Change Section

Claims

1. an object detection unit that detects one or more objects from a surrounding image that shows the surroundings of a vehicle that can be driven under autonomous driving control and that satisfies a predetermined safety standard; a gaze detection unit that detects a gaze direction of the driver from a face image representing a face area of ​​the driver of the vehicle; An identification unit that identifies an object in the line of sight of the driver among the one or more objects; a condition storage unit that, when detecting a danger avoidance operation by the driver to avoid danger while the vehicle is traveling, stores the specified object in a storage unit in association with a situation condition that represents a situation at the time when the specified object was detected; a standard change unit that changes the predetermined safety standard so that the driver feels safer when an object stored in the storage unit is detected during the autonomous driving control of the vehicle and the situation at the time when the object is detected satisfies the situation condition; and A driving control device comprising:

2. The cruise control device according to claim 1 , wherein the situation condition includes at least a distance of the identified object from the vehicle.

3. a condition learning unit that creates a learned situation condition that represents a situation in which the danger avoidance operation is detected when an object is detected, based on a plurality of situation conditions that are associated with any one of the objects stored in the storage unit and stored at different times, The driving control device according to claim 1 , wherein the criterion change unit determines that the situation when the object is detected satisfies the situation condition when the situation when the object is detected satisfies the learning situation condition.

4. the condition storage unit further stores a ratio at which the danger avoidance operation is detected when a situation at which the object is detected while the vehicle is traveling satisfies the situation condition; and The driving control device according to claim 3 , wherein the condition learning unit creates the learning situation conditions such that the situation conditions in which the risk avoidance operation is detected at a higher rate are given priority over the situation conditions in which the risk avoidance operation is detected at a lower rate.

5. Detecting one or more objects from a surrounding image showing the surroundings of a vehicle that can be driven under autonomous driving control that satisfies a predetermined safety standard; Detecting a line of sight direction of a driver of the vehicle from a face image representing a face area of ​​the driver of the vehicle; Identifying an object in the driver's line of sight direction among the one or more objects; When a danger avoidance operation for avoiding danger by the driver is detected while the vehicle is traveling, the specified object is associated with a situation condition that represents a situation at the time when the specified object was detected and stored in a storage unit; When an object stored in the storage unit is detected during the traveling of the vehicle under the automatic driving control, and the situation at the time when the object is detected satisfies the situation condition, the predetermined safety standard is changed so that the driver feels safer. A driving control method comprising the steps of:

Images