Autonomous driving system

Abstract

To provide an automatic driving system in which vehicle control according to a mechanical learning model is effectively monitored.SOLUTION: An automatic driving system 1 detects information related to a travel situation and a peripheral situation of a vehicle V according to a sensor 2, executes vehicle control using a mechanical learning model M on the basis of the information related to a travel situation and a peripheral situation, and presents a detection result of the sensor 2 if detection performance of the sensor 2 is lower than a regulated level, and presents a detection result of the sensor 2 corresponding to the situation if the vehicle V is subjected to a situation where the risk is higher than a regulated value.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] One aspect of the present invention relates to an automatic driving system.

Background Art

[0002] An automatic driving system that performs automatic driving of a vehicle by vehicle control using a machine learning model is known. As this type of technology, for example, Patent Document 1 describes a teacher data collection device that collects teacher data that can be used for machine learning for generating an automatic driving model.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, a machine learning model generated by machine learning does not necessarily have the ability to realize appropriate vehicle control in all scenarios. The range in which vehicle control can be appropriately executed using a machine learning model can vary depending on traffic environments such as, for example, weather, time of day, traffic volume, etc. Therefore, in order to appropriately apply vehicle control by a machine learning model, it is desirable to effectively monitor the vehicle control.

[0005] Therefore, one aspect of the present invention aims to provide an automatic driving system capable of effectively monitoring vehicle control by a machine learning model.

Means for Solving the Problems

[0006] An automated driving system according to one aspect of the present invention detects information regarding at least one of the vehicle's driving conditions and surrounding conditions using one or more sensors, performs vehicle control using a machine learning model based on the information regarding at least one of the driving conditions and surrounding conditions, presents the detection results of the sensors if the detection performance of the sensors is lower than a specified level, and presents the detection results of the sensors corresponding to the situation if a situation with a risk higher than a specified value occurs in the vehicle.

[0007] An autonomous driving system according to one aspect of the present invention may be in any of the following cases when the sensor's detection performance is lower than a specified level: when the sensor has not detected surrounding objects or lane markings; when the sensor's stability is lower than past stability; when the sensor's stability is lower than the design value; when the sensor's stability is lower than the stability of other sensors whose fields of view overlap with that sensor; or when the sensor's stability is lower than the stability of other sensors of other vehicles that have traveled in the same location. [Effects of the Invention]

[0008] According to one aspect of the present invention, it is possible to provide an autonomous driving system that can effectively monitor vehicle control using a machine learning model. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a block diagram showing the configuration of an automated driving system according to one embodiment. [Figure 2] Figure 2 is a flowchart showing an example of the processing performed by the autonomous driving system in Figure 1. [Figure 3] Figure 3 is a flowchart showing another example of the processing performed by the autonomous driving system shown in Figure 1. [Modes for carrying out the invention]

[0010] The embodiments will be described in detail below with reference to the attached drawings. In the description of the drawings, the same or equivalent elements will be denoted by the same reference numeral, and redundant descriptions will be omitted.

[0011] As shown in Figure 1, the autonomous driving system 1 according to this embodiment is a system that performs autonomous driving of vehicle V by vehicle control using a machine learning model. The autonomous driving system 1 is installed in vehicle V. Vehicle V may be a passenger car or a cargo vehicle. Vehicle V can accommodate one or more occupants. Vehicle V is an autonomous driving vehicle capable of autonomous driving. Vehicle V may also be capable of manual driving by a driver.

[0012] The autonomous driving system 1 comprises a sensor 2, an actuator 3, a communication unit 4, an autonomous driving ECU 5 (Electronic Control Unit), and an HMI (Human Machine Interface). Sensor 2 is a sensor that detects information regarding at least one of the driving conditions of the vehicle V and the surrounding conditions. Sensor 2 has external sensors and internal sensors. External sensors are sensors that acquire information about the surrounding environment of the vehicle V. External sensors include at least one of, for example, a camera, millimeter-wave radar, and LiDAR (Light Detection and Ranging). Internal sensors are detection devices that detect the driving conditions of the vehicle V. External sensors include sensors that monitor the front of the vehicle V, sensors that monitor the rear of the vehicle V, and sensors that monitor the sides of the vehicle V. Internal sensors include at least one of, for example, a vehicle speed sensor, an acceleration sensor, and a yaw rate sensor. Sensor 2 transmits the detection results to the autonomous driving ECU 5.

[0013] Actuator 3 is a controller for controlling the speed of vehicle V. Actuator 3 may include, for example, an actuator for controlling the output of an engine or motor, and a brake actuator. Communication unit 4 is a communication device that controls wireless communication between vehicle V and the outside world. Communication unit 4 communicates various information with the server 100 for the automated driving system via a communication network N, for example. Communication unit 4 also communicates various information with other vehicles in the vicinity, for example. Communication unit 4 is not particularly limited, and various known communication devices can be used.

[0014] The autonomous driving ECU 5 is an electronic control unit that includes a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc. The autonomous driving ECU 5 implements various functions, for example, by loading a program recorded in ROM into RAM and executing the program loaded into RAM with the CPU. The autonomous driving ECU 5 may be composed of multiple electronic control units.

[0015] The autonomous driving ECU 5 performs vehicle control using a machine learning model M based on the detection results of sensor 2 (information regarding at least one of the vehicle V's driving status and surrounding conditions). For example, the autonomous driving ECU 5 inputs the detection results of sensor 2 into the machine learning model M and outputs the resulting control signals to actuator 3, thereby controlling the vehicle V's drive, brake, and steering to perform autonomous driving.

[0016] The machine learning model M is a recurrent deep learning model. The machine learning model M is a recurrent neural network (RNN). At least a portion of the neural network may be a convolutional neural network (CNN) containing multiple layers, including multiple convolutional layers and pooling layers. Deep learning is performed in the machine learning model M. The machine learning model M is a trained model that has been trained using vehicle V data under predetermined training conditions. For example, the machine learning model M may be trained using training data for the output to actuator 3 when various detection results from sensor 2 are input.

[0017] HMI6 is an interface for inputting and outputting various types of information between the vehicle V and its occupants (natural persons). HMI6 includes, for example, a display and a speaker. HMI6 outputs images from the display and audio from the speaker in response to control signals from ECU10. HMI6 may also include a HUD (Head-Up Display). HMI6 presents various types of information to the occupants of vehicle V.

[0018] The autonomous driving system server 100 is a server capable of communicating with the autonomous driving system 1. The autonomous driving system server 100 includes a communication unit 101 and a display unit 102. The communication unit 101 communicates various information with the vehicle V, for example, via a communication network N. The communication unit 101 is not particularly limited, and various known communication devices can be used. The display unit 102 is an interface that presents various information to the user (natural person). The display unit 102 presents various information to the user by image output. The display unit 102 is not particularly limited, and various known display devices can be used.

[0019] In this embodiment, if the detection performance of sensor 2 is lower than a specified level, the autonomous driving ECU 5 displays the detection result of sensor 2 via HMI 6. If a situation occurs in vehicle V where the risk is higher than a specified value, the autonomous driving ECU 5 displays the detection result of sensor 2 corresponding to that situation via HMI 6. The display of the detection result of sensor 2 may be performed by the display unit 102 via communication unit 4,101, instead of or in addition to being performed by HMI 6 (the same applies hereinafter). Sensor 2 corresponding to a situation where the risk is higher than a specified value is, for example, a sensor that directly or indirectly detects that situation and related parameters.

[0020] When the detection performance of sensor 2 is lower than the specified level, when sensor 2 fails to detect surrounding objects or lane lines, when the stability of sensor 2 is lower than its past stability, when the stability of sensor 2 is lower than the designed value, when the stability of sensor 2 is lower than the stability of other sensors with overlapping fields of view for this sensor 2, and when the stability of sensor 2 is lower than the stability of other sensors of other vehicles that have traveled at the same location. The surrounding objects are not particularly limited and include, for example, moving obstacles and fixed obstacles. The stability of sensor 2 means the stability of a general sensor, for example, the difficulty of fluctuation of the detected value detected in a certain situation.

[0021] When at least one of, for example, the acceleration / deceleration and steering angular velocity of vehicle V is above the threshold value, the automatic driving ECU 5 presents the detection result of the sensor for forward monitoring among sensor 2s, assuming that a situation where the risk is higher than the specified value has occurred in vehicle V. The threshold value may be preset and stored in the automatic driving ECU 5, and may be a fixed value or a variable value. The risk is, for example, a risk related to the driving of vehicle V. The specified value is a preset value. The risk and the specified value are not particularly limited and may be various known risks and specified values.

[0022] When vehicle V changes lanes to an adjacent lane due to the intervention of the driver of vehicle V, for example, when the distance between the driving trajectory of vehicle V before the lane change and other vehicles traveling in the adjacent lane is below the threshold value, the automatic driving ECU 5 presents the detection result of the sensor for rear monitoring and the detection result of the sensor for side monitoring among sensor 2s, assuming that a situation where the risk is higher than the specified value has occurred in vehicle V. The threshold value may be preset and stored in the automatic driving ECU 5, and may be a fixed value or a variable value.

[0023] When the automatic driving ECU 5 presents the detection result of the sensor 2, it may present the detection result of the sensor 2 together with the sample data. The sample data may be predetermined and stored in the automatic driving ECU 5 in advance. The sample data is data that serves as a sample of the detection result of the sensor 2, for example, data when the sensor 2 is normal. Thereby, it is possible to easily grasp whether the detection result of the sensor 2 is abnormal or normal.

[0024] When a situation where the risk is higher than the specified value occurs in the vehicle V, the automatic driving ECU 5 may preferentially present the detection result of the second sensor, which has a higher influence degree on the situation than the first sensor among the sensors 2, by at least one of the HMI 6 and the display unit 102. Thereby, it is possible to preferentially grasp the detection result of the sensor 2 that has a greater influence on the situation where the risk is higher than the specified value.

[0025] Next, an example of the monitoring process for monitoring the detection result of the sensor 2 by the automatic driving system 1 of the present embodiment will be described with reference to the flowcharts of FIGS. 2 and 3. The monitoring process here may be executed, for example, while the vehicle V is running. When the process reaches the end, the process may be restarted from the start again after a predetermined time.

[0026] As shown in FIG. 2, the automatic driving ECU 5 acquires the detection result of the sensor 2 (step S1). The automatic driving ECU 5 determines whether there is a sensor 2 among the sensors 2 whose detection performance is lower than the specified level based on the detection result of the sensor 2 (step S2). For example, in step S2 above, when the surrounding object or the lane line is not detected in the detection result of the sensor 2, when the current stability of the sensor 2 is lower than the past stability, when the stability of the sensor 2 is lower than the design value, when the stability of the sensor 2 is lower than the stability of another sensor whose detection range overlaps with the sensor 2, and when the stability of the sensor 2 is lower than the stability of another sensor of another vehicle that has traveled in the same place, it may be determined that there is a sensor 2 whose detection performance is lower than the specified level in at least one of these cases.

[0027] If the answer in step S2 is NO, the processing for the current cycle is terminated, and the process moves to step S1 for the next cycle. On the other hand, if the answer in step S2 is YES, the detection result of sensor 2, whose detection performance is lower than the specified level, is presented by HMI6 along with sample data (step S3).

[0028] Furthermore, as shown in Figure 3, the autonomous driving ECU 5 acquires the detection results of the sensor 2 (step S11). Based on the detection results of the sensor 2, the autonomous driving ECU 5 determines whether a situation has occurred where the risk is higher than a specified value (step S12). For example, in step S12, it may be determined that a situation has occurred where the risk is higher than a specified value if at least one of the acceleration / deceleration and steering angular velocity of the vehicle V is above a threshold, if the vehicle V decelerates above a threshold due to driver intervention, if a lane change is performed due to driver intervention, or if the planned driving path of the vehicle V is discontinuous in time.

[0029] If the answer in step S12 is NO, the processing for the current cycle is terminated, and the system proceeds to step S11 of the next cycle. On the other hand, if the answer in step S12 is YES, the detection result of sensor 2 corresponding to a situation where the risk is higher than the specified value is presented by HMI 6 along with sample data (step S13). In step S13, if at least one of the acceleration / deceleration and steering angular velocity of vehicle V is above a threshold, the detection result of the front sensor may be presented. In step S13, if the vehicle V decelerates above a threshold due to driver intervention, the detection result of the front sensor may be presented. In step S13, if a lane change is performed due to driver intervention, the detection results of the rear sensor and side sensor may be presented. The presentation of the detection results may include the presentation of an image of the road surface and the presentation of the position of vehicle V relative to the recognized white line.

[0030] Note that either step S1 or S11 may be performed. If step S1 is not performed, the detection result of sensor 2 obtained in step S11 may be used in the determination in S2. If step S11 is not performed, the detection result of sensor 2 obtained in step S1 may be used in the determination in S12.

[0031] In summary, in the autonomous driving system 1, when vehicle V is driven autonomously by vehicle control using a machine learning model M, by understanding the detection results of sensor 2 with low detection performance, it is easy to determine, for example, whether the delay in the autonomous driving response to the surrounding conditions of vehicle V is due to sensor 2. In addition, when a situation occurs where the risk is higher than a predetermined value, by understanding the detection results of sensor 2 that affects that situation, it is easy to determine whether that situation is due to sensor 2. In other words, the autonomous driving system 1 makes it possible to effectively monitor vehicle control by machine learning model M.

[0032] In the autonomous driving system 1, if the detection performance of sensor 2 is lower than the specified level, it is at least one of the following: sensor 2 has not detected surrounding objects or lane markings; the stability of sensor 2 is lower than past stability; the stability of sensor 2 is lower than the design value; the stability of sensor 2 is lower than the stability of other sensors whose field of view overlaps with that of sensor 2; or the stability of sensor 2 is lower than the stability of other sensors of other vehicles that have traveled to the same location. In this case, it is possible to specifically determine when the sensor's detection performance is lower than the specified level.

[0033] In the automated driving system 1, if at least one of the vehicle V's acceleration / deceleration or steering angular velocity exceeds a threshold, the system indicates that a situation with a higher-than-specified risk has occurred in vehicle V and presents the detection results of the forward-monitoring sensor 2. This makes it easy to understand the detection results of sensor 2 that affect the situation when at least one of the vehicle V's acceleration / deceleration or steering angular velocity exceeds a threshold.

[0034] In the automated driving system 1, when vehicle V changes lanes to an adjacent lane due to driver intervention, if the distance between the vehicle's trajectory before the lane change and another vehicle traveling in the adjacent lane is below a threshold, the system may indicate that a situation with a higher risk than a specified value has occurred in vehicle V, and present the detection results of the rear-monitoring sensor 2 and the side-monitoring sensor 2. This makes it easy to understand the detection results of the sensors 2 that affect situations with a high probability of contact with other vehicles during lane changes.

[0035] Although embodiments have been described above, the present invention is not limited to the embodiments described above. One embodiment of the present invention can be implemented in various forms, starting with the embodiments described above, with various modifications and improvements based on the knowledge of those skilled in the art.

[0036] In the above embodiment, the type of sensor 2 is not limited and may include various detection devices. Also, the number of sensors 2 installed is not limited and may be one or multiple. In the above embodiment, the case in which the detection performance of sensor 2 is lower than the specified level is not particularly limited and may be other than the case described above. In the above embodiment, the situation in which the risk is higher than the specified value is not particularly limited and may be other than the situation described above.

[0037] In the above embodiment, if the camera and lidar output detection results separately, the detection performance of the camera and lidar may be determined based on their respective detection results. On the other hand, if the camera and lidar output a single detection result, it may be determined whether the detection performance of the camera and lidar is low or not based on that single detection result. In the above embodiment, if the reliability of a single detection result obtained by integrating multiple sensors 2 is low, the detection performance of the multiple sensors 2 may be considered low, and the detection results of the multiple sensors 2 may be presented accordingly. [Explanation of Symbols]

[0038] 1...Autonomous driving system, 2...Sensor, M...Machine learning model, V...Vehicle.

Claims

1. One or more sensors detect information regarding at least one of the vehicle's driving conditions and surrounding conditions. Based on information regarding at least one of the aforementioned driving conditions and surrounding conditions, vehicle control is performed using a machine learning model. If the detection performance of the aforementioned sensor is lower than a specified level, the detection result of the sensor will be displayed. If a situation occurs in the vehicle where the risk is higher than the specified value, the detection result of the sensor corresponding to that situation will be presented. An autonomous driving system that, if at least one of the vehicle's acceleration / deceleration or steering angular velocity exceeds a threshold, indicates that a situation has occurred in the vehicle where the risk is higher than a specified value, and presents the detection results of the forward-monitoring sensor.

2. One or more sensors detect information relating to at least one of the vehicle's driving conditions and surrounding conditions, Based on information regarding at least one of the aforementioned driving conditions and surrounding conditions, vehicle control is performed using a machine learning model. If the detection performance of the aforementioned sensor is lower than a specified level, the detection result of the sensor will be displayed. If a situation occurs in the vehicle where the risk is higher than the specified value, the detection result of the sensor corresponding to that situation will be presented. An automated driving system that, when the vehicle changes lanes to an adjacent lane due to driver intervention, and the distance between the vehicle's trajectory before the lane change and another vehicle traveling in the adjacent lane is below a threshold, indicates that a situation with a risk higher than a specified value has occurred in the vehicle, and presents the detection results of the rear-monitoring sensor and the side-monitoring sensor.

3. The automated driving system according to claim 1 or 2, wherein if the detection performance of the sensor is lower than the specified level, at least one of the following occurs: the sensor has not detected any surrounding objects or lane markings; the stability of the sensor is lower than past stability; the stability of the sensor is lower than the design value; the stability of the sensor is lower than the stability of other sensors whose fields of view overlap with that sensor; and the stability of the sensor is lower than the stability of other sensors of other vehicles that have traveled in the same location.

Images