Automatic driving system

Abstract

To provide an automatic driving system in which vehicle control according to a mechanical learning model is effectively monitored.SOLUTION: An automatic driving system 1 detects information related to a travel situation and a peripheral situation of a vehicle V according to a sensor 2, executes vehicle control using a mechanical learning model M on the basis of the information related to a travel situation and a peripheral situation, and presents a detection result of the sensor 2 if detection performance of the sensor 2 is lower than a regulated level, and presents a detection result of the sensor 2 corresponding to the situation if the vehicle V is subjected to a situation where the risk is higher than a regulated value.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] One aspect of the present invention relates to an automated driving system. [Background technology]

[0002] There is known an autonomous driving system that performs autonomous driving of a vehicle by vehicle control using a machine learning model. As an example of this type of technology, Patent Document 1 describes a teacher data collection device that collects teacher data that can be used for machine learning to generate an autonomous driving model. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2019 / 116423 Summary of the Invention [Problem to be solved by the invention]

[0004] However, a machine learning model generated by machine learning does not necessarily have the ability to realize appropriate vehicle control in all situations. The range in which vehicle control can be appropriately performed using a machine learning model may change depending on traffic conditions such as weather, time of day, and traffic volume. Therefore, in order to appropriately operate vehicle control using a machine learning model, it is desirable to effectively monitor the vehicle control.

[0005] Therefore, one aspect of the present invention aims to provide an autonomous driving system capable of effectively monitoring vehicle control using a machine learning model. [Means for solving the problem]

[0006] An autonomous driving system according to one aspect of the present invention detects information regarding at least one of the vehicle's driving conditions and surrounding conditions using one or more sensors, performs vehicle control using a machine learning model based on the information regarding at least one of the driving conditions and surrounding conditions, and presents the detection result of the sensor if the detection performance of the sensor is lower than a specified level, and presents the detection result of the sensor corresponding to the situation if a situation occurs in the vehicle where the risk is higher than a specified value.

[0007] An autonomous driving system according to one aspect of the present invention may be in at least any of the following cases: when the detection performance of a sensor is lower than a specified level, when the sensor does not detect surrounding objects or lane lines, when the stability of the sensor is lower than past stability, when the stability of the sensor is lower than a design value, when the stability of the sensor is lower than the stability of other sensors whose field of view overlaps with the sensor, and when the stability of the sensor is lower than the stability of other sensors of other vehicles traveling in the same location. Effect of the Invention

[0008] According to one aspect of the present invention, it is possible to provide an autonomous driving system capable of effectively monitoring vehicle control using a machine learning model. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing a configuration of an autonomous driving system according to an embodiment. [Diagram 2] FIG. 2 is a flowchart showing an example of processing by the autonomous driving system of FIG. [Diagram 3] FIG. 3 is a flowchart showing another example of the processing by the autonomous driving system of FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or corresponding elements are designated by the same reference numerals, and duplicated explanations will be omitted.

[0011] As shown in FIG. 1, an autonomous driving system 1 according to an embodiment is a system that autonomously drives a vehicle V by vehicle control using a machine learning model. The autonomous driving system 1 is mounted on the vehicle V. The vehicle V may be a passenger car or a cargo vehicle. The vehicle V can accommodate one or more passengers. The vehicle V is an autonomous driving vehicle capable of autonomous driving. The vehicle V may also be capable of manual driving by a driver.

[0012] The autonomous driving system 1 includes a sensor 2, an actuator 3, a communication unit 4, an autonomous driving ECU 5 [Electronic Control Unit], and an HMI [Human Machine Interface]. The sensor 2 is a sensor that detects information on at least one of the driving conditions and the surrounding conditions of the vehicle V. The sensor 2 includes an external sensor and an internal sensor. The external sensor is a sensor that acquires the surrounding environment of the vehicle V. The external sensor includes at least one of a camera, a millimeter wave radar, a LIDAR [Light Detection and Ranging], and the like. The internal sensor is a detection device that detects the driving condition of the vehicle V. The external sensor includes a sensor that monitors the front of the vehicle V, a sensor that monitors the rear of the vehicle V, and a sensor that monitors the side of the vehicle V. The internal sensor includes at least one of a vehicle speed sensor, an acceleration sensor, a yaw rate sensor, and the like. The sensor 2 transmits the detection result to the autonomous driving ECU 5.

[0013] The actuator 3 is a controller for controlling the speed of the vehicle V. The actuator 3 may include, for example, an actuator for controlling the output of the engine or the motor, and a brake actuator. The communication unit 4 is a communication device for controlling wireless communication with the outside of the vehicle V. The communication unit 4 communicates various information with the server 100 for the autonomous driving system via, for example, the communication network N. The communication unit 4 communicates various information with, for example, other vehicles in the vicinity. The communication unit 4 is not particularly limited, and various known communication devices can be used.

[0014] The autonomous driving ECU 5 is an electronic control unit having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The autonomous driving ECU 5, for example, loads a program recorded in the ROM into the RAM and executes the program loaded into the RAM with the CPU, thereby realizing various functions. The autonomous driving ECU 5 may be composed of multiple electronic control units.

[0015] The autonomous driving ECU 5 executes vehicle control using the machine learning model M based on the detection results of the sensor 2 (information on at least one of the driving conditions and the surrounding conditions of the vehicle V). For example, the autonomous driving ECU 5 inputs the detection results of the sensor 2 to the machine learning model M and outputs a control signal obtained thereby to the actuator 3, thereby controlling the driving, braking, and steering of the vehicle V to perform autonomous driving.

[0016] The machine learning model M is a recurrent deep learning model. The machine learning model M is a recurrent neural network [RNN: Recurrent neural network]. A convolutional neural network [CNN: Convolutional Neural Network] including multiple layers including multiple convolution layers and pooling layers may be used as at least a part of the neural network. In the machine learning model M, deep learning is performed by deep learning. The machine learning model M is a trained model trained using data of the vehicle V under a predetermined learning condition. For example, the machine learning model M may be trained using teacher data regarding the output to the actuator 3 when various detection results of the sensor 2 are input.

[0017] The HMI 6 is an interface for inputting and outputting various types of information to and from an occupant (natural person) of the vehicle V. The HMI 6 includes, for example, a display and a speaker. The HMI 6 outputs images on the display and sounds from the speaker in response to a control signal from the ECU 10. The HMI 6 may include a HUD (Head Up Display). The HMI 6 presents various types of information to the occupant of the vehicle V.

[0018] The server 100 for the autonomous driving system is a server capable of communicating with the autonomous driving system 1. The server 100 for the autonomous driving system includes a communication unit 101 and a display unit 102. The communication unit 101 communicates various information with the vehicle V, for example, via a communication network N. The communication unit 101 is not particularly limited, and various known communication devices can be used. The display unit 102 is an interface that presents various information to a user (natural person). The display unit 102 presents various information to the user by image output. The display unit 102 is not particularly limited, and various known display devices can be used.

[0019] In this embodiment, when the detection performance of the sensor 2 is lower than a specified level, the autonomous driving ECU 5 presents the detection result of the sensor 2 through the HMI 6. When a situation in which the risk is higher than a specified value occurs in the vehicle V, the autonomous driving ECU 5 presents the detection result of the sensor 2 corresponding to the situation through the HMI 6. Instead of or in addition to being presented through the HMI 6, the presentation of the detection result of the sensor 2 may be performed through the display unit 102 via the communication units 4, 101 (the same applies below). The sensor 2 corresponding to a situation in which the risk is higher than a specified value is, for example, a sensor that directly or indirectly detects the situation and parameters related thereto.

[0020] The detection performance of the sensor 2 is lower than the specified level when the sensor 2 does not detect a surrounding object or a lane marking, when the stability of the sensor 2 is lower than the past stability, when the stability of the sensor 2 is lower than the design value, when the stability of the sensor 2 is lower than the stability of other sensors whose fields of view overlap with the sensor 2, and when the stability of the sensor 2 is lower than the stability of other sensors of other vehicles that have traveled in the same place. The surrounding objects are not particularly limited and include, for example, moving obstacles and fixed obstacles. The stability of the sensor 2 means the general stability of a sensor, and is, for example, the resistance to fluctuation of the detection value detected in a certain situation.

[0021] For example, when at least one of the acceleration / deceleration and steering angular velocity of the vehicle V is equal to or greater than a threshold, the autonomous driving ECU 5 determines that a situation in which the risk is higher than a specified value is occurring in the vehicle V, and presents the detection result of the sensor monitoring the forward direction among the sensors 2. The threshold may be set in advance and stored in the autonomous driving ECU 5, and may be a fixed value or a variable value. The risk is, for example, a risk related to the driving of the vehicle V. The specified value is a value set in advance. The risk and the specified value are not particularly limited, and may be various known risks and specified values.

[0022] For example, when the vehicle V changes lanes to an adjacent lane due to intervention by the driver of the vehicle V, if the distance between the travel path of the vehicle V before the lane change and another vehicle traveling in the adjacent lane is equal to or less than a threshold, the autonomous driving ECU 5 determines that a situation in which the risk is higher than a specified value is occurring in the vehicle V, and presents the detection results of the rearward monitoring sensor and the sideward monitoring sensor among the sensors 2. The threshold may be set in advance and stored in the autonomous driving ECU 5, and may be a fixed value or a variable value.

[0023] When presenting the detection result of the sensor 2, the autonomous driving ECU 5 may present the detection result of the sensor 2 together with sample data. The sample data may be predetermined and stored in the autonomous driving ECU 5. The sample data is data that serves as a sample of the detection result of the sensor 2, such as data when the sensor 2 is normal. This makes it easy to grasp whether the detection result of the sensor 2 is abnormal or normal.

[0024] When a situation occurs in the vehicle V where the risk is higher than a specified value, the autonomous driving ECU 5 may preferentially present the detection result of the second sensor of the sensors 2, which has a higher influence on the situation than the first sensor, via at least one of the HMI 6 and the display unit 102. This allows the detection result of the sensor 2, which has a greater influence on the situation where the risk is higher than a specified value, to be grasped preferentially.

[0025] Next, an example of a monitoring process in which the autonomous driving system 1 of this embodiment monitors the detection result of the sensor 2 will be described with reference to the flowcharts of Figures 2 and 3. The monitoring process here may be performed, for example, while the vehicle V is traveling, and when the process reaches an end, the process may be restarted from the start after a predetermined time.

[0026] 2, the autonomous driving ECU 5 acquires the detection result of the sensor 2 (step S1). The autonomous driving ECU 5 determines whether or not there is a sensor 2 whose detection performance is lower than a specified level based on the detection result of the sensor 2 (step S2). For example, in the above step S2, it may be determined that there is a sensor 2 whose detection performance is lower than a specified level in at least any of the following cases: when a surrounding object or a lane marking is not detected in the detection result of the sensor 2, when the current stability of the sensor 2 is lower than the past stability, when the stability of the sensor 2 is lower than the design value, when the stability of the sensor 2 is lower than the stability of other sensors whose detection ranges overlap with the sensor 2, and when the stability of the sensor 2 is lower than the stability of other sensors of other vehicles that have traveled in the same place.

[0027] If the answer is NO in step S2, the process of the current cycle is terminated and the process proceeds to step S1 in the next cycle. On the other hand, if the answer is YES in step S2, the detection result of the sensor 2 whose detection performance is lower than the specified level is presented by the HMI 6 together with the sample data (step S3).

[0028] 3, the autonomous driving ECU 5 acquires the detection result of the sensor 2 (step S11). The autonomous driving ECU 5 determines whether or not a situation in which the risk is higher than a specified value has occurred based on the detection result of the sensor 2 (step S12). For example, in the above step S12, it may be determined that a situation in which the risk is higher than a specified value has occurred when at least one of the acceleration / deceleration and steering angular velocity of the vehicle V is equal to or higher than a threshold, when the vehicle V decelerates by a threshold or more due to the driver's intervention, when a lane change is performed by the driver's intervention, or when the planned result of the driving route of the vehicle V is discontinuous in time.

[0029] If the answer is NO in step S12, the process of the current cycle is terminated, and the process proceeds to step S11 in the next cycle. On the other hand, if the answer is YES in step S12, the detection result of the sensor 2 corresponding to a situation in which the risk is higher than a specified value is presented by the HMI 6 together with sample data (step S13). In step S13, if at least one of the acceleration / deceleration and steering angular velocity of the vehicle V is equal to or greater than a threshold, the detection result of the front sensor may be presented. In step S13, if the vehicle V decelerates by a threshold or more due to driver intervention, the detection result of the front sensor may be presented. In step S13, if a lane change is performed due to driver intervention, the detection results of the rear sensor and the side sensor may be presented. Presentation of the detection result may include presentation of an image of the road surface, and presentation of the position of the vehicle V relative to the recognized white line.

[0030] Note that only one of the above steps S1 and S11 may be executed. If the above step S1 is not executed, the detection result of the sensor 2 acquired in the above step S11 may be used in the determination of the above step S2. If the above step S11 is not executed, the detection result of the sensor 2 acquired in the above step S1 may be used in the determination of the above step S12.

[0031] As described above, in the autonomous driving system 1, when autonomous driving of the vehicle V is performed by vehicle control using the machine learning model M, by grasping the detection result of the sensor 2 with low detection performance, for example, when the autonomous driving response to the surrounding situation of the vehicle V is delayed, it is possible to easily grasp whether or not the delayed response is caused by the sensor 2. In addition, when a situation occurs in which the risk is higher than a specified value, by grasping the detection result of the sensor 2 that affects the situation, it is possible to easily grasp whether or not the situation is caused by the sensor 2. In other words, according to the autonomous driving system 1, it is possible to effectively monitor the vehicle control using the machine learning model M.

[0032] In the autonomous driving system 1, the detection performance of the sensor 2 is lower than the specified level when the sensor 2 does not detect surrounding objects or lane lines, when the stability of the sensor 2 is lower than past stability, when the stability of the sensor 2 is lower than a design value, when the stability of the sensor 2 is lower than the stability of other sensors whose fields of view overlap with the sensor 2, and when the stability of the sensor 2 is lower than the stability of other sensors of other vehicles that have traveled in the same place. In this case, it is possible to specifically determine when the detection performance of the sensor is lower than the specified level.

[0033] In the autonomous driving system 1, when at least one of the acceleration / deceleration and steering angular velocity of the vehicle V is equal to or greater than a threshold, it indicates that a situation in which the risk is higher than a specified value is occurring in the vehicle V, and presents the detection result of the sensor 2 monitoring the forward direction. This makes it easy to grasp the detection result of the sensor 2 that affects the situation when at least one of the acceleration / deceleration and steering angular velocity of the vehicle V is equal to or greater than a threshold.

[0034] In the autonomous driving system 1, when the vehicle V changes lanes to an adjacent lane due to the driver's intervention, if the distance between the travel path of the vehicle before the lane change and another vehicle traveling in the adjacent lane is equal to or less than a threshold, it may be determined that a situation in which the risk is higher than a specified value has occurred in the vehicle V, and the detection results of the sensor 2 monitoring the rear and the sensor 2 monitoring the side may be presented. This makes it easy to grasp the detection results of the sensor 2 that affect the situation in which the possibility of contact with another vehicle during a lane change is high.

[0035] Although the embodiment has been described above, one aspect of the present invention is not limited to the above embodiment. One aspect of the present invention can be embodied in various forms including the above embodiment and various modifications and improvements based on the knowledge of a person skilled in the art.

[0036] In the above embodiment, the type of the sensor 2 is not limited, and may include various detection devices. The number of sensors 2 mounted is also not limited, and may be one or more. In the above embodiment, the case where the detection performance of the sensor 2 is lower than the specified level is not particularly limited, and may be a case other than the above-mentioned case. In the above embodiment, the situation where the risk is higher than the specified value is not particularly limited, and may be a situation other than the above-mentioned case.

[0037] In the above embodiment, when the camera and the LIDAR output detection results separately, the detection performance of each of the camera and the LIDAR may be determined based on the detection results of each of the camera and the LIDAR. On the other hand, when the camera and the LIDAR are integrated to output one detection result, it may be determined whether the detection performance of the camera and the LIDAR is low or not based on the one detection result. In the above embodiment, when the reliability of one detection result integrated with the multiple sensors 2 is low, the detection result of the multiple sensors 2 may be presented as having low detection performance. [Explanation of symbols]

[0038] 1...autonomous driving system, 2...sensor, M...machine learning model, V...vehicle.

Claims

1. Detecting information regarding at least one of a vehicle's traveling conditions and a surrounding condition using one or more sensors; Carrying out vehicle control using a machine learning model based on information regarding at least one of the driving situation and the surrounding situation; If the detection performance of the sensor is lower than a specified level, presenting the detection result of the sensor; An autonomous driving system that, when a situation occurs in the vehicle in which the risk is higher than a specified value, presents the detection results of the sensor corresponding to the situation.

2. The autonomous driving system of claim 1, wherein the detection performance of the sensor is lower than the specified level when at least one of the following occurs: the sensor does not detect surrounding objects or lane lines; the stability of the sensor is lower than past stability; the stability of the sensor is lower than a design value; the stability of the sensor is lower than the stability of other sensors whose field of view overlaps with the sensor; and the stability of the sensor is lower than the stability of other sensors of other vehicles that have traveled in the same location.

3. 3. The autonomous driving system according to claim 1, wherein when at least one of the acceleration / deceleration and steering angular velocity of the vehicle is equal to or greater than a threshold, the detection result of the sensor monitoring the forward direction is presented, indicating that a situation in which the risk is higher than a specified value is occurring in the vehicle.

4. 3. The autonomous driving system according to claim 1, wherein when the vehicle changes lanes to an adjacent lane due to driver intervention, if the distance between the driving trajectory of the vehicle before the lane change and another vehicle traveling in the adjacent lane is equal to or less than a threshold, the autonomous driving system indicates that a situation in which the risk is higher than a specified value is occurring in the vehicle, and presents the detection results of the sensor monitoring the rear and the detection results of the sensor monitoring the side.

Images