Server for autonomous driving systems

Abstract

To provide a server for an automatic operation system capable of effectively monitoring vehicle control by a machine learning model.SOLUTION: A server 100 for an automatic operation system is the server capable of communicating with an automatic operation system 1 for performing an automatic operation of a vehicle V by vehicle control with the use of a machine learning model M. The server 100 for the automatic operation system acquires vehicle control information being the information related to the vehicle control, and then extracts a recognition object to be recognized in the periphery of the vehicle V during vehicle control and the travel conditions of the vehicle V during vehicle control from the vehicle control information when a situation where difficulty is estimated in performing coping of the automatic operation occurs in the vehicle V.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] One aspect of the present invention relates to a server for an automatic driving system.

Background Art

[0002] There is known a server communicable with an automatic driving system that performs automatic driving of a vehicle by controlling the vehicle using a machine learning model. As this type of technology, for example, Patent Document 1 describes a teacher data collection device that collects teacher data that can be used for machine learning for generating an automatic driving model.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, a machine learning model generated by machine learning does not necessarily have the ability to realize appropriate vehicle control in all scenarios. The range in which vehicle control can be appropriately executed using a machine learning model can vary depending on traffic environments such as weather, time of day, traffic volume, etc. Therefore, in order to appropriately apply vehicle control by a machine learning model, it is desirable to effectively monitor the vehicle control.

[0005] Therefore, an object of one aspect of the present invention is to provide a server for an automatic driving system that can effectively monitor vehicle control by a machine learning model.

Means for Solving the Problems

[0006] A server for an autonomous driving system according to one aspect of the present invention is a server that can communicate with an autonomous driving system that performs autonomous driving of a vehicle by vehicle control using a machine learning model, and acquires vehicle control information, which is information related to vehicle control, and when a situation occurs in the vehicle in which it is estimated that it is difficult to respond to autonomous driving, it extracts from the vehicle control information at least one of the recognition object recognized in the vicinity of the vehicle during vehicle control and the driving conditions of the vehicle during vehicle control.

[0007] A server for an automated driving system according to one aspect of the present invention has predetermined conditions under which at least one of several events occurs, which are considered to be situations in which it is estimated that automated driving will be difficult. If at least one of the several events occurs in the vehicle, the server may extract at least one of the recognition target and driving conditions. If at least one of the several events occurs in the vehicle, it may be at least one of the following: the vehicle driver intervenes in automated driving; the vehicle's acceleration / deceleration or steering angular velocity is above a threshold; the distance between the vehicle and another vehicle is less than a threshold; the vehicle's lateral position is unstable; or the vehicle performs a behavior that surprises the driver to a degree exceeding a threshold. [Effects of the Invention]

[0008] According to one aspect of the present invention, it is possible to provide a server for an autonomous driving system that can effectively monitor vehicle control using a machine learning model. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a block diagram showing the configuration of a server for an autonomous driving system and an autonomous driving system according to one embodiment. [Figure 2] Figure 2 is a flowchart showing an example of processing performed by the server for the autonomous driving system shown in Figure 1. [Figure 3] Figure 3 is a flowchart that continues from Figure 2. [Figure 4]Figure 4 is a block diagram showing the configuration of the server for the automated driving system and the automated driving system in a modified example. [Figure 5] Figure 5 is a flowchart showing an example of processing performed by the server for the autonomous driving system shown in Figure 4. [Modes for carrying out the invention]

[0010] The embodiments will be described in detail below with reference to the attached drawings. In the description of the drawings, the same or equivalent elements will be denoted by the same reference numeral, and redundant descriptions will be omitted.

[0011] As shown in Figure 1, the server 100 for the autonomous driving system according to this embodiment is a server that can communicate with the autonomous driving system 1, which performs autonomous driving of vehicle V by vehicle control using a machine learning model.

[0012] The autonomous driving system 1 is installed in vehicle V. Vehicle V may be a passenger car or a cargo vehicle. Vehicle V can accommodate one or more occupants. Vehicle V is an autonomous driving vehicle capable of autonomous driving. Vehicle V may also be capable of manual driving by a driver.

[0013] The autonomous driving system 1 comprises a sensor 2, an actuator 3, a communication unit 4, and an autonomous driving ECU 5 (Electronic Control Unit). Sensor 2 includes an external sensor and an internal sensor. The external sensor is a sensor that acquires information about the surrounding environment of the vehicle V. The external sensor includes, for example, at least one of a camera, millimeter-wave radar, or lidar (Light Detection and Ranging). The internal sensor is a detection device that detects the driving state of the vehicle V. The internal sensor includes at least one of a vehicle speed sensor, an acceleration sensor, and a yaw rate sensor. Sensor 2 transmits the detection results to the autonomous driving ECU 5.

[0014] Actuator 3 is a controller for controlling the speed of the vehicle V. Actuator 3 may include, for example, an actuator for controlling the output of an engine or motor, and a brake actuator. Communication unit 4 is a communication device that controls wireless communication between the vehicle V and the outside world. Communication unit 4 communicates various information with the server 100 for the automated driving system via, for example, a communication network N. Communication unit 4 is not particularly limited, and various known communication devices can be used.

[0015] The autonomous driving ECU 5 is an electronic control unit that includes a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc. The autonomous driving ECU 5 implements various functions, for example, by loading a program recorded in ROM into RAM and executing the program loaded into RAM with the CPU. The autonomous driving ECU 5 may be composed of multiple electronic control units.

[0016] The autonomous driving ECU 5 performs vehicle control using a machine learning model M based on the detection results of sensor 2. For example, the autonomous driving ECU 5 inputs the detection results of sensor 2 into the machine learning model M and outputs the resulting control signals to actuator 3, thereby controlling the driving, braking, and steering of vehicle V to perform autonomous driving.

[0017] The machine learning model M is a recursive deep learning model. The machine learning model M is a recurrent neural network [RNN: Recurrent neural network]. A convolutional neural network [CNN: Convolutional Neural Network] including a plurality of layers including a plurality of convolutional layers and pooling layers may be used for at least a part of the neural network. In the machine learning model M, deep learning by deep learning is performed. The machine learning model M is a trained model trained using data of the vehicle V under predetermined learning conditions. For example, the machine learning model M may be trained using teacher data for the output to the actuator 3 when various detection results of the sensor 2 are input.

[0018] The vehicle V transmits vehicle control information, which is information related to vehicle control of the vehicle V, to the server 100 for the automatic driving system via the communication unit 4. The vehicle control information includes at least any one of the detection result of the sensor 2, the weather in the area where the vehicle V is located, the current time, the road surface information of the road surface on which the vehicle V travels, the country in which the vehicle V is traveling, and the like. The vehicle control information can be appropriately acquired by known methods or devices. The transmission of the vehicle control information to the server 100 for the automatic driving system may be performed periodically.

[0019] The server 100 for the automatic driving system of the present embodiment includes a communication unit 101, a display unit 102, and a processing unit 103. The communication unit 101 communicates various information with the vehicle V via, for example, the communication network N. The communication unit 101 is not particularly limited, and various known communication devices can be used. The display unit 102 is an interface that presents various information to the user. The display unit 102 presents various information to the user by image output. The display unit 102 is not particularly limited, and various known display devices can be used.

[0020] The processing unit 103 acquires vehicle control information via the communication unit 101. When a situation where it is estimated that it is difficult to handle autonomous driving occurs in the vehicle V, the processing unit 103 extracts at least one of the recognition target recognized around the vehicle V during vehicle control and the driving conditions of the vehicle V during vehicle control from the vehicle control information.

[0021] It is predetermined that at least one of a plurality of events occurs as a situation where it is estimated that it is difficult to handle autonomous driving in the processing unit 103. The processing unit 103 determines whether or not at least one of the plurality of events has occurred in the vehicle V based on the vehicle control information. When the processing unit 103 determines that at least one of the plurality of events has occurred in the vehicle V, the processing unit 103 extracts at least one of the recognition target and the driving conditions from the vehicle control information.

[0022] The case where an event has occurred in the vehicle V means that the driver of the vehicle V has intervened in the autonomous driving, the acceleration / deceleration or the steering angular velocity of the vehicle V is equal to or greater than the threshold value, the distance between the vehicle V and another vehicle is smaller than the threshold value, the lateral position of the vehicle V is unstable, and the vehicle V has behaved with a degree of surprise of the driver equal to or greater than the threshold value. The event determined by the processing unit 103 is an event that the machine learning model M cannot fully handle.

[0023] Each threshold value may be predetermined and stored in the autonomous driving system server 100. Each threshold value may be a fixed value or a variable value. The recognition target is a target recognized by the vehicle V, for example, an object and an obstacle (such as a pedestrian, a bicycle, a motorcycle, another vehicle, etc.) around the vehicle V. The driving conditions are the conditions under which the vehicle V travels, for example, the weather in the area where the vehicle V is located, the current time, the road surface information (such as whether it is soil, stone paving, or asphalt) of the road surface on which the vehicle V travels, the country in which the vehicle V is traveling, etc. Each threshold value, recognition target, and driving condition are not particularly limited. Whether or not an event has occurred in the vehicle V can be determined using various known methods.

[0024] When the processing unit 103 extracts at least one of the recognition target and driving conditions, it classifies the extraction results for each of the multiple events that occurred. The processing unit 103 may present the extraction results in a list via the display unit 102. The manner in which the list is presented is not particularly limited and may be in various forms, or it may be a list for each event that occurred. The processing unit 103 may also present the extraction results with priority assigned via the display unit 102. The priority may be set, for example, according to the frequency and importance of the events that occurred. The priority may be expressed as a numerical value or as a degree.

[0025] Next, an example of a monitoring process for monitoring the vehicle control of vehicle V by the server 100 for the automated driving system of this embodiment will be described with reference to the flowchart in Figure 2. This monitoring process may be performed, for example, while vehicle V is in motion, and when the process reaches its end, it may be restarted from the beginning after a predetermined time.

[0026] As shown in Figure 2, first, the processing unit 103 acquires vehicle control information from the vehicle V via the communication unit 101 (step S1). Based on the vehicle control information, the processing unit 103 determines whether at least one of the predetermined events has occurred (step S2). If the result in step S2 is NO, the processing for the current cycle is terminated and the process moves to step S1 of the next cycle. On the other hand, if the result in step S2 is YES, the occurring events are assigned to the vehicle control information, and the vehicle control information is classified by event (step S3).

[0027] The processing unit 103 extracts the recognition target and driving conditions included in the occurring event based on the vehicle control information (step S4). In step S4, only one of the recognition target or driving conditions may be extracted. The processing unit 103 calculates a priority according to the frequency and importance of the occurring event and assigns the priority to the extracted recognition target and driving conditions (step S5). The processing unit 103 displays the recognition target and driving conditions with assigned priorities in a list separated by the occurring event on the display unit 102 (step S6).

[0028] In the server 100 for the autonomous driving system of this embodiment, if the reliability of the vehicle control is lower than a threshold, the processing unit 103 may assume that a situation has occurred in the vehicle where it is difficult to respond to autonomous driving, and may extract at least one of the recognition target and driving conditions. Specifically, referring to the flowchart in Figure 3, first, the processing unit 103 acquires vehicle control information from the vehicle V via the communication unit 101 (step S11). Based on the vehicle control information, the processing unit 103 determines whether or not the reliability of the vehicle control is lower than a threshold (step S12).

[0029] The reliability of vehicle control can be determined using a known method. For example, the reliability may be set lower if other vehicles are present around vehicle V. Alternatively, the reliability may be set higher if the safety of the situation of vehicle V is relative to that of other vehicles. Furthermore, if a second event occurs that occurs more frequently than the first event, the reliability may be set higher than when the first event occurred. The threshold in step S12 above is set in advance and stored. This threshold is not particularly limited and may be a fixed value or a variable value.

[0030] Next, based on the vehicle control information, the recognition target and driving conditions are extracted (step S13). A confidence level is assigned to the extracted recognition target and driving conditions. The processing unit 103 displays a list of the recognition target and driving conditions with the assigned confidence level on the display unit 102 (step S14).

[0031] In summary, the autonomous driving system server 100 can grasp recognition targets and / or driving conditions that are difficult for the machine learning model M to handle (conditions that the system struggles with). In other words, it becomes possible to effectively monitor the vehicle control by the machine learning model M. The autonomous driving system server 100 (for example, the operator's side) can improve the efficiency of monitoring the autonomous driving system 1.

[0032] The server 100 for the autonomous driving system has predetermined conditions under which autonomous driving is expected to be difficult, namely the occurrence of at least one of several events. If at least one of these events occurs in vehicle V, the server extracts at least one of the recognition target and / or driving conditions. In this case, the occurrence of events makes it possible to understand recognition targets and / or driving conditions that are difficult to handle with vehicle control by the machine learning model M.

[0033] In the server 100 for the autonomous driving system, if at least one of several events occurs in vehicle V, it means that at least one of the following has occurred: the driver of vehicle V has intervened in autonomous driving; the acceleration / deceleration or steering angular velocity of vehicle V is above a threshold; the distance to another vehicle VT is below a threshold; the lateral position of vehicle V is unstable; or vehicle V has performed an action that would surprise the driver above a threshold. In this case, it becomes possible to specifically use the occurrence of events to grasp recognition targets and / or driving conditions that are difficult to handle with vehicle control by machine learning model M.

[0034] In the autonomous driving system server 100, if the reliability of vehicle control is lower than a threshold, it may be assumed that a situation has occurred in vehicle V where the difficulty of autonomous driving is higher than a specified value, and at least one of the recognition target and driving conditions may be extracted. This makes it possible to use the reliability of vehicle control to identify recognition targets and / or driving conditions that are difficult to handle with vehicle control by the machine learning model M.

[0035] The autonomous driving system server 100 classifies the extraction results for each of multiple events. The autonomous driving system server 100 presents the extraction results in a list. The autonomous driving system server 100 presents the extraction results with priority assigned to them. In at least one of these cases, the extraction results can be easily understood. Furthermore, the reasons or conditions under which the recognition target and / or driving conditions were extracted can also be understood.

[0036] Although embodiments have been described above, the present invention is not limited to the embodiments described above. One embodiment of the present invention can be implemented in various forms, starting with the embodiments described above, with various modifications and improvements based on the knowledge of those skilled in the art.

[0037] In the automated driving system server 100 of the above embodiment, as shown in Figure 4, the processing unit 103 may estimate, based on information from other vehicle VTs, whether or not a situation has occurred in vehicle V that makes automated driving difficult. Specifically, other vehicle VTs transmit vehicle control information, which is information related to the vehicle control of vehicle V, to the automated driving system server 100 via a communication unit 13 similar to the communication unit 4 (see Figure 1). The vehicle control information includes at least one of the following: the detection results of sensors 12 similar to sensor 2, the weather in the area where vehicle V is located, the current time, road surface information of the road surface on which vehicle V is traveling, and the country in which vehicle V is traveling. Vehicle control information can be acquired by other vehicle VTs using known methods or equipment as appropriate. Transmission of vehicle control information to the automated driving system server 100 may be performed periodically. Note that there may be multiple other vehicle VTs.

[0038] In this monitoring process, as shown in Figure 5, for example, the processing unit 103 first acquires vehicle control information from other vehicle VTs via the communication unit 101 (step S21). Based on the vehicle control information, the processing unit 103 determines whether at least one of the predetermined events has occurred (step S22). If the result in step S22 is NO, the processing for the current cycle ends and proceeds to step S21 of the next cycle. If the result in step S22 is YES, the processing unit 103 assigns the occurring event to the vehicle control information and classifies the vehicle control information by event (step S23).

[0039] The processing unit 103 extracts the recognition target and driving conditions included in the occurring event from the vehicle control information (step S24). The processing unit 103 calculates a priority according to the frequency and importance of the occurring event and assigns the priority to the recognition target and driving conditions (step S25). The processing unit 103 displays the recognition target and driving conditions with assigned priorities in a list separated by the occurring event on the display unit 102 (step S26).

[0040] Thus, the modified automated driving system server 100 can estimate, based on information from other vehicles VT, whether or not a situation has occurred in vehicle V where automated driving is presumed to be difficult. By utilizing information from other vehicles VT, it becomes possible to grasp recognition targets and / or driving conditions that are difficult to handle with vehicle control by the machine learning model M. [Explanation of symbols]

[0041] 1...Autonomous driving system, 2...Sensor, 100...Server for autonomous driving system, M...Machine learning model, V...Vehicle, VT...Other vehicles.

Claims

1. A server capable of communicating with an autonomous driving system that performs autonomous driving of a vehicle using a machine learning model, Vehicle control information, which is information related to the aforementioned vehicle control, is acquired. The situations in which it is estimated that the aforementioned autonomous driving response will be difficult are predetermined to occur if at least one of several events occurs. If at least one of the multiple events described above occurs in the vehicle, at least one of the recognition object recognized around the vehicle during vehicle control and at least one of the vehicle's driving conditions during vehicle control are extracted from the vehicle control information. For each of the multiple events that occurred, the extracted results are classified, a priority is calculated according to the frequency and importance of the events that occurred, and the extracted results are presented with this priority assigned to them. A server for an autonomous driving system, which includes cases where, if at least one of the multiple events described above occurs in the vehicle, the vehicle performs an action that surprises the driver of the vehicle to a certain degree or greater than a threshold.

2. A server for an automated driving system according to claim 1, wherein if at least one of the multiple events described above occurs in the vehicle, the driver of the vehicle intervenes in automated driving, the acceleration / deceleration or steering angular velocity of the vehicle is greater than or equal to a threshold, the distance between the vehicle and another vehicle is less than a threshold, and the lateral position of the vehicle is unstable.

3. A server for an automated driving system according to claim 1 or 2, wherein, when the reliability of the vehicle control is lower than a threshold, it is estimated that a situation has occurred in the vehicle in which it is difficult to respond to the automated driving, and the server extracts at least one of the recognition target and the driving conditions.

4. A server for an automated driving system according to claim 1 or 2, which estimates whether a situation has occurred in the vehicle that makes it difficult to respond to the automated driving, based on information from other vehicles.

Images