Server for automatic operation system

Abstract

To provide a server for an automatic operation system capable of effectively monitoring vehicle control by a machine learning model.SOLUTION: A server 100 for an automatic operation system is the server capable of communicating with an automatic operation system 1 for performing an automatic operation of a vehicle V by vehicle control with the use of a machine learning model M. The server 100 for the automatic operation system acquires vehicle control information being the information related to the vehicle control, and then extracts a recognition object to be recognized in the periphery of the vehicle V during vehicle control and the travel conditions of the vehicle V during vehicle control from the vehicle control information when a situation where difficulty is estimated in performing coping of the automatic operation occurs in the vehicle V.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] One aspect of the present invention relates to a server for an autonomous driving system. [Background technology]

[0002] There is known a server capable of communicating with an autonomous driving system that performs autonomous driving of a vehicle by vehicle control using a machine learning model. As an example of this type of technology, Patent Document 1 describes a teacher data collection device that collects teacher data that can be used for machine learning to generate an autonomous driving model. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2019 / 116423 Summary of the Invention [Problem to be solved by the invention]

[0004] However, a machine learning model generated by machine learning does not necessarily have the ability to realize appropriate vehicle control in all situations. The range in which vehicle control can be appropriately performed using a machine learning model may change depending on traffic conditions such as weather, time of day, and traffic volume. Therefore, in order to appropriately operate vehicle control using a machine learning model, it is desirable to effectively monitor the vehicle control.

[0005] Therefore, one aspect of the present invention aims to provide a server for an autonomous driving system that can effectively monitor vehicle control using a machine learning model. [Means for solving the problem]

[0006] A server for an autonomous driving system according to one aspect of the present invention is a server capable of communicating with an autonomous driving system that performs autonomous driving of a vehicle by controlling the vehicle using a machine learning model, and obtains vehicle control information, which is information relating to vehicle control, and when a situation occurs in the vehicle that is estimated to be difficult for autonomous driving to respond to, extracts from the vehicle control information at least one of a recognition object recognized in the vicinity of the vehicle during vehicle control and the driving conditions of the vehicle during vehicle control.

[0007] In the server for an autonomous driving system according to one aspect of the present invention, as a case where a situation occurs in which it is estimated that autonomous driving is difficult to deal with, a case where at least one of a plurality of events occurs is predetermined, and when at least one of the plurality of events occurs in the vehicle, at least one of the recognition target and the driving condition may be extracted. When at least one of the plurality of events occurs in the vehicle, the case may be at least one of the following: when the driver of the vehicle intervenes in autonomous driving, when the acceleration / deceleration or steering angular velocity of the vehicle is equal to or greater than a threshold, when the distance between the vehicle and another vehicle is smaller than a threshold, when the lateral position of the vehicle is unstable, and when the vehicle behaves in a manner that surprises the driver to a degree equal to or greater than a threshold. Effect of the Invention

[0008] According to one aspect of the present invention, it is possible to provide a server for an autonomous driving system capable of effectively monitoring vehicle control using a machine learning model. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing the configuration of a server for an autonomous driving system and an autonomous driving system according to an embodiment. [Diagram 2] FIG. 2 is a flowchart showing an example of processing by the server for the autonomous driving system of FIG. [Diagram 3] FIG. 3 is a flowchart showing a continuation of FIG. [Figure 4]FIG. 4 is a block diagram showing the configuration of a server for an autonomous driving system and an autonomous driving system according to a modified example. [Diagram 5] FIG. 5 is a flowchart showing an example of processing by the server for the autonomous driving system in FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. In the description of the drawings, the same or corresponding elements are designated by the same reference numerals, and duplicated explanations will be omitted.

[0011] As shown in FIG. 1, a server 100 for an autonomous driving system according to the embodiment is a server capable of communicating with an autonomous driving system 1 that autonomously drives a vehicle V by vehicle control using a machine learning model.

[0012] The autonomous driving system 1 is mounted on a vehicle V. The vehicle V may be a passenger car or a cargo vehicle. The vehicle V can accommodate one or more passengers. The vehicle V is an autonomous driving vehicle capable of autonomous driving. The vehicle V may also be capable of manual driving by a driver.

[0013] The autonomous driving system 1 includes a sensor 2, an actuator 3, a communication unit 4, and an autonomous driving ECU 5 [Electronic Control Unit]. The sensor 2 includes an external sensor and an internal sensor. The external sensor is a sensor that acquires the surrounding environment of the vehicle V. The external sensor includes at least one of a camera, a millimeter wave radar, a LIDAR (Light Detection and Ranging), etc. The internal sensor is a detection device that detects the driving state of the vehicle V. The internal sensor includes at least one of a vehicle speed sensor, an acceleration sensor, a yaw rate sensor, etc. The sensor 2 transmits the detection result to the autonomous driving ECU 5.

[0014] The actuator 3 is a controller for controlling the speed of the vehicle V. The actuator 3 may include, for example, an actuator for controlling the output of the engine or the motor, and a brake actuator. The communication unit 4 is a communication device for controlling wireless communication with the outside of the vehicle V. The communication unit 4 communicates various information with the server 100 for the autonomous driving system via, for example, the communication network N. The communication unit 4 is not particularly limited, and various known communication devices can be used.

[0015] The autonomous driving ECU 5 is an electronic control unit having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The autonomous driving ECU 5, for example, loads a program recorded in the ROM into the RAM and executes the program loaded into the RAM with the CPU, thereby realizing various functions. The autonomous driving ECU 5 may be composed of multiple electronic control units.

[0016] The autonomous driving ECU 5 executes vehicle control using the machine learning model M based on the detection result of the sensor 2. For example, the autonomous driving ECU 5 inputs the detection result of the sensor 2 to the machine learning model M and outputs a control signal obtained thereby to the actuator 3, thereby controlling the driving, braking, and steering of the vehicle V to perform autonomous driving.

[0017] The machine learning model M is a recurrent deep learning model. The machine learning model M is a recurrent neural network [RNN: Recurrent neural network]. A convolutional neural network [CNN: Convolutional Neural Network] including multiple layers including multiple convolution layers and pooling layers may be used as at least a part of the neural network. In the machine learning model M, deep learning is performed by deep learning. The machine learning model M is a trained model trained using data of the vehicle V under a predetermined learning condition. For example, the machine learning model M may be trained using teacher data regarding the output to the actuator 3 when various detection results of the sensor 2 are input.

[0018] The vehicle V transmits vehicle control information, which is information related to vehicle control of the vehicle V, to the server 100 for the autonomous driving system via the communication unit 4. The vehicle control information includes at least any of the detection results of the sensor 2, the weather in the area where the vehicle V is located, the current time, road surface information of the road surface on which the vehicle V is traveling, the country in which the vehicle V is traveling, etc. The vehicle control information can be appropriately acquired by known methods or devices. The transmission of the vehicle control information to the server 100 for the autonomous driving system may be performed periodically.

[0019] The server 100 for an autonomous driving system of this embodiment includes a communication unit 101, a display unit 102, and a processing unit 103. The communication unit 101 communicates various information with the vehicle V via, for example, a communication network N. The communication unit 101 is not particularly limited, and various known communication devices can be used. The display unit 102 is an interface that presents various information to the user. The display unit 102 presents various information to the user by image output. The display unit 102 is not particularly limited, and various known display devices can be used.

[0020] The processing unit 103 acquires vehicle control information via the communication unit 101. When a situation in which autonomous driving is estimated to be difficult occurs in the vehicle V, the processing unit 103 extracts at least one of a recognition target recognized in the periphery of the vehicle V during vehicle control and a driving condition of the vehicle V during vehicle control from the vehicle control information.

[0021] The processing unit 103 predetermines cases where at least one of a plurality of events occurs as cases where a situation occurs in which it is estimated that autonomous driving is difficult to handle. The processing unit 103 determines whether or not at least one of the plurality of events occurs in the vehicle V based on the vehicle control information. When the processing unit 103 determines that at least one of the plurality of events occurs in the vehicle V, it extracts at least one of the recognition target and the driving condition from the vehicle control information.

[0022] An event occurs in the vehicle V when at least one of the following occurs: the driver of the vehicle V intervenes in automatic driving, the acceleration / deceleration or steering angular velocity of the vehicle V is equal to or greater than a threshold, the distance between the vehicle V and another vehicle is smaller than a threshold, the lateral position of the vehicle V is unstable, or the vehicle V behaves in a manner that surprises the driver to a degree equal to or greater than a threshold. The events determined by the processing unit 103 are events that the machine learning model M cannot handle.

[0023] Each threshold may be determined in advance and stored in the server 100 for the autonomous driving system. Each threshold may be a fixed value or a variable value. The recognition target is an object recognized by the vehicle V, such as an object and an obstacle (pedestrian, bicycle, motorcycle, other vehicle, etc.) around the vehicle V. The driving condition is a condition under which the vehicle V drives, such as the weather in the area where the vehicle V exists, the current time, road surface information on the road surface on which the vehicle V drives (whether it is dirt, cobblestones, or asphalt, etc.), the country in which the vehicle V drives, etc. Each threshold, the recognition target, and the driving condition are not particularly limited. Whether an event occurs in the vehicle V can be determined using various known methods.

[0024] When the processing unit 103 extracts at least one of the recognition target and the driving condition, the processing unit 103 classifies the extraction results for each of a plurality of events that have occurred. The processing unit 103 may present the extraction results in a list via the display unit 102. The manner in which the list is presented is not particularly limited, and may be in various forms, or may be a list for each event that has occurred. The processing unit 103 may present the s extraction results with a priority via the display unit 102. The priority may be set, for example, according to the frequency of occurrence and the importance of the event that has occurred. The priority may be expressed as a numerical value or a degree.

[0025] Next, an example of a monitoring process for monitoring vehicle control of the vehicle V by the autonomous driving system server 100 of this embodiment will be described with reference to the flowchart in Fig. 2. The monitoring process here may be executed, for example, while the vehicle V is traveling, and when the process reaches an end, the process may be restarted from the start after a predetermined time.

[0026] As shown in Fig. 2, first, the processing unit 103 acquires vehicle control information from the vehicle V via the communication unit 101 (step S1). The processing unit 103 determines whether or not at least any of predetermined events has occurred based on the vehicle control information (step S2). If the answer is NO in the above step S2, the processing of the current cycle is terminated and the process proceeds to the above step S1 of the next cycle. On the other hand, if the answer is YES in the above step S2, the occurring event is assigned to the vehicle control information, and the vehicle control information is classified by event (step S3).

[0027] The processing unit 103 extracts the recognition object and the driving condition included in the occurring event based on the vehicle control information (step S4). In the above step S4, only one of the recognition object and the driving condition may be extracted. The processing unit 103 calculates a priority according to the occurrence frequency and importance of the occurring event, and assigns the priority to the extracted recognition object and driving condition (step S5). The processing unit 103 makes a list of the recognition object and the driving condition to which the priority has been assigned, divided by the occurring event, and displays the list on the display unit 102 (step S6).

[0028] In the server 100 for an autonomous driving system according to the present embodiment, the processing unit 103 may extract at least one of the recognition target and the driving condition when the reliability of the vehicle control is lower than a threshold value, assuming that a situation in which autonomous driving is difficult to deal with is occurring in the vehicle. Specifically, as will be described with reference to the flowchart of Fig. 3, first, the processing unit 103 acquires vehicle control information from the vehicle V via the communication unit 101 (step S11). The processing unit 103 determines whether the reliability of the vehicle control is lower than a threshold value based on the vehicle control information (step S12).

[0029] The reliability of the vehicle control can be obtained using a known method. For example, when another vehicle is present around the vehicle V, the reliability may be set to be lower. Also, for example, the reliability may be set to be higher as the safety of the situation of the vehicle V increases. Also, for example, when a second event occurs that occurs more frequently than a first event, the reliability may be set to be higher than when the first event occurs. The threshold value in step S12 is set and stored in advance. The threshold value is not particularly limited and may be a fixed value or a variable value.

[0030] Next, based on the vehicle control information, the recognition object and the driving condition are extracted (step S13). Reliability is assigned to the extracted recognition object and the driving condition. The processing unit 103 causes the display unit 102 to display a list of the recognition object and the driving condition with the reliability assigned (step S14).

[0031] As described above, the server 100 for the autonomous driving system can grasp recognition targets and / or driving conditions that are difficult to handle (such as those that the system is weak at) using vehicle control by the machine learning model M. In other words, it becomes possible to effectively monitor vehicle control by the machine learning model M. It becomes possible to improve the efficiency of monitoring the autonomous driving system 1 on the server 100 for the autonomous driving system side (for example, the vendor side).

[0032] In the autonomous driving system server 100, cases in which at least one of a plurality of events occurs are predefined as cases in which autonomous driving is estimated to be difficult to handle, and when at least one of the plurality of events occurs in the vehicle V, at least one of the recognition target and the driving condition is extracted. In this case, by utilizing the occurrence of the event, it becomes possible to grasp the recognition target and / or the driving condition that is difficult to handle by vehicle control using the machine learning model M.

[0033] In the autonomous driving system server 100, when at least one of a plurality of events occurs in the vehicle V, it is at least one of the following: when the driver of the vehicle V intervenes in autonomous driving; when the acceleration / deceleration or steering angular velocity of the vehicle V is equal to or greater than a threshold; when the distance to the other vehicle VT is smaller than a threshold; when the lateral position of the vehicle V is unstable; and when the vehicle V behaves in a manner that surprises the driver to a degree equal to or greater than a threshold. In this case, it is specifically possible to grasp the recognition target and / or driving conditions that are difficult to handle using vehicle control by the machine learning model M by utilizing the occurrence of the event.

[0034] In the autonomous driving system server 100, when the reliability of vehicle control is lower than a threshold, it may be determined that a situation in which the difficulty of autonomous driving is higher than a specified value occurs in the vehicle V, and at least one of the recognition target and the driving conditions may be extracted. This makes it possible to use the reliability of vehicle control to grasp recognition targets and / or driving conditions that are difficult to handle using vehicle control by the machine learning model M.

[0035] The server 100 for the autonomous driving system classifies the extraction results by multiple events. The server 100 for the autonomous driving system presents the extraction results in a list. The server 100 for the autonomous driving system presents the extraction results with a priority. In at least one of these cases, it becomes possible to easily understand the extraction results. It also becomes possible to understand the reasons or conditions under which the recognition target and / or driving condition was extracted.

[0036] Although the embodiment has been described above, one aspect of the present invention is not limited to the above embodiment. One aspect of the present invention can be embodied in various forms including the above embodiment and various modifications and improvements based on the knowledge of a person skilled in the art.

[0037] In the server 100 for the automatic driving system according to the above embodiment, as shown in FIG. 4, the processing unit 103 may estimate whether or not a situation in which automatic driving is estimated to be difficult to deal with occurs in the vehicle V based on information from the other vehicle VT. Specifically, the other vehicle VT transmits vehicle control information, which is information related to vehicle control of the vehicle V, to the server 100 for the automatic driving system via a communication unit 13 similar to the communication unit 4 (see FIG. 1). The vehicle control information includes at least any of the detection results of the sensor 12 similar to the sensor 2, the weather in the area where the vehicle V exists, the current time, road surface information on the road surface on which the vehicle V is traveling, the country in which the vehicle V is traveling, and the like. In the other vehicle VT, the vehicle control information can be appropriately acquired by a known method or device. The transmission of the vehicle control information to the server 100 for the automatic driving system may be performed periodically. Note that the other vehicle VT may be a plurality of vehicles.

[0038] In the monitoring process in this case, as shown in Fig. 5, for example, first, the processing unit 103 acquires vehicle control information from the other vehicle VT via the communication unit 101 (step S21). The processing unit 103 determines whether or not at least any of predetermined events has occurred based on the vehicle control information (step S22). If the result of step S22 is NO, the processing of the current cycle is terminated and the process proceeds to step S21 of the next cycle. If the result of step S22 is YES, the processing unit 103 assigns the occurring event to vehicle control information and classifies the vehicle control information by event (step S23).

[0039] The processing unit 103 extracts the recognition object and the driving condition included in the occurring event from the vehicle control information (step S24). The processing unit 103 calculates a priority according to the occurrence frequency and importance of the occurring event, and assigns the priority to the recognition object and the driving condition (step S25). The processing unit 103 makes a list of the recognition object and the driving condition to which the priority has been assigned, divided by the occurring event, and displays the list on the display unit 102 (step S26).

[0040] In this way, the server 100 for the autonomous driving system according to the modified example can estimate whether a situation that is estimated to be difficult for autonomous driving to handle has occurred in the vehicle V, based on information from the other vehicle VT. By using the information from the other vehicle VT, it becomes possible to grasp recognition targets and / or driving conditions that are difficult to handle using vehicle control by the machine learning model M. [Explanation of symbols]

[0041] 1...autonomous driving system, 2...sensor, 100...server for autonomous driving system, M...machine learning model, V...vehicle, VT...other vehicles.

Claims

1. A server capable of communicating with an autonomous driving system that performs autonomous driving of a vehicle by vehicle control using a machine learning model, Acquire vehicle control information which is information related to the vehicle control, A server for an autonomous driving system that extracts, when a situation occurs in the vehicle where it is estimated that it is difficult for the autonomous driving to respond, from the vehicle control information, at least one of a recognized object recognized in the vicinity of the vehicle during the vehicle control and the driving conditions of the vehicle during the vehicle control.

2. A case where at least one of a plurality of events occurs is determined in advance as a case where a situation in which it is estimated that it is difficult for the autonomous driving to respond occurs, 2. The server for an autonomous driving system according to claim 1, wherein when at least one of the plurality of events occurs in the vehicle, at least one of the recognition target and the driving condition is extracted.

3. 3. The server for an autonomous driving system according to claim 2, wherein when at least any of the plurality of events occurs in the vehicle, at least any of the following occurs: when the driver of the vehicle intervenes in autonomous driving; when the acceleration / deceleration or steering angular velocity of the vehicle is equal to or greater than a threshold; when the distance between the vehicle and another vehicle is smaller than a threshold; when the lateral position of the vehicle is unstable; and when the vehicle behaves in a manner that surprises the driver to a degree equal to or greater than a threshold.

4. 3. The server for an autonomous driving system according to claim 1 or 2, wherein, when the reliability of the vehicle control is lower than a threshold value, a situation in which it is estimated that the autonomous driving cannot respond is occurring in the vehicle, and at least one of the recognition target and the driving conditions is extracted.

5. 3. The server for an autonomous driving system according to claim 1 or 2, which estimates whether a situation in which it is estimated that the autonomous driving would be difficult to respond to is occurring in the vehicle based on information from other vehicles.

Images