Autonomous Driving System

Abstract

To appropriately store log data related to automated driving control.SOLUTION: An automated driving system mounted on a vehicle executes at least part of automated driving control of the vehicle by using a machine learning model. During execution of the automated driving control, the automated driving system acquires target data including log data related to the automated driving control using the machine learning model. The automated driving system stores a target storage amount of the target data in one or more storage devices per unit time or unit distance. An upload position is a position where at least part of the target data stored in the one or more storage devices is uploaded to an external device. The target storage amount includes a first storage amount that increases as a remaining distance or a remaining time to the upload position decreases.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to an autonomous driving system installed in a vehicle. [Background technology]

[0002] There is known a technique for controlling automatic driving of a vehicle using a machine learning model. Patent Document 1 discloses a method for collecting training data that can be used for learning a machine learning model. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2019 / 116423 Summary of the Invention [Problem to be solved by the invention]

[0004] One method for retrospectively verifying the autonomous driving control of a vehicle is to store log data related to the autonomous driving control in an on-board storage device. However, since the capacity of on-board storage devices is limited, it is desirable to avoid situations where the minimum necessary log data cannot be stored.

[0005] One object of the present disclosure is to provide a technology that can appropriately store log data related to autonomous driving control. [Means for solving the problem]

[0006] One aspect of the present disclosure relates to an automated driving system installed in a vehicle. The autonomous driving system includes one or more processors and one or more storage devices. The one or more processors utilize machine learning models to perform at least a portion of the autonomous driving control of the vehicle. During execution of the autonomous driving control, one or more processors acquire target data including log data related to the autonomous driving control using the machine learning model. The one or more processors store target data of a target storage amount per unit time or unit distance in one or more storage devices. The upload location is a location where at least a portion of the target data stored in one or more storage devices is uploaded to an external device. The target storage amount includes a first storage amount that increases as the remaining distance or remaining time to the upload location decreases. [Effects of the Invention]

[0007] The target storage amount per unit time or unit distance includes the first storage amount. The first storage amount increases as the remaining distance or remaining time to the upload location decreases. When the remaining distance or remaining time is long, the first storage amount is set to a small (conservative) amount. This prevents the free space in the storage device from being depleted at an early stage. In other words, it prevents the necessary log data from being saved in the storage device.

[0008] On the other hand, as the remaining distance or remaining time to the upload location decreases, the first storage amount increases, and the target data stored in the storage device becomes richer. This allows more useful data to be stored, which means the storage device's capacity can be used more effectively.

[0009] In this way, according to the present disclosure, it is possible to effectively utilize the capacity of the storage device while appropriately storing log data related to autonomous driving control. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing an example of a configuration related to automatic driving control of a vehicle according to an embodiment; [Figure 2] FIG. 1 is a conceptual diagram illustrating a configuration example of an autonomous driving system according to an embodiment. [Figure 3] FIG. 10 is a conceptual diagram for explaining an upload location according to an embodiment. [Figure 4] FIG. 4 is a conceptual diagram for explaining a first example of a log data saving process according to the embodiment. [Figure 5] FIG. 10 is a conceptual diagram for explaining a second example of the log data saving process according to the embodiment. [Figure 6] FIG. 10 is a conceptual diagram for explaining a third example of the log data saving process according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] 1. Autonomous driving of vehicles FIG. 1 is a block diagram showing an example of a configuration related to automatic driving control of a vehicle 1 according to this embodiment. Automatic driving refers to automatically performing at least one of steering, acceleration, and deceleration of the vehicle 1 without the need for driving operations by an operator. Automatic driving control is a concept that includes not only fully automatic driving control, but also risk avoidance control, lane keep assist control, and the like. The operator may be a driver on board the vehicle 1, or a remote operator that remotely controls the vehicle 1.

[0012] The vehicle 1 includes a sensor group 10, a recognition unit 20, a planner 30, a control amount calculation unit 40, and a traveling device 50.

[0013] The sensor group 10 includes a recognition sensor 11 used to recognize the situation around the vehicle 1. Examples of the recognition sensor 11 include a camera, a LIDAR (Laser Imaging Detection and Ranging), a radar, etc. The sensor group 10 may further include a status sensor 12 that detects the status of the vehicle 1, a position sensor 13 that detects the position of the vehicle 1, etc. Examples of the status sensor 12 include a speed sensor, an acceleration sensor, a yaw rate sensor, a steering angle sensor, etc. Examples of the position sensor 13 include a GNSS (Global Navigation Satellite System) sensor.

[0014] The sensor detection information SEN is information obtained by the sensor group 10. For example, the sensor detection information SEN includes images captured by a camera. As another example, the sensor detection information SEN may include point cloud information obtained by a LIDAR. The sensor detection information SEN may include vehicle state information indicating the state of the vehicle 1. The sensor detection information SEN may include location information indicating the location of the vehicle 1.

[0015] The recognition unit 20 receives the sensor detection information SEN. The recognition unit 20 recognizes the situation around the vehicle 1 based on the information obtained by the recognition sensor 11. For example, the recognition unit 20 recognizes objects around the vehicle 1. Examples of objects include pedestrians, other vehicles (leading vehicles, parked vehicles, etc.), white lines, road structures (e.g., guardrails, curbs), fallen objects, traffic lights, intersections, signs, etc. The recognition result information RES indicates the recognition result by the recognition unit 20. For example, the recognition result information RES includes object information indicating the relative position and relative speed of the object with respect to the vehicle 1.

[0016] The planner 30 receives the recognition result information RES from the recognition unit 20. The planner 30 may also receive vehicle state information, position information, and pre-generated map information. The map information may be high-precision three-dimensional map information. The planner 30 generates a driving plan for the vehicle 1 based on the received information. The driving plan may be for reaching a predetermined destination or for avoiding risks. Examples of the driving plan include maintaining the current driving lane, changing lanes, overtaking, turning right or left, steering, accelerating, decelerating, and stopping. Furthermore, the planner 30 generates a target trajectory TRJ required for the vehicle 1 to travel according to the driving plan. The target trajectory TRJ includes a target position and a target speed.

[0017] The control amount calculation unit 40 receives the target trajectory TRJ from the planning unit 30. The control amount calculation unit 40 calculates a control amount CON required for the vehicle 1 to follow the target trajectory TRJ. The control amount CON can also be said to be a control amount required to reduce the deviation between the vehicle 1 and the target trajectory TRJ. The control amount CON includes at least one of a steering control amount, a drive control amount, and a braking control amount. Examples of the steering control amount include a target steering angle, a target torque, a target motor angle, a target motor drive current, etc. Examples of the drive control amount include a target speed, a target acceleration, etc. Examples of the braking control amount include a target speed, a target deceleration, etc.

[0018] The traveling device 50 includes a steering device 51, a drive device 52, and a braking device 53. The steering device 51 steers the wheels. For example, the steering device 51 includes an electric power steering (EPS) device. The drive device 52 is a power source that generates a driving force. Examples of the drive device 52 include an engine, an electric motor, and an in-wheel motor. The braking device 53 generates a braking force. The traveling device 50 receives a control amount CON from the control amount calculation unit 40. The traveling device 50 operates the steering device 51, the drive device 52, and the braking device 53 in accordance with the steering control amount, the drive control amount, and the braking control amount, respectively. This causes the vehicle 1 to travel so as to follow the target trajectory TRJ.

[0019] The recognition unit 20 includes at least one of a rule-based model and a machine learning model. The rule-based model performs recognition processing based on a set of predetermined rules. Examples of machine learning models include a neural network (NN), a support vector machine (SVM), a regression model, and a decision tree model. The NN may be a convolutional neural network (CNN), a recurrent neural network (RNN), or a combination thereof. The type of each layer, the number of layers, and the number of nodes in the NN are arbitrary. The machine learning model is generated in advance through machine learning. The recognition unit 20 performs recognition processing by inputting sensor detection information SEN to the model. The recognition result information RES is output from the model or generated based on the output from the model.

[0020] Similarly, the planning unit 30 includes at least one of a rule-based model and a machine learning model. The planning unit 30 performs planning processing by inputting recognition result information RES to the model. The target trajectory TRJ is output from the model or generated based on the output from the model.

[0021] Similarly, the control amount calculation unit 40 includes at least one of a rule-based model and a machine learning model. The control amount calculation unit 40 performs a control amount calculation process by inputting the target trajectory TRJ to the model. The control amount CON is output from the model or generated based on the output from the model.

[0022] Two or more of the recognition unit 20, the planning unit 30, and the control amount calculation unit 40 may be configured integrally. All of the recognition unit 20, the planning unit 30, and the control amount calculation unit 40 may be configured integrally (end-to-end configuration). For example, the recognition unit 20 and the planning unit 30 may be configured integrally by a neural network (NN) that outputs a target trajectory TRJ from the sensor detection information SEN. Even in the case of an integrated configuration, intermediate products such as recognition result information RES and a target trajectory TRJ may be output. For example, when the recognition unit 20 and the planning unit 30 are configured integrally by a neural network (NN), the recognition result information RES may be the output of an intermediate layer of the NN.

[0023] The recognition unit 20, the planning unit 30, and the control amount calculation unit 40 constitute an "automatic driving control unit" that controls the automatic driving of the vehicle 1. In this embodiment, a machine learning model is used in at least a part of the automatic driving control unit. That is, at least one of the recognition unit 20, the planning unit 30, and the control amount calculation unit 40 includes a machine learning model. The automatic driving control unit performs at least a part of the automatic driving control of the vehicle 1 using the machine learning model.

[0024] 2 is a conceptual diagram showing an example of the configuration of an autonomous driving system 100 according to this embodiment. The autonomous driving system 100 is mounted on a vehicle 1 and performs autonomous driving control of the vehicle 1. The autonomous driving system 100 has at least the functions of the autonomous driving control unit described above. The autonomous driving system 100 may further include a sensor group 10 and a driving device 50.

[0025] The autonomous driving system 100 includes one or more processors 110 (hereinafter simply referred to as processors 110) and one or more storage devices 120 (hereinafter simply referred to as storage devices 120). The processors 110 execute various processes. Examples of the processors 110 include a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), and a field-programmable gate array (FPGA). The recognition unit 20, the planning unit 30, and the control amount calculation unit 40 may be implemented by a single processor 110 or by separate processors 110. The storage device 120 stores various information. Examples of the storage device 120 include a hard disk drive (HDD), a solid state drive (SSD), a volatile memory, and a non-volatile memory.

[0026] The program 130 is a computer program for controlling the vehicle 1, and is executed by the processor 110. Various processes by the autonomous driving system 100 may be realized by cooperation between the processor 110 that executes the program 130 and the storage device 120. The program 130 is stored in the storage device 120. The program 130 may be recorded on a computer-readable recording medium.

[0027] The model data 140 is data of a model included in the recognition unit 20, the planning unit 30, and the control amount calculation unit 40. As described above, in this embodiment, at least one of the recognition unit 20, the planning unit 30, and the control amount calculation unit 40 includes a "machine learning model." The model data 140 is stored in the storage device 120 and is used for autonomous driving control.

[0028] During autonomous driving control, the processor 110 acquires "log data LOG" related to the autonomous driving control using a machine learning model. The log data LOG is used for verifying the autonomous driving control using a machine learning model, etc. The log data LOG may include sensor detection information SEN input to the autonomous driving control unit. The log data LOG may include control amount CON output from the autonomous driving control unit. The log data LOG may include recognition result information RES output from the recognition unit 20. The log data LOG may include a target trajectory TRJ output from the planner 30. The log data LOG may include reasons for decisions made in the recognition processing by the recognition unit 20. The log data LOG may include reasons for decisions made in the planning processing by the planner 30. The log data LOG may include whether or not an operator intervened in the autonomous driving control.

[0029] The processor 110 stores the log data LOG acquired during the automatic driving control in the storage device 120. The processor 110 may temporarily save the log data LOG in the storage device 120 for only a certain period of time.

[0030] The management server 200 is an external device located outside the vehicle 1. The management server 200 communicates with one or more vehicles 1 via a communication network. During or after the autonomous driving control is completed, the processor 110 of the vehicle 1 may upload at least a portion of the log data LOG stored in the local storage device 120 to the management server 200. The processor 110 may also delete the log data LOG uploaded to the management server 200 from the local storage device 120.

[0031] The management server 200 has a database 220. The management server 200 acquires log data LOG uploaded from one or more vehicles 1. Then, the management server 200 saves (stores) the acquired log data LOG in the database 220. The management server 200 saves (stores) the log data LOG for at least a predetermined period of time. The log data LOG is used for verifying autonomous driving control using a machine learning model, etc.

[0032] 2. Efficient log data storage In the following description, the upload target data DUP refers to data to be uploaded from the vehicle 1 to the management server 200. The upload target data DUP includes at least log data LOG related to autonomous driving control using a machine learning model. The upload target data DUP may also include data that is not directly related to autonomous driving control.

[0033] FIG. 3 is a conceptual diagram for explaining the upload position PU. The upload position PU is a position where at least a portion of the upload target data DUP stored in the storage device 120 of the vehicle 1 is uploaded to the management server 200. Typically, the upload position PU is determined in advance. For example, the upload position PU is the destination of the vehicle 1. As another example, the upload position PU may be an intermediate point located between the departure point and the destination. The upload position PU may also be the base of the vehicle 1. For example, if the vehicle 1 is a mobility service vehicle such as a bus, the upload position PU may be the base of the mobility service vehicle (e.g., a bus center). The upload position PU is registered in advance in map information.

[0034] Typically, the vehicle 1 stops at the upload position PU. At the upload position PU, the processor 110 may temporarily suspend the autonomous driving control. At the upload position PU, the processor 110 communicates with the management server 200 and uploads at least a portion of the upload target data DUP to the management server 200. The processor 110 may upload all of the upload target data DUP to the management server 200. The processor 110 may erase the log data LOG uploaded to the management server 200 from the local storage device 120.

[0035] The log data LOG included in the upload target data DUP is used for verifying autonomous driving control using a machine learning model, etc. It is desirable that the minimum amount of log data LOG required for verification, etc. be stored in the local storage device 120 of the vehicle 1 and then uploaded to the management server 200. However, the capacity of the storage device 120 of the vehicle 1 is limited. Taking this into consideration, it is desirable to avoid a situation in which the minimum amount of log data LOG cannot be stored. On the other hand, it is also desirable to make effective use of the capacity of the storage device 120 of the vehicle 1.

[0036] From the above viewpoint, the present embodiment proposes a technique that allows the log data LOG to be appropriately stored in the storage device 120 of the vehicle 1.

[0037] 2-1. First example 4 is a conceptual diagram illustrating a first example of the log data storage process. A vehicle 1 moves from a departure point PS to an upload position PU. The horizontal axis of FIG. 4 represents a position between the departure point PS and the upload position PU, or a time corresponding to that position.

[0038] The amount of upload target data DUP stored in storage device 120 per unit time or per unit distance is hereinafter referred to as the "target storage amount Q." Processor 110 stores the upload target data DUP of the target storage amount Q in storage device 120 per unit time or per unit distance. In a first example, the target storage amount Q includes a "basic storage amount Q0" and a "first storage amount Q1."

[0039] The basic storage amount Q0 is the minimum amount of predetermined log data LOG required for verification, etc. The type of minimum required predetermined log data LOG is specified in advance, for example, by an administrator of the autonomous driving system 100 or the management server 200. For example, the minimum required log data LOG includes sensor detection information SEN and control amount CON. Note that the basic storage amount Q0 may vary depending on the location, time, surrounding environment, scene, etc.

[0040] On the other hand, the first storage amount Q1 is the amount of "additional data" other than the minimum required predetermined log data LOG. For example, the additional data includes additional log data LOG other than the minimum required predetermined log data LOG. As another example, the additional data may include learning data used to train a machine learning model. The learning data is data related to autonomous driving control using a machine learning model in the vehicle 1. The learning data may be a combination of input data input to the machine learning model and output data output from the machine learning model. Note that the learning data may be used to train (update) a machine learning model used for autonomous driving control of the vehicle 1, or may be used to train another machine learning model. As yet another example, the additional data may include data not directly related to autonomous driving control.

[0041] The data to be uploaded DUP stored in the storage device 120 per unit time or per unit distance includes "predetermined log data LOG of the basic storage amount Q0" and "additional data of the first storage amount Q1." By storing not only the predetermined log data LOG but also the additional data in the storage device 120, the capacity of the storage device 120 can be used effectively.

[0042] However, the basic storage capacity Q0 may vary depending on the location, time, surrounding environment, scene, etc. Predicting the basic storage capacity Q0 is not necessarily easy. If the capacity of the storage device 120 is consumed haphazardly from the beginning, there is a risk that the free space of the storage device 120 will be depleted later. If the free space of the storage device 120 is depleted, there is a risk that even the minimum required predetermined log data LOG will no longer be stored in the storage device 120.

[0043] Therefore, the first storage amount Q1 is set taking into consideration the remaining distance Dr or remaining time Tr to the upload position PU. In other words, the first storage amount Q1 is expressed as a function (f1) of the remaining distance Dr or remaining time Tr to the upload position PU. The remaining distance Dr can be calculated based on the current position of the vehicle 1 and the upload position PU. The current position of the vehicle 1 is obtained by the position sensor 13. The upload position PU is registered in advance in map information. The remaining time Tr can be estimated from the remaining distance Dr and the speed of the vehicle 1. The speed of the vehicle 1 is detected by the status sensor 12. Alternatively, if the vehicle 1 is a mobility service vehicle such as a bus, the remaining time Tr to the upload position PU can be obtained from the operation schedule of the mobility service vehicle.

[0044] More specifically, as illustrated in FIG. 4, the first storage quantity Q1 increases as the remaining distance Dr or remaining time Tr to the upload position PU decreases. This increase may be monotonically or stepwise. For example, the travel route of the vehicle 1 from the departure point PS to the upload position PU includes a first section and a subsequent second section. In this case, the first storage quantity Q1 in the second section is set to be greater than the first storage quantity Q1 in the first section. When the mobility service vehicle departs from a certain base (e.g., a bus center) and returns to the same base, the first section may be the outbound section, and the second section may be the return section. The first storage quantity Q1 in the first section may be zero.

[0045] The additional data may be additional log data LOG other than the minimum required predetermined log data LOG. In this case, for example, the types of log data LOG stored in the storage device 120 may increase as the remaining distance Dr or remaining time Tr to the upload position PU decreases. As another example, the sampling rate of the log data LOG may increase as the remaining distance Dr or remaining time Tr to the upload position PU decreases.

[0046] The profile of the first conserved quantity Q1 as illustrated in FIG. 4 may be set in advance by an administrator of the autonomous driving system 100 or the management server 200.

[0047] As described above, in the first example, the target storage amount Q per unit time or unit distance includes the first storage amount Q1. The first storage amount Q1 increases as the remaining distance Dr or remaining time Tr to the upload position PU decreases. When the remaining distance Dr or remaining time Tr is long, the first storage amount Q1 is set to a small (modest) value. This prevents the free space in the storage device 120 from being depleted at an early stage. In other words, it prevents the minimum required amount of predetermined log data LOG from being stored in the storage device 120.

[0048] On the other hand, as the remaining distance Dr or remaining time Tr to the upload position PU decreases, the first storage amount Q1 increases, and the additional data stored in the storage device 120 becomes richer. This makes it possible to store more useful data. In other words, the capacity of the storage device 120 can be used more effectively.

[0049] As described above, according to the first example, it is possible to effectively utilize the capacity of the storage device 120 and appropriately store the log data LOG related to the automatic driving control.

[0050] 2-2. Second example 5 is a conceptual diagram for explaining a second example of the log data storage process. Explanations that overlap with the first example will be omitted as appropriate. In the second example, the target storage amount Q includes a "basic storage amount Q0" and a "second storage amount Q2."

[0051] Like the first storage amount Q1, the second storage amount Q2 is the amount of "additional data" other than the minimum required log data LOG. This second storage amount Q2 is set taking into account the remaining capacity Cr of the storage device 120. In other words, the second storage amount Q2 is expressed as a function (f2) of the remaining capacity Cr of the storage device 120. More specifically, the second storage amount Q2 decreases as the remaining capacity Cr of the storage device 120 decreases. This decrease may be monotonic or gradual. As shown in FIG. 5, the second storage amount Q2 may become zero when the remaining capacity Cr is sufficient. For example, when the remaining capacity Cr falls below a predetermined threshold, the second storage amount Q2 is set to zero. The profile of the second storage amount Q2 may be preset by an administrator.

[0052] When the remaining capacity Cr of the storage device 120 is relatively large, the second storage capacity Q2 is set to a large value, and the additional data stored in the storage device 120 becomes rich. This makes it possible to store more useful data. In other words, the capacity of the storage device 120 can be used effectively.

[0053] On the other hand, when the remaining capacity Cr of the storage device 120 becomes relatively small, the second storage amount Q2 becomes smaller. This prevents the free space of the storage device 120 from being depleted. In other words, it prevents the minimum required amount of predetermined log data LOG from being stored in the storage device 120.

[0054] In this way, according to the second example, it is possible to effectively utilize the capacity of the storage device 120 and appropriately store the log data LOG related to the automatic driving control.

[0055] 2-3. Third Example 6 is a conceptual diagram for explaining a third example of the log data storage process. Explanations that overlap with the first example will be omitted as appropriate. In the third example, the target storage amount Q includes a "basic storage amount Q0" and a "third storage amount Q3."

[0056] Like the first storage amount Q1, the third storage amount Q3 is also the amount of "additional data" other than the minimum required predetermined log data LOG. This third storage amount Q3 is secured at a pinpoint location designated by the administrator. For example, the administrator may designate a location where an event is likely to occur (e.g., an intersection). The third storage amount Q3 may be a predetermined amount or may be designated by the administrator. The profile of the third storage amount Q3 may be preset by the administrator.

[0057] According to the third example, it is also possible to effectively utilize the capacity of the storage device 120 and appropriately store the log data LOG related to the automatic driving control.

[0058] 2-4. Fourth Example Combinations of two or more of the above-described first to third examples are also possible. For example, the third example may be combined with the first or second example. For example, the third conserved quantity Q3 may be set precisely at a position where the first conserved quantity Q1 or the second conserved quantity Q2 is weak. Combinations of the first, second, and third examples are also possible. [Explanation of symbols]

[0059] 1...vehicle, 10...sensor group, 20...recognition unit, 30...planning unit, 40...control amount calculation unit, 50...traveling device, 100...autonomous driving system, 200...management server, CON...control amount, LOG...log data, RES...recognition result information, SEN...sensor detection information, TRJ...target trajectory

Claims

1. An automated driving system installed in a vehicle, one or more processors; one or more storage devices; Equipped with the one or more processors: utilizing a machine learning model to perform at least a portion of the autonomous driving control of the vehicle; During execution of the autonomous driving control, target data including log data related to the autonomous driving control using the machine learning model is acquired; The target data of the target storage amount per unit time or per unit distance is stored in the one or more storage devices. It is configured as follows: the upload location is a location where at least a portion of the target data stored in the one or more storage devices is uploaded to an external device; The target storage amount includes a first storage amount that increases as the remaining distance or remaining time to the upload location decreases. Autonomous driving system.

2. The automated driving system according to claim 1, the target data includes predetermined log data that is required at a minimum and additional data other than the predetermined log data, The one or more processors store the first stored amount of the additional data in the one or more storage devices. Autonomous driving system.

3. The automated driving system according to claim 2, The additional data is data related to the autonomous driving control using the machine learning model, and includes learning data used for learning the machine learning model or another machine learning model. Autonomous driving system.

4. The automated driving system according to any one of claims 1 to 3, The target storage capacity further includes a second storage capacity that decreases as the remaining capacity of the one or more storage devices decreases. Autonomous driving system.

5. The automated driving system according to any one of claims 1 to 3, The target storage amount further includes a third storage amount secured at a location designated by the administrator. Autonomous driving system.

Images