Automatic drive system and control method

Abstract

To appropriately store the history of data related to an automatic drive control of a vehicle.SOLUTION: An automatic drive system mounted on a vehicle executes: processing for acquiring recognition data while recognizing a vehicle surrounding situation; processing for generating a vehicle route plan on the basis of the recognition data; processing for executing the automatic drive control for the vehicle according to the route plan; and processing for storing the data log related to the automatic drive control in the storage device. The data log includes the data log used for generation of the route plan. The processing for generating the route plan includes: a process for acquiring a residual capacity of the storage device; and a process for generating target data reduced in accuracy of the recognition data when the residual capacity is a prescribed capacity or lower, and generating the route plan which can be generated using the target data.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure relates to a technique for storing a history of data related to autonomous driving of a vehicle. [Background technology]

[0002] Patent Document 1 discloses a transportation system that includes multiple nodes and allows communication between the multiple nodes. The transportation system uses information transmitted between the nodes to calculate the collision risk between a target node and a target object, etc.

[0003] In addition to Patent Document 1, Patent Document 2 can also be mentioned as an example of a document that shows the technical level at the time of filing in the technical field of the present disclosure or a technical field related thereto. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2021-149162 A [Patent Document 2] JP 2020-021326 A Summary of the Invention [Problem to be solved by the invention]

[0005] There is known a technology for controlling the autonomous driving of a vehicle by using a machine learning model. As a method for retroactively verifying the autonomous driving control of a vehicle, it is possible to store a data log related to the autonomous driving control in an on-board storage device. However, the capacity of the on-board storage device is limited. Therefore, it is desirable to avoid a situation in which the capacity is insufficient and necessary data cannot be stored.

[0006] One objective of the present disclosure is to provide a technology that can prevent insufficient capacity of an on-board storage device and enable appropriate storage of data logs related to autonomous driving control. [Means for solving the problem]

[0007] A first aspect of the present disclosure relates to an autonomous driving system installed in a vehicle. The autonomous driving system includes a memory device and one or more processors. The one or more processors A process of recognizing the situation around the vehicle and acquiring recognition data; generating a route plan for the vehicle based on the perception data; A process of controlling automatic driving of a vehicle according to a route plan; A process of storing a data log related to the automatic driving control in a storage device; The apparatus is configured to execute the following steps: The data log contains a log of the data used to generate the route plan. The process of generating a route plan includes: A process of acquiring remaining capacity of a storage device; A process of generating target data with reduced accuracy of the recognition data when the remaining capacity is equal to or less than a predetermined amount, and generating a route plan that can be generated using the target data; Includes.

[0008] A second aspect of the present disclosure relates to a control method for controlling automatic driving of a vehicle. The control method is Recognizing a situation around the vehicle and acquiring recognition data; generating a path plan for the vehicle based on the recognition data; Performing automatic driving control of a vehicle according to a route plan; storing a data log related to the automatic driving control in a storage device in the vehicle; Includes. The data log contains a log of the data used to generate the route plan. Generating a path plan includes: Obtaining remaining capacity of a storage device; When the remaining capacity is equal to or less than a predetermined amount, generating target data with reduced accuracy of the recognition data, and generating a route plan that can be generated using the target data; Includes. Effect of the Invention

[0009] According to the technology disclosed herein, the remaining capacity of a storage device is acquired. Then, if the remaining capacity is equal to or less than a predetermined amount, the accuracy of the target data is reduced, and a route plan that can be generated using the reduced-accuracy target data is generated. By reducing the accuracy of the target data, the capacity of newly stored data can be reduced. In this way, the remaining capacity can be conserved, and a situation in which data that should be stored cannot be stored can be prevented.

[0010] In this way, according to the present disclosure, it is possible to prevent a situation in which the remaining capacity of the storage device is insufficient, and to properly store data logs related to autonomous driving control. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram showing an example of a configuration related to automatic driving control of a vehicle according to an embodiment. FIG. [Diagram 2] FIG. 1 is a conceptual diagram illustrating a configuration example of an autonomous driving system according to an embodiment. [Diagram 3] 11 is a conceptual diagram for explaining a specific example of a process for lowering the accuracy of target data. FIG. [Figure 4] 11 is a flowchart illustrating an example of a process according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] 1.Autonomous driving of vehicles 1 is a block diagram showing an example of a configuration related to the automatic driving control of a vehicle 1 according to the present embodiment. Automatic driving refers to automatically performing at least one of steering, acceleration, and deceleration of the vehicle 1 without the driver's operation. Automatic driving control is a concept that includes not only fully automatic driving control, but also risk avoidance control, lane keep assist control, and the like. The operator may be a driver who rides in the vehicle 1, or a remote operator who remotely operates the vehicle 1.

[0013] The vehicle 1 includes a sensor group 10, a recognition unit 20, a planner 30, a control amount calculation unit 40, and a traveling device 50.

[0014] The sensor group 10 includes a recognition sensor 11 used to recognize the situation around the vehicle 1. Examples of the recognition sensor 11 include a camera, a LIDAR (Laser Imaging Detection and Ranging), a radar, and the like. The sensor group 10 may further include a status sensor 12 that detects the status of the vehicle 1, a position sensor 13 that detects the position of the vehicle 1, and the like. Examples of the status sensor 12 include a speed sensor, an acceleration sensor, a yaw rate sensor, a steering angle sensor, and the like. Examples of the position sensor 13 include a GNSS (Global Navigation Satellite System) sensor.

[0015] The sensor detection information SEN is information obtained by the sensor group 10. The sensor detection information SEN includes recognition sensor information detected by the recognition sensor 11. For example, the sensor detection information SEN includes images captured by a camera and point cloud information obtained by a LIDAR. The sensor detection information SEN may also include vehicle state information indicating the state of the vehicle 1. The sensor detection information SEN may also include position information indicating the position of the vehicle 1.

[0016] The recognition unit 20 receives the sensor detection information SEN. The recognition unit 20 recognizes the situation around the vehicle 1 based on the information obtained by the recognition sensor 11. For example, the recognition unit 20 recognizes objects around the vehicle 1. Examples of objects include pedestrians, other vehicles (preceding vehicles, parked vehicles, etc.), white lines, road structures (e.g., guardrails, curbs), fallen objects, traffic lights, intersections, signs, etc. The recognition result information RES indicates the recognition result by the recognition unit 20. For example, the recognition result information RES includes object information indicating the relative position and relative speed of the object with respect to the vehicle 1.

[0017] The planner 30 receives the recognition result information RES from the recognition unit 20. The planner 30 may also receive vehicle state information, position information, and pre-generated map information. The map information may be high-precision three-dimensional map information. Based on the received information, the planner 30 generates a route plan PLN for the vehicle 1 to travel along a predetermined travel route. The travel route is input in advance, for example, by an operator of the vehicle 1. As another example, a destination of the vehicle 1 may be input by the operator, and the planner 30 may determine a travel route to the input destination.

[0018] The route plan PLN includes maintaining the current driving lane, changing lanes, overtaking, turning right or left, steering, accelerating, decelerating, stopping, etc. Furthermore, the route plan PLN made by the planning unit 30 may include a driving trajectory of the vehicle 1. The driving trajectory includes a target position and a target speed on the road on which the vehicle 1 is traveling.

[0019] The control amount calculation unit 40 receives the route plan PLN from the planner 30. The control amount calculation unit 40 calculates a control amount CON required for the vehicle 1 to travel according to the route plan PLN. When the route plan PLN includes a travel trajectory, it can also be said that the control amount CON is a control amount required to reduce the deviation between the vehicle 1 and the travel trajectory. The control amount CON includes at least one of a steering control amount, a drive control amount, and a braking control amount. Examples of the steering control amount include a target steering angle, a target torque, a target motor angle, a target motor drive current, etc. Examples of the drive control amount include a target speed, a target acceleration, etc. Examples of the braking control amount include a target speed, a target deceleration, etc.

[0020] The traveling device 50 includes a steering device 51, a driving device 52, and a braking device 53. The steering device 51 steers the wheels. For example, the steering device 51 includes an electric power steering (EPS) device. The driving device 52 is a power source that generates a driving force. Examples of the driving device 52 include an engine, an electric motor, and an in-wheel motor. The braking device 53 generates a braking force. The traveling device 50 receives a control amount CON from the control amount calculation unit 40. The traveling device 50 operates the steering device 51, the driving device 52, and the braking device 53 according to the steering control amount, the driving control amount, and the braking control amount, respectively. As a result, the vehicle 1 travels according to the route plan PLN.

[0021] The recognition unit 20 includes at least one of a rule-based model and a machine learning model. The rule-based model performs recognition processing based on a group of predetermined rules. Examples of the machine learning model include a Neural Network (NN), a Support Vector Machine (SVM), a regression model, a decision tree model, and the like. The NN may be a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), or a combination thereof. The type of each layer, the number of layers, and the number of nodes in the NN are arbitrary. The machine learning model is generated in advance through machine learning. The recognition unit 20 performs recognition processing by inputting the sensor detection information SEN to the model. The recognition result information RES is output from the model, or is generated based on the output from the model.

[0022] Similarly, the planner 30 includes at least one of a rule-based model and a machine learning model. The planner 30 performs planning processing by inputting recognition result information RES to the model. The path plan PLN is output from the model or is generated based on the output from the model.

[0023] Similarly, the control amount calculation unit 40 includes at least one of a rule-based model and a machine learning model. The control amount calculation unit 40 performs a control amount calculation process by inputting a path plan PLN to the model. The control amount CON is output from the model or is generated based on the output from the model.

[0024] Two or more of the recognition unit 20, the planner 30, and the control amount calculation unit 40 may be integrally configured. All of the recognition unit 20, the planner 30, and the control amount calculation unit 40 may be integrally configured (end-to-end configuration). For example, the recognition unit 20 and the planner 30 may be integrally configured by a NN that outputs a path plan PLN from the sensor detection information SEN. Even in the case of an integral configuration, intermediate products such as the recognition result information RES and the path plan PLN may be output. For example, when the recognition unit 20 and the planner 30 are integrally configured by a NN, the recognition result information RES may be the output of an intermediate layer of the NN.

[0025] The recognition unit 20, the planner 30, and the control amount calculation unit 40 constitute an “automatic driving control unit” that controls the automatic driving of the vehicle 1.

[0026] 2 is a block diagram showing an example of a hardware configuration of the autonomous driving system 100 according to this embodiment. The autonomous driving system 100 has at least the functions of the autonomous driving control unit described above.

[0027] The autonomous driving system 100 includes a processing unit 101, a sensor group 10, a traveling device 50, and an actuator group 60. The processing unit 101 is configured to be able to communicate with the sensor group 10, the traveling device 50, and the actuator group 60.

[0028] The processing unit 101 is a computer including one or more processors 110 (hereinafter simply referred to as processors 110) and one or more storage devices 120 (hereinafter simply referred to as storage devices 120).

[0029] The processor 110 executes various processes. Examples of the processor 110 include a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), and a field-programmable gate array (FPGA). The recognition unit 20, the planning unit 30, and the control amount calculation unit 40 may be realized by a single processor 110, or may be realized by separate processors 110. The storage device 120 stores various information. Examples of the storage device 120 include a hard disk drive (HDD), a solid state drive (SSD), a volatile memory, and a non-volatile memory.

[0030] The storage device 120 stores a computer program 130 and model data 140 .

[0031] The computer program 130 is executed by the processor 110. Various processes in the autonomous driving system 100 are realized by cooperation between the processor 110, which executes the computer program 130, and the storage device 120. The computer program 130 may be recorded in a computer-readable recording medium.

[0032] The model data 140 is data of models included in the recognition unit 20, the planning unit 30, and the control amount calculation unit 40. The model data 140 is stored in the storage device 120. The processor 110 configures the recognition unit 20, the planning unit 30, and the control amount calculation unit 40 by selecting and using a model from the model data 140 in executing the autonomous driving function.

[0033] The actuator group 60 includes actuators for various devices provided in the vehicle 1. For example, the actuator group 60 includes a vibration actuator for vibrating the steering wheel and a reaction torque actuator for generating a reaction torque on the steering wheel. The actuator group 60 is configured to be controllable by the processing unit 101.

[0034] 2. Processing by the processor The processes realized by the processor 110 executing the computer program 130 include a recognition data acquisition process, a route plan generation process, and a data log storage process.

[0035] The processor 110 acquires the recognition data by the recognition data acquisition process. The recognition data is data obtained by the processor 110 recognizing the surrounding situation of the vehicle 1. In other words, the recognition data here is data on at least one of the recognition sensor information and the recognition result information RES.

[0036] The processor 110 generates a route plan for the vehicle 1 based on the recognition data by a route plan generation process. The route plan generation process is executed in the planner 30.

[0037] The processor 110 uses a data log storage process to store a data log related to the autonomous driving control in the storage device 120. The stored data log includes a log of data used to generate a route plan.

[0038] The route plan generation process also includes a remaining capacity acquisition process, in which the processor 110 acquires the remaining capacity of the storage device 120.

[0039] If the remaining capacity acquisition process results in the remaining capacity being equal to or less than a predetermined amount, the processor 110 generates data with reduced accuracy of the recognition data in the route plan generation process. Hereinafter, the recognition data with reduced accuracy will be referred to as target data.

[0040] Furthermore, if the remaining capacity acquired by the remaining capacity acquisition process is equal to or less than a predetermined amount, the processor 110 generates a route plan that can be generated from the target data in the route plan generation process.

[0041] The effects of performing the above process will be described. The data log is stored in the data log storage process in preparation for cases where it becomes necessary to later verify how the automatic driving control using the machine learning model was performed. Therefore, it is desirable to store all the data logs of the recognition data used to generate the route plan without selecting any of them.

[0042] However, if the remaining capacity of the storage device 120 decreases and becomes insufficient for the required capacity, the processor 110 will not be able to store new data in the storage device 120. The remaining capacity acquisition process is performed to prevent such a situation.

[0043] That is, when the remaining capacity falls below a predetermined amount, suggesting the possibility of insufficient capacity, a route plan is generated using target data with reduced accuracy. The target data is data to be stored in the data log. That is, by reducing the capacity of the data to be stored in the data log, the capacity of the data log newly stored in the storage device 120 can be reduced. In this way, the remaining capacity of the storage device 120 can be conserved, and a situation in which the remaining capacity is insufficient and the recognition data used to generate the route plan cannot be stored can be prevented.

[0044] Furthermore, when the remaining capacity is low, a route plan that can be generated using the target data is generated. Specifically, a route plan that has a larger margin in the control of the vehicle 1 than usual is generated. By generating a route plan with a larger margin than usual, it is possible to ensure the safety of the vehicle 1's travel even when the target data is used.

[0045] The target data is generated when the remaining capacity of the storage device 120 falls below a predetermined amount. In other words, when the remaining capacity of the storage device 120 is sufficient and there is no need to conserve the remaining capacity, a route plan is generated using recognition data with normal accuracy. Therefore, it is no longer necessary to reduce the degree of freedom of the route plan of the vehicle 1 in order to secure a margin, and the satisfaction of the occupants of the vehicle 1 can be maintained.

[0046] 3. Processing to generate target data Figure 3 shows three specific examples of processes that generate target data, that is, processes that reduce the accuracy of recognition data.

[0047] In the first example, the resolution of the recognition data is reduced. Resolution means the density of information contained in the recognition data. For example, if the recognition data is an image captured by a camera, reducing the resolution means reducing the number of pixels or dots that make up the image. Alternatively, if the recognition data is point cloud information obtained by LIDAR, reducing the resolution means reducing the density of points that make up the point cloud.

[0048] In the second example, the sampling rate of the recognition data is reduced. For example, when the recognition data is information obtained by the recognition sensor 11, at least one of the detection frequency by the recognition sensor 11 and the acquisition frequency of information from the recognition sensor 11 by the recognition unit 20 is reduced. Alternatively, the frequency at which the planner 30 acquires the recognition result information RES from the recognition unit 20 may be reduced.

[0049] In the third example, the expression of the recognition data is changed. More specifically, the expression of the recognition data is changed to a more abstract expression. For example, the information about other vehicles and pedestrians indicated by the object information of the recognition result information RES is changed from detailed information including the shapes of other vehicles and pedestrians to information expressing other vehicles and pedestrians as bounding boxes. Alternatively, the object information of the recognition result information RES includes information about a curb that exists outside the lane in which the vehicle 1 is traveling. In this case, for example, the object information is changed from detailed information including the position and height of the curb to information indicating only the presence of a curb.

[0050] In other words, generating target data can be said to make the recognition data coarse. In this way, the accuracy of the recognition data is lowered and the data is made coarse, so the data volume can be reduced. For example, when the resolution of the recognition data is lowered, the amount of information per unit space is reduced, so the volume of data acquired when the same target is recognized is reduced. Or, when the sampling rate of the recognition data is lowered, the amount of information per unit time is reduced, so the volume of data acquired in the same time is reduced. Or, when the expression of the recognition data is changed to an expression with a high degree of abstraction, the amount of data acquired is also reduced. For example, information indicating only the existence of a curbstone has a smaller data volume than information including the detailed position of the curbstone. By reducing the volume of newly acquired data in this way, the remaining capacity of the storage device 120 can be preserved, and a situation in which the remaining capacity is insufficient can be prevented.

[0051] Furthermore, when generating target data, the processor 110 generates a route plan that can be generated using the target data. When using the target data, there is a possibility that errors in the recognition sensor information and the recognition result information RES may become large. Therefore, the processor 110 generates a route plan PLN for the vehicle 1 that allows the vehicle 1 to travel with a margin larger than normal so that the vehicle 1 can respond even if errors in the recognition sensor information and the recognition result information RES become large.

[0052] For example, when the processor 110 uses the target data, the processor 110 generates a route plan PLN in which the distance between the vehicle and the preceding vehicle is set larger than usual. Or, when it is necessary to decelerate the vehicle 1, the processor 110 generates a route plan PLN in which deceleration starts earlier than usual. Or, the processor 110 generates a route plan PLN in which the vehicle 1 changes lanes less frequently than usual. Or, when traveling on a curved road surface, even if the vehicle normally travels on either the left or right edge of the lane for efficient traveling, the processor 110 generates a route plan PLN in which the vehicle travels in the center of the lane when the target data is used.

[0053] 4. Processing example Fig. 4 is a flowchart showing an example of processing executed by the processor 110. The processing according to the flowchart shown in Fig. 4 is executed repeatedly at a predetermined processing cycle, for example, while the automatic driving control is being executed.

[0054] In step S110, the processor 110 obtains the remaining capacity of the storage device 120.

[0055] Next, in step S120, the processor 110 determines whether the remaining capacity of the storage device 120 acquired in step S110 is equal to or less than a predetermined amount.

[0056] If it is determined that the remaining capacity is equal to or less than the predetermined amount (step S120; Yes), the process proceeds to step S130. On the other hand, if it is determined that the remaining capacity is greater than the predetermined amount (step S120; No), the process proceeds to step S150.

[0057] In step S130, the processor 110 acquires recognition data. Furthermore, the processor 110 generates target data used for generating the path plan PLN by reducing the accuracy of the recognition data. More specifically, the processor 110 performs at least one of reducing the resolution, reducing the sampling rate, and changing the representation of the acquired recognition data.

[0058] Next, in step S140, the processor 110 generates a route plan PLN based on the target data generated in step S130. In step S140, the processor 110 generates a route plan PLN that can be generated using the target data. Specifically, a route plan PLN that provides a margin for control of the vehicle 1 is generated.

[0059] Meanwhile, in step S150, processor 110 acquires recognition data. The acquisition of recognition data is similar to step S130. However, in step S150, target data is not generated, that is, the accuracy of the recognition data is not reduced.

[0060] Next, in step S160, the processor 110 generates a route plan PLN based on the recognition data acquired in step S150. Here, a normal route plan PLN is generated. That is, the margin for controlling the vehicle 1 is not increased.

[0061] After the process of step S140 or step S160 is performed, the process proceeds to step S170. In step S170, the processor 110 stores a data log of the data used to generate the route plan in the storage device 120. After step S170, the process ends.

[0062] 5. Application Examples Since the data log stored by the data log storage process is used for verifying the automatic driving control using the machine learning model, the data that is particularly required to be stored in the data log is the data that is input to the part to which the machine learning model is applied. Therefore, it is more appropriate to use the recognition data whose accuracy is reduced as the data that is input to the part to which the machine learning model is applied. For example, when the machine learning model is applied only to the recognition unit 20, it is more appropriate to use the recognition data whose accuracy is reduced in the route plan generation process as the recognition sensor information that is input to the recognition unit 20.

[0063] Furthermore, when the remaining capacity of the storage device 120 is equal to or less than a predetermined amount, the recognition data for which the processor 110 reduces the accuracy may be limited to data related to the generation of a traveling trajectory on a road. The recognition data related to the generation of a traveling trajectory on a road is, for example, a recognition result related to an object around the vehicle 1. For example, the processor 110 recognizes an obstacle present in front of the vehicle 1, and generates a traveling trajectory for avoiding the obstacle based on the recognition result. When the remaining capacity of the storage device 120 is equal to or less than a predetermined amount, the accuracy of such recognition data related to the generation of a traveling trajectory may be reduced.

[0064] At least a part of the processor 110 may be a processor included in a server external to the vehicle 1. Then, a part or all of the processing illustrated in the flowchart of FIG. [Explanation of symbols]

[0065] 1...vehicle, 10...sensor group, 20...recognition unit, 30...planning unit, 40...control amount calculation unit, 50...traveling device, 60...actuator group, 100...automatic driving system, 110...processor, 120...storage device, CON...control amount, RES...recognition result information, SEN...sensor detection information, PLN...route planning

Claims

1. An automated driving system installed in a vehicle, A storage device and one or more processors, The one or more processors: A process of recognizing a situation around the vehicle and acquiring recognition data; generating a route plan for the vehicle based on the recognition data; A process of performing automatic driving control of the vehicle according to the route plan; A process of storing a data log related to the automatic driving control in the storage device; configured to run the data log includes a log of data used to generate the route plan; The process of generating a route plan includes: A process of acquiring a remaining capacity of the storage device; generating target data in which the accuracy of the recognition data is reduced when the remaining capacity is equal to or less than a predetermined amount, and generating a route plan that can be generated using the target data; Includes An automated driving system characterized by

2. The automated driving system according to claim 1, The process of generating the target data includes at least one of a process of reducing the resolution of the recognition data, a process of reducing the sampling rate of the recognition data, and a process of changing the expression of the recognition data. An automated driving system characterized by

3. The automated driving system according to claim 1 or 2, The generation of the route plan is the generation of a driving trajectory for the automatic driving control on the road on which the vehicle travels. An automated driving system characterized by

4. A control method for controlling automatic driving of a vehicle, comprising: Recognizing a situation around the vehicle and acquiring recognition data; generating a route plan for the vehicle based on the recognition data; performing automatic driving control of the vehicle according to the route plan; storing a data log related to the automatic driving control in a storage device; Including, the data log includes a log of data used to generate the route plan; Generating the path plan comprises: Obtaining a remaining capacity of the storage device; generating target data in which the accuracy of the recognition data is reduced when the remaining capacity is equal to or less than a predetermined amount, and generating a route plan that can be generated using the target data; Includes A control method comprising:

Images