Automated driving system and control method
The autonomous driving system addresses storage capacity limitations by generating reduced-accuracy data logs, ensuring data is stored and maintaining vehicle safety through margin-based route planning.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- TOYOTA JIDOSHA KK
- Filing Date
- 2023-07-10
- Publication Date
- 2026-06-23
AI Technical Summary
The limited capacity of in-vehicle storage devices poses a challenge for storing data logs related to automatic driving control, risking insufficient storage capacity and loss of necessary data.
An autonomous driving system that generates target data with reduced recognition accuracy when storage capacity is low, allowing for reduced data storage needs and ensuring sufficient capacity by generating path plans that can be created using this target data.
Prevents storage device capacity insufficiency by reducing data storage requirements, ensuring data logs are stored and maintaining vehicle safety through margin-based route planning even with reduced accuracy data.
Smart Images
Figure 0007878189000001 
Figure 0007878189000002 
Figure 0007878189000003
Abstract
Description
Technical Field
[0001] This disclosure relates to a technique for storing a history of data related to the automatic driving of a vehicle.
Background Art
[0002] Patent Document 1 discloses a traffic system including a plurality of nodes capable of communicating with each other. The traffic system calculates the risk of collision between a target node and a target object using information transmitted and received between the nodes.
[0003] In addition to Patent Document 1, Patent Document 2 can be exemplified as a document showing the technical level at the time of filing in the technical field of this disclosure or a technical field related thereto.
Prior Art Documents
Patent Documents
[0004]
Patent Document 1
Patent Document 2
Summary of the Invention
Problems to be Solved by the Invention
[0005] Techniques for performing automatic driving control of a vehicle using a machine learning model are known. As a method for retrospectively verifying the automatic driving control of a vehicle, it is conceivable to store a data log related to the automatic driving control in an in-vehicle storage device. However, the capacity of the in-vehicle storage device is limited. Therefore, it is desirable to avoid a situation where a shortage of capacity occurs and necessary data cannot be stored.
[0006] One object of this disclosure is to provide a technique that can prevent the capacity of an in-vehicle storage device from being insufficient and can appropriately store a data log related to automatic driving control. [Means for solving the problem]
[0007] The first aspect of this disclosure relates to an autonomous driving system installed in a vehicle. The autonomous driving system comprises a memory device and one or more processors. One or more processors, The process involves recognizing the surrounding environment of the vehicle and acquiring recognition data. A process that generates a vehicle route plan based on recognition data, The process involves controlling the vehicle's automatic driving according to the route plan, The process of storing data logs related to autonomous driving control in a storage device, It is configured to execute. The data log contains a log of the data used to generate the path plan. The process of generating a path plan is: The process of obtaining the remaining capacity of the storage device, When the remaining capacity is below a predetermined amount, the process generates target data with reduced recognition accuracy and generates a path plan that can be created using the target data. Includes.
[0008] The second aspect of this disclosure relates to a control method for controlling the autonomous driving of a vehicle. The control method is, The system recognizes the surrounding environment of the vehicle and acquires recognition data. To generate a vehicle route plan based on recognition data, To perform automatic driving control of the vehicle according to the route plan, The vehicle stores data logs related to autonomous driving control in its storage device, Includes. The data log contains a log of the data used to generate the path plan. Generating a path plan is Obtaining the remaining capacity of the storage device, When the remaining capacity is below a predetermined amount, the system generates target data with reduced recognition accuracy and generates a path plan that can be created using that target data. including
Advantages of the Invention
[0009] According to the technology of the present disclosure, the remaining capacity of the storage device is acquired. When the remaining capacity is less than or equal to a predetermined amount, the accuracy of the target data is reduced, and a path plan that can be generated with the target data with reduced accuracy is generated. By reducing the accuracy of the target data, the capacity of newly stored data can be reduced. Thus, the remaining capacity can be conserved, and a situation where data to be stored cannot be stored can be prevented.
[0010] Thus, according to the present disclosure, it is possible to prevent a situation where the remaining capacity of the storage device is insufficient and appropriately store a data log related to automatic driving control.
Brief Description of the Drawings
[0011] [Figure 1] It is a block diagram showing a configuration example related to automatic driving control of a vehicle according to an embodiment. [Figure 2] It is a conceptual diagram showing a configuration example of an automatic driving system according to an embodiment. [Figure 3] It is a conceptual diagram for explaining a specific example of a process for reducing the accuracy of target data. [Figure 4] It is a flowchart for explaining an example of a process according to an embodiment.
Embodiments of the Invention
[0012] 1. Automatic Driving of a Vehicle FIG. 1 is a block diagram showing a configuration example related to automatic driving control of a vehicle 1 according to the present embodiment. Automatic driving means automatically performing at least one of steering, acceleration, and deceleration of the vehicle 1 without depending on an operation by an operator. Automatic driving control is a concept that includes not only full automatic driving control but also risk avoidance control, lane keeping assist control, and the like. The operator may be a driver who rides in the vehicle 1 or a remote operator who remotely operates the vehicle 1.
[0013] Vehicle 1 includes a sensor group 10, a recognition unit 20, a planning unit 30, a control amount calculation unit 40, and a traveling device 50.
[0014] The sensor group 10 includes a recognition sensor 11 used to recognize the situation around vehicle 1. Examples of the recognition sensor 11 include a camera, LIDAR (Laser Imaging Detection and Ranging), a radar, etc. The sensor group 10 may further include a state sensor 12 that detects the state of vehicle 1, a position sensor 13 that detects the position of vehicle 1, etc. Examples of the state sensor 12 include a speed sensor, an acceleration sensor, a yaw rate sensor, a steering angle sensor, etc. An example of the position sensor 13 is a GNSS (Global Navigation Satellite System) sensor.
[0015] The sensor detection information SEN is information obtained by the sensor group 10. The sensor detection information SEN includes the recognition sensor information detected by the recognition sensor 11. For example, the sensor detection information SEN includes an image captured by a camera and point cloud information obtained by LIDAR. Also, the sensor detection information SEN may include vehicle state information indicating the state of vehicle 1. The sensor detection information SEN may include position information indicating the position of vehicle 1.
[0016] The recognition unit 20 receives the sensor detection information SEN. The recognition unit 20 recognizes the situation around vehicle 1 based on the information obtained by the recognition sensor 11. For example, the recognition unit 20 recognizes the objects around vehicle 1. Examples of the objects include pedestrians, other vehicles (preceding vehicles, parked vehicles, etc.), white lines, road structures (e.g., guardrails, curbs), falling objects, traffic lights, intersections, signs, etc. The recognition result information RES indicates the recognition result by the recognition unit 20. For example, the recognition result information RES includes object information indicating the relative position and relative speed of the object with respect to vehicle 1.
[0017] The planning unit (planner) 30 receives recognition result information RES from the recognition unit 20. The planning unit 30 may also receive vehicle status information, location information, and pre-generated map information. The map information may be high-precision 3D map information. Based on the received information, the planning unit 30 generates a route plan PLN for vehicle 1 to travel along a predetermined route. The route is pre-input by the operator of vehicle 1, for example. As another example, the destination of vehicle 1 may be input by the operator, and the planning unit 30 may determine the route to the input destination.
[0018] The route planning PLN includes actions such as maintaining the current lane, changing lanes, overtaking, turning left or right, steering, accelerating, decelerating, and stopping. Furthermore, the route planning PLN performed by the planning unit 30 may also include the vehicle 1's driving trajectory. The driving trajectory includes the target position and target speed within the road on which the vehicle 1 is traveling.
[0019] The control quantity calculation unit 40 receives the route plan PLN from the planning unit 30. The control quantity calculation unit 40 calculates the control quantity CON required for vehicle 1 to travel according to the route plan PLN. If the route plan PLN includes a travel trajectory, the control quantity CON can also be said to be the control quantity required to reduce the deviation between vehicle 1 and the travel trajectory. The control quantity CON includes at least one of the steering control quantity, drive control quantity, and braking control quantity. Examples of steering control quantities include target steering angle, target torque, target motor angle, target motor drive current, etc. Examples of drive control quantities include target speed, target acceleration, etc. Examples of braking control quantities include target speed, target deceleration, etc.
[0020] The running gear 50 includes a steering gear 51, a drive gear 52, and a braking gear 53. The steering gear 51 steers the wheels. For example, the steering gear 51 includes an electric power steering (EPS) system. The drive gear 52 is a power source that generates driving force. Examples of the drive gear 52 include an engine, an electric motor, an in-wheel motor, etc. The braking gear 53 generates braking force. The running gear 50 receives a control amount CON from the control amount calculation unit 40. The running gear 50 operates the steering gear 51, the drive gear 52, and the braking gear 53 according to the steering control amount, the drive control amount, and the braking control amount, respectively. As a result, the vehicle 1 travels according to the route plan PLN.
[0021] The recognition unit 20 includes at least one of a rule-based model and a machine learning model. The rule-based model performs recognition processing based on a predetermined set of rules. Examples of machine learning models include NN (Neural Network), SVM (Support Vector Machine), regression models, decision tree models, etc. The NN may be a CNN (Convolutional Neural Network), an RNN (Recurrent Neural Network), or a combination thereof. The type of each layer, the number of layers, and the number of nodes in the NN are arbitrary. The machine learning model is generated in advance through machine learning. The recognition unit 20 performs recognition processing by inputting sensor detection information SEN into the model. Recognition result information RES is output from the model or generated based on the output from the model.
[0022] Similarly, the planning unit 30 includes at least one of a rule-based model and a machine learning model. The planning unit 30 performs planning by inputting recognition result information RES into the model. The path plan PLN is output from the model or generated based on the output from the model.
[0023] Similarly, the control variable calculation unit 40 includes at least one of a rule-based model and a machine learning model. The control variable calculation unit 40 performs control variable calculation processing by inputting a path planning PLN to the model. The control variable CON is output from the model or generated based on the output from the model.
[0024] Two or more of the recognition unit 20, planning unit 30, and control variable calculation unit 40 may be configured as a single unit. The recognition unit 20, planning unit 30, and control variable calculation unit 40 may all be configured as a single unit (end-to-end configuration). For example, the recognition unit 20 and planning unit 30 may be configured as a single unit by a neural network (NN) that outputs a path plan PLN from sensor detection information SEN. Even in the case of a single unit configuration, intermediate products such as recognition result information RES and path plan PLN may be output. For example, if the recognition unit 20 and planning unit 30 are configured as a single unit by an NN, the recognition result information RES may be the output of the intermediate layer of the NN.
[0025] The recognition unit 20, the planning unit 30, and the control quantity calculation unit 40 constitute an "automatic driving control unit" that controls the automatic driving of vehicle 1.
[0026] Figure 2 is a block diagram showing an example of the hardware configuration of the automated driving system 100 according to this embodiment. The automated driving system 100 has at least the functions of the automated driving control unit described above.
[0027] The autonomous driving system 100 includes a processing unit 101, a sensor group 10, a driving device 50, and an actuator group 60. The processing unit 101 is configured to communicate with the sensor group 10, the driving device 50, and the actuator group 60.
[0028] The processing unit 101 is a computer that includes one or more processors 110 (hereinafter simply referred to as processor 110) and one or more storage devices 120 (hereinafter simply referred to as storage devices 120).
[0029] The processor 110 performs various processes. Examples of the processor 110 include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), etc. The recognition unit 20, the planning unit 30, and the control variable calculation unit 40 may be implemented by a single processor 110 or by separate processors 110. The storage device 120 stores various information. Examples of the storage device 120 include an HDD (Hard Disk Drive), an SSD (Solid State Drive), volatile memory, non-volatile memory, etc.
[0030] The storage device 120 stores the computer program 130 and model data 140.
[0031] The computer program 130 is executed by the processor 110. Various processes in the autonomous driving system 100 are realized through the cooperation of the processor 110, which executes the computer program 130, and the storage device 120. The computer program 130 may be recorded on a computer-readable recording medium.
[0032] Model data 140 is model data included in the recognition unit 20, the planning unit 30, and the control variable calculation unit 40. Model data 140 is stored in the storage device 120. In executing the autonomous driving function, the processor 110 configures the recognition unit 20, the planning unit 30, and the control variable calculation unit 40 by selecting and using a model from the model data 140.
[0033] The actuator group 60 includes actuators for various devices provided by the vehicle 1. For example, the actuator group 60 includes a vibration actuator for vibrating the steering wheel and a reaction torque actuator for generating a reaction torque on the steering wheel. The actuator group 60 is configured to be controllable by the processing unit 101.
[0034] 2. Processing by the processor The processing realized by the processor 110 executing the computer program 130 includes recognition data acquisition processing, path plan generation processing, and data log storage processing.
[0035] The processor 110 acquires recognition data through a recognition data acquisition process. Recognition data is data obtained by the processor 110 recognizing the surrounding conditions of the vehicle 1. In other words, the recognition data referred to here is data relating to at least one of the recognition sensor information and the recognition result information RES.
[0036] The processor 110 generates a route plan for vehicle 1 based on recognition data through a route plan generation process. The route plan generation process is performed in the planning unit 30.
[0037] The processor 110 stores data logs related to autonomous driving control in the storage device 120 through data log storage processing. The stored data logs include logs of data used to generate the route plan.
[0038] Furthermore, the route planning generation process includes a remaining capacity acquisition process. The processor 110 acquires the remaining capacity of the storage device 120 through the remaining capacity acquisition process.
[0039] If the remaining capacity obtained during the remaining capacity acquisition process is less than or equal to a predetermined amount, the processor 110 generates data with reduced accuracy in the path planning generation process. Hereinafter, the reduced-accuracy recognition data will be referred to as the target data.
[0040] Furthermore, if the remaining capacity obtained through the remaining capacity acquisition process is less than or equal to a predetermined amount, the processor 110 generates a route plan that can be generated using the target data in the route plan generation process.
[0041] The effects of the above processing will now be explained. The storage of data logs in the data log storage process is done in preparation for situations where it may be necessary to verify later how the automated driving control using the machine learning model was performed. Therefore, it is desirable that all data logs of the recognition data used to generate the path plan be stored without any selection or omission.
[0042] However, if the remaining capacity of the storage device 120 decreases and falls short of the required capacity, the processor 110 will be unable to store new data in the storage device 120. The remaining capacity acquisition process is performed to prevent such a situation.
[0043] In other words, if the remaining capacity falls below a predetermined amount and a potential capacity shortage is suggested, a route plan is generated using target data with reduced accuracy. The target data is the data that is to be stored in the data log. This means that by reducing the capacity of the data that is to be stored in the data log, the capacity of the data log to be newly stored in the storage device 120 can be reduced. In this way, the remaining capacity of the storage device 120 can be preserved, and a situation in which the recognition data used to generate the route plan cannot be stored due to insufficient remaining capacity can be prevented.
[0044] Furthermore, if the remaining capacity is low, a route plan that can be generated using the target data will be generated. Specifically, a route plan will be generated that has a larger margin in the control of vehicle 1 than under normal circumstances. By generating a route plan with a larger margin than under normal circumstances, the safety of vehicle 1's operation can be ensured even when using the target data.
[0045] The target data is generated only when the remaining capacity of the storage device 120 falls below a predetermined amount. In other words, when there is sufficient remaining capacity in the storage device 120 and there is no need to conserve capacity, the route plan is generated using recognition data with normal accuracy. Therefore, it is not necessary to reduce the degrees of freedom of the route plan for vehicle 1 in order to secure a margin, and the satisfaction of the occupants of vehicle 1 can be maintained.
[0046] 3. Process for generating target data Figure 3 shows three specific examples of processes that generate target data, that is, processes that reduce the accuracy of recognition data.
[0047] In the first example, the resolution of the recognition data is reduced. Resolution refers to the density of information contained in the recognition data. For example, if the recognition data is an image captured by a camera, reducing the resolution means reducing the number of pixels or dots that make up the image. Alternatively, if the recognition data is point cloud information obtained by LIDAR, reducing the resolution means reducing the density of points that make up the point cloud.
[0048] In the second example, the sampling rate of the recognition data is reduced. For example, if the recognition data is information obtained by the recognition sensor 11, at least one of the detection frequency by the recognition sensor 11 or the frequency of information acquisition from the recognition sensor 11 by the recognition unit 20 is reduced. Alternatively, the frequency at which the planning unit 30 acquires recognition result information RES from the recognition unit 20 may be reduced.
[0049] In the third example, the representation of the recognition data is changed. More specifically, the representation of the recognition data is changed to a more abstract representation. For example, the information about other vehicles and pedestrians indicated by the object information in the recognition result information RES is changed from detailed information including the shape of other vehicles and pedestrians to information that represents other vehicles and pedestrians with bounding boxes. Alternatively, suppose the object information in the recognition result information RES includes information about a curb located outside the lane in which vehicle 1 is traveling. In this case, for example, the object information is changed from detailed information including the position and height of the curb to information that only indicates the existence of a curb.
[0050] In other words, generating target data can be rephrased as making the recognition data coarser. By reducing the precision of the recognition data and making it coarser, the data size can be reduced. For example, if the resolution of the recognition data is reduced, the amount of information per unit space decreases, so the amount of data acquired when recognizing the same object decreases. Alternatively, if the sampling rate of the recognition data is reduced, the amount of information per unit time decreases, so the amount of data acquired in the same amount of time decreases. Alternatively, if the representation of the recognition data is changed to a more abstract representation, the amount of data acquired will also decrease. For example, information that only indicates the existence of a curb will take up less data space than information that includes the detailed location of the curb. By reducing the amount of newly acquired data in this way, the remaining capacity of the storage device 120 can be preserved, and a situation where the remaining capacity runs out can be prevented.
[0051] Furthermore, when the processor 110 generates target data, it generates a route plan that can be generated using the target data. When using target data, there is a possibility that the errors in the recognition sensor information and recognition result information RES will increase. Therefore, the processor 110 generates a route plan PLN for vehicle 1 that has a larger margin than usual to cope even if the errors in the recognition sensor information and recognition result information RES increase.
[0052] For example, when using the target data, the processor 110 generates a route plan PLN that sets a larger distance between the vehicle and the preceding vehicle than usual. Alternatively, when it is necessary to decelerate vehicle 1, the processor 110 generates a route plan PLN that starts deceleration earlier than usual. Alternatively, the processor 110 generates a route plan PLN that reduces the number of lane changes made by vehicle 1 compared to usual. Alternatively, when driving on a curved road, even if the vehicle normally drives close to either the left or right edge of the lane for efficient driving, when using the target data, the processor 110 generates a route plan PLN that drives in the middle of the lane.
[0053] 4. Processing Example Figure 4 is a flowchart showing an example of processing performed by the processor 110. The processing shown in the flowchart in Figure 4 is repeatedly executed at a predetermined processing cycle, for example, while automatic driving control is being performed.
[0054] In step S110, the processor 110 obtains the remaining capacity of the storage device 120.
[0055] Next, in step S120, the processor 110 determines whether the remaining capacity of the storage device 120 acquired in step S110 is less than or equal to a predetermined amount.
[0056] If it is determined that the remaining capacity is less than or equal to a predetermined amount (Step S120; Yes), the process proceeds to Step S130. On the other hand, if it is determined that the remaining capacity is greater than a predetermined amount (Step S120; No), the process proceeds to Step S150.
[0057] In step S130, the processor 110 acquires recognition data. Furthermore, the processor 110 generates target data used to generate the path planning PLN by reducing the precision of the recognition data. More specifically, the processor 110 performs at least one of the following on the acquired recognition data: reducing the resolution, reducing the sampling rate, and changing the representation.
[0058] Next, in step S140, the processor 110 generates a route plan PLN based on the target data generated in step S130. In step S140, the processor 110 generates a route plan PLN that can be generated using the target data. Specifically, a route plan PLN is generated that provides a margin for the control of vehicle 1.
[0059] On the other hand, in step S150, the processor 110 acquires recognition data. This is the same as in step S130 in that recognition data is acquired. However, in step S150, no target data is generated; in other words, the accuracy of the recognition data is not reduced.
[0060] Next, in step S160, the processor 110 generates a route plan PLN based on the recognition data acquired in step S150. Here, a normal route plan PLN is generated. In other words, no increase is made in the margin for controlling vehicle 1.
[0061] After the processing in step S140 or step S160 is completed, the process proceeds to step S170. In step S170, the processor 110 stores a data log of the data used to generate the path plan in the storage device 120. After step S170, the process ends.
[0062] 5. Application Examples The data logs stored through the data log storage process are used for verifying automated driving control using machine learning models, etc. Therefore, the storage of data logs is particularly important for data that serves as input to the parts to which the machine learning model is applied. For this reason, it is more appropriate to consider recognition data, even if its accuracy is reduced, as input data to the parts to which the machine learning model is applied. For example, if the machine learning model is applied only to the recognition unit 20, it is more appropriate to consider the recognition data, whose accuracy is reduced in the path planning generation process, as recognition sensor information that serves as input to the recognition unit 20.
[0063] Furthermore, the recognition data whose accuracy is reduced by the processor 110 when the remaining capacity of the storage device 120 is below a predetermined amount may be limited to data related to the generation of driving trajectories on a road. Recognition data related to the generation of driving trajectories on a road is, for example, the recognition results regarding objects around the vehicle 1. For example, the processor 110 recognizes an obstacle in front of the vehicle 1 and generates a driving trajectory to avoid the obstacle based on the recognition results. When the remaining capacity of the storage device 120 is below a predetermined amount, the accuracy of such recognition data related to the generation of driving trajectories may be reduced.
[0064] Furthermore, at least a portion of the processor 110 may be a processor included in a server outside of the vehicle 1. Some or all of the processes illustrated in the flowchart of Figure 4 may be performed on the server outside of the vehicle 1. [Explanation of symbols]
[0065] 1...Vehicle, 10...Sensor group, 20...Recognition unit, 30...Planning unit, 40...Control quantity calculation unit, 50...Traction unit, 60...Actuator group, 100...Automated driving system, 110...Processor, 120...Storage device, CON...Control quantity, RES...Recognition result information, SEN...Sensor detection information, PLN...Route plan
Claims
1. An autonomous driving system installed in a vehicle, A storage device and one or more processors are included. The one or more processors described above are: A process for recognizing the surrounding conditions of the vehicle and acquiring recognition data, A process for generating a route plan for the vehicle based on the aforementioned recognition data, A process for performing automatic driving control of the vehicle according to the aforementioned route plan, A process of storing data logs related to the automatic driving control in the storage device, It is configured to perform, The data log includes a log of the data used to generate the path plan, The process for generating the aforementioned path plan is: The process of obtaining the remaining capacity of the aforementioned storage device, When the remaining capacity is less than or equal to a predetermined amount, the process involves generating target data with reduced accuracy of the recognition data, and generating a path plan that can be generated using the target data. including An autonomous driving system characterized by the following features.
2. An automated driving system according to claim 1, The process for generating the target data includes at least one of the following: a process for reducing the resolution of the recognition data, a process for reducing the sampling rate of the recognition data, and a process for changing the representation of the recognition data. An autonomous driving system characterized by the following features.
3. An automated driving system according to claim 1 or 2, The generation of the aforementioned route plan is the generation of the driving trajectory for the automated driving control within the road on which the vehicle travels. An autonomous driving system characterized by the following features.
4. A control method for controlling the automatic driving of a vehicle, The vehicle's surroundings are recognized and recognition data is acquired. To generate a route plan for the vehicle based on the aforementioned recognition data, To perform automatic driving control of the vehicle in accordance with the aforementioned route plan, The data log related to the aforementioned automatic driving control is stored in a storage device, Includes, The data log includes a log of the data used to generate the path plan, Generating the aforementioned path plan means To obtain the remaining capacity of the aforementioned storage device, When the remaining capacity is less than or equal to a predetermined amount, the system generates target data with reduced accuracy of the recognition data, and generates a path plan that can be generated using the target data. including A control method characterized by the following: