Testing autonomous driving control device
By using a neural network-based autonomous driving testing method, a simulated environment dataset is generated and the testing is automated, solving the problem of long testing time in existing technologies and achieving efficient and safe autonomous driving testing.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- FORD GLOBAL TECH LLC
- Filing Date
- 2025-10-13
- Publication Date
- 2026-04-17
AI Technical Summary
Existing testing methods for autonomous vehicles are time-consuming in both real-world and virtual scenarios, and traditional renderers generate environmental data that is not close to reality, resulting in low testing efficiency.
A testing method based on first and second neural networks is adopted. By generating a simulated environment dataset and automating the test, the second neural network autonomously generates new driving scenarios, reducing implementation complexity and improving data security.
It enables a more efficient testing process, generates environmental data that is closer to real-world scenarios, reduces testing complexity, and improves data security, all without the need to collect data in a real environment.
Smart Images
Figure CN121879211A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to a testing method for an autonomous driving control device, and also to a testing apparatus, computer program, and storage medium for executing the method. This testing method for the control device of an autonomous driving vehicle has higher efficiency. Background Technology
[0002] Testing can be conducted during the development of autonomous vehicles. For example, testing can be carried out in real-world scenarios: image data collected by one or more cameras on the vehicle is transmitted in real time to the vehicle's control unit, which then autonomously controls the vehicle based on this image data. This requires equipping the vehicle with the necessary hardware and assigning personnel to perform the testing. This means that testing in real-world scenarios is time-consuming. Another testing method is virtual testing: similar to computer games, it uses algorithms such as renderers to artificially generate image data for testing. However, developing such renderers is also time-consuming. Summary of the Invention
[0003] A control device is configured to autonomously control a motor vehicle based on environmental data describing the vehicle's surrounding environment. The testing method for this control device includes the following steps: a) inputting text describing a driving scenario into a first neural network to generate an environmental dataset of simulated environmental data; b) inputting the environmental dataset into the control device, which then generates a control device output dataset, including control data configured by the control device for controlling the motor vehicle and state data describing the motor vehicle's state; c) outputting the control device output dataset from the control device to a second neural network, which is different from the first neural network; d) outputting a network output dataset from the second neural network, which is generated based on the control device output dataset; e) generating another environmental dataset simulating the environment based on the network output dataset and the first neural network, where at least a portion of the other environmental dataset differs from the previous environmental dataset; f) repeating steps b) to e) at least once, in step b) inputting the other environmental dataset generated in the previous step e) into the control device.
[0004] For example, the text input in step a) can be done manually. Since both the environmental dataset and the other environmental dataset are generated by the first neural network, this method generates environmental data that more closely resembles real-world scenarios compared to methods that generate environmental data using renderers or other algorithms. This method is simple to operate as it eliminates the need for sensor-equipped vehicles to collect data in real-world environments to generate realistic environmental data. Because the output dataset of the second neural network can be used to generate another environmental dataset, subsequent testing can proceed without requiring manual input of text descriptions for new driving scenarios, thus automating the testing process. Therefore, the second neural network can autonomously generate new driving scenarios, significantly reducing the implementation complexity of this testing method. Furthermore, the second neural network can be designed as a small network, which has the advantage of being able to be stored locally on devices such as personal computers, rather than on remote servers in the cloud, which improves data security. Simultaneously, the data output of the second neural network has lower bandwidth requirements. Specifically, the second neural network can generate driving scenarios with progressively increasing difficulty.
[0005] The second neural network can be a simplified version of the first neural network. It can be trained through fine-tuning, meaning that the structure of the first layer of the neural network remains unchanged, and only the layers after the first layer are adjusted.
[0006] This method can be terminated when the status data indicates that there is a driving error in the control device.
[0007] When status data indicates a driving error in the control unit, an error message can be output. This error message may include specific environmental data that caused the driving error, in order to conduct error analysis.
[0008] In step a), the input text can be processed by a text encoder to generate a text encoder output dataset, which is then input into the first neural network. Thus, the text encoder can convert human-readable, editable input text into a text encoder output dataset in the form of a numerical sequence, which can serve as input data for the first neural network.
[0009] The text encoder can employ a large language model (LLM) or contain a large language model. The first neural network can employ a conditional generative adversarial network (cGAN) or contain a conditional generative adversarial network. The second neural network can employ a large language model or contain a large language model, and the second neural network can employ a transformer architecture (Transformer architecture).
[0010] Environmental data preferably includes image data, and may optionally include radar data and / or lidar data.
[0011] An environmental dataset can simulate environmental data at a single point in time, and correspondingly, another environmental dataset can also simulate environmental data at a single point in time; or, an environmental dataset can simulate environmental data at multiple consecutive points in time, and correspondingly, another environmental dataset can also simulate environmental data at multiple consecutive points in time.
[0012] The test apparatus disclosed herein is configured to perform the above-described test method. The test apparatus may include a storage medium and / or a processor for implementing the method, and may also include a control device.
[0013] The computer program disclosed herein contains instructions that, when executed by a computer, cause the computer to perform the methods described above.
[0014] The computer-readable storage medium disclosed herein contains instructions that, when a computer executes a computer program stored in the storage medium, cause the computer to perform the methods described above. Brief description of the attached figures The present invention will now be described in further detail with reference to the accompanying drawings, in which: Figure 1 A diagram illustrating the data flow; and Figure 2 This is a schematic diagram of the method steps. Detailed Implementation
[0015] Figure 1 The diagram illustrates a test method for control device 6, which is configured to autonomously control the vehicle using environmental data 5 describing the environment surrounding the vehicle. The control device may include a device processor and / or device storage medium for achieving autonomous control of the vehicle based on the environmental data. The model and / or software to be tested may be stored in the device storage medium; alternatively, the hardware of control device 6 (including the device processor and / or device storage medium) may also be tested.
[0016] Environmental data 5 may include, for example, image data such as image data acquired by one or more cameras, radar data, and / or lidar data.
[0017] The method includes step a): inputting text describing a driving scenario into a first neural network 4 to generate an environmental dataset of simulated environment data 5. For example, text input 1 can be manually completed, and the text can contain words and / or complete sentences to describe the driving scenario. For example, the text can contain information such as lighting conditions (e.g., day or night), weather conditions (e.g., rain, snow, cloudy or sunny), traffic conditions (e.g., no other vehicles, traffic congestion, parked vehicles or no parked vehicles), and / or road type (e.g., urban roads, rural roads, highways). The first neural network 4 can, for example, employ a conditional generative adversarial network, or incorporate a conditional generative adversarial network.
[0018] In step a), the text input 1 can be processed by the text encoder 2 to generate a text encoder output dataset 3, which is then input into the first neural network 4, specifically directly into the first neural network 4. For example, the text encoder 2 may employ a large language model, or contain a large language model.
[0019] The method includes step b): inputting an environmental dataset to a control device 6, which then generates a control device output dataset. This output dataset includes control data used by the control device 6 to control the motor vehicle and state data describing the state of the motor vehicle. The control data may include information such as steering angle, accelerator pedal position, gear position, and / or brake pedal position; the state data may include information such as vehicle speed, vehicle direction of travel, and whether driving errors exist and / or whether driving scenarios were successfully handled.
[0020] The method includes step c): outputting the control device output dataset from the control device 6 to a second neural network 7 that is completely independent of the first neural network 4, and the two are different neural networks. For example, the second neural network 7 may employ a large language model, or contain a large language model. The second neural network 7 may employ a converter architecture.
[0021] The method includes step d): outputting a network output dataset 8 from the second neural network 7, which is generated based on the output dataset of the control device. The network output dataset 8 may be in the form of a numerical sequence rather than text; it may be directly input to the text encoder 2. The neural layers through which the text input 1 is transmitted in the text encoder 2 are different from those through which the network output dataset 8 is transmitted in the text encoder 2. Besides being directly input to the text encoder 2, the network output dataset 8 may also be directly input to the first neural network 4.
[0022] The method includes step e): combining the network output dataset 8 and the first neural network 4 to generate another environment dataset for the simulated environment 5, which differs from the previous environment dataset in at least some way.
[0023] The method includes step f): repeating steps b) to e) at least once, for example, multiple times, wherein in step b), another environmental dataset generated in the previous step e) is input to the control device 6.
[0024] The first neural network 4 is configured to simulate environment data 5; the second neural network 7 is configured to parse the output dataset based on the state data contained in the output dataset of the control device and generate a network output dataset 8, and the first neural network 4 generates another environment dataset based on the network output dataset 8.
[0025] When the status data indicates that there is a driving error in the control device 6, this test method can be terminated; when there is a driving error in the control device 6, error information 9 can be output, which may include specific environmental data 5 that shows the driving error.
[0026] An environmental dataset can simulate environmental data 5 at a single point in time, and correspondingly, another environmental dataset can also simulate environmental data 5 at a single point in time; or, an environmental dataset can simulate environmental data 5 at multiple consecutive points in time, and correspondingly, another environmental dataset can also simulate environmental data 5 at multiple consecutive points in time.
[0027] Figure 2 Steps 11 to 16 illustrate the specific flow of this method. In step 11, text input 1 is performed, where the user can describe the driving scenario. In step 12, text input 1 is fed to text encoder 2, which outputs text encoder output dataset 3. In step 13, text encoder output dataset 3 is fed to a first neural network 4, which outputs environmental data 5 in the form of an environmental dataset or another environmental dataset. In step 14, environmental data 5 is fed to a control device 6, which outputs a control device output dataset. In step 15, the control device output dataset is fed to a second neural network 7, which outputs a network output dataset 8. In step 18, network output dataset 8 is transmitted to the text encoder, after which the method flow returns to step 12 to continue subsequent testing.
[0028] Explanation of reference numerals in the attached figures 1. Text Input 2. Text Encoder 3. Text encoder output dataset 4. First Neural Network 5. Environmental Data 6. Control device 7. Second Neural Network 8. Network output dataset 9. Error Message 10. Steps 11. Steps 12. Steps 13. Steps 14. Steps 15. Steps 16. Steps
Claims
1. A method for testing a control device, the control device being configured to autonomously control the vehicle based on environmental data describing the vehicle's surrounding environment, the method comprising the following steps: a) Input the text describing the driving scenario into the first neural network to generate an environmental dataset simulating the environmental data; b) The environmental dataset is input to the control device, and the control device generates a control device output dataset, which includes control data configured by the control device for controlling the motor vehicle, and status data describing the state of the motor vehicle. c) Outputting the control device output dataset from the control device to a second neural network, the second neural network being different from the first neural network; d) Output a dataset from the second neural network, the dataset being generated based on the dataset output by the control device; e) Based on the network output dataset and the first neural network, generate another environmental dataset simulating the environment, wherein at least a portion of the other environmental dataset differs from the previous environmental dataset; f) Repeat steps b) through e) at least once, in step b) inputting the other environmental dataset generated in the previous step e) into the control device.
2. The method according to claim 1, wherein, If the status data indicates that the control device has a driving error, then the method is terminated and / or an error message is output.
3. The method according to claim 2, wherein, The error message includes displaying environmental data that caused the driving error.
4. The method according to claim 1, wherein, In step a), a first text encoder output dataset is generated from the input text using a text encoder, and the first text encoder output dataset is input into the first neural network.
5. The method according to claim 4, wherein, The text encoder contains or is a large language model.
6. The method according to claim 1, wherein, The first neural network includes or is a conditional generative adversarial network.
7. The method according to claim 6, wherein, The second neural network has a converter architecture and / or contains a large language model.
8. The method according to claim 1, wherein, The environmental data includes image data, radar data, and / or lidar data.
9. The method according to claim 1, wherein, The environmental dataset simulates environmental data at only a single point in time, and the other environmental dataset simulates environmental data at only a single point in time.
10. The method according to claim 1, wherein, The environmental dataset simulates environmental data at multiple consecutive time points, and the other environmental dataset simulates environmental data at multiple consecutive time points.
11. A testing apparatus, the testing apparatus being configured as a testing control device, the control device being configured to autonomously control the motor vehicle based on environmental data describing the surrounding environment of the motor vehicle, the testing control device comprising: a) Input the text describing the driving scenario into the first neural network to generate an environmental dataset simulating the environmental data; b) The environmental dataset is input to the control device, and the control device generates a control device output dataset, which includes control data configured by the control device for controlling the motor vehicle, and status data describing the state of the motor vehicle. c) Outputting the control device output dataset from the control device to a second neural network, the second neural network being different from the first neural network; d) Output a dataset from the second neural network, the dataset being generated based on the dataset output by the control device; e) Based on the network output dataset and the first neural network, generate another environmental dataset simulating the environment, wherein at least a portion of the other environmental dataset differs from the previous environmental dataset; f) Repeat steps b) through e) at least once, in step b) inputting the other environmental dataset generated in the previous step e) into the control device.
12. The method according to claim 11, wherein, If the status data indicates that the control device has a driving error, then the method is terminated and / or an error message is output.
13. The method according to claim 11, wherein, The error message includes displaying environmental data that caused the driving error.
14. The method according to claim 11, wherein, In step a), a first text encoder output dataset is generated from the input text using a text encoder, and the first text encoder output dataset is input into the first neural network.
15. The method according to claim 14, wherein, The text encoder contains or is a large language model.
16. The method according to claim 11, wherein, The first neural network includes or may include a conditional generative adversarial network.
17. The method according to claim 16, wherein, The second neural network has a converter architecture and / or contains a large language model.
18. The method according to claim 11, wherein, The environmental data includes image data, radar data, and / or lidar data.
19. The method according to claim 11, wherein, The environmental dataset simulates environmental data at only a single point in time, and the other environmental dataset simulates environmental data at only a single point in time.
20. The method according to claim 11, wherein, The environmental dataset simulates environmental data at multiple consecutive time points, and the other environmental dataset simulates environmental data at multiple consecutive time points.