Multi-mode instruction following data set construction method special for automatic driving
By constructing multimodal instruction-compliant data sets, using control signal processing, meta-action rule formulation and behavioral interpretation text generation methods, the problem of lack of interpretability in the decision-making process of end-to-end autonomous driving system is solved, and a high-quality data set supporting training of multimodal large language model is provided.
Patent Information
- Application Number
- CN202510221303.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-02-27
- Publication Date
- 2025-03-25
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The lack of interpretability in the decision-making process of existing end-to-end autonomous driving systems leads to low public trust, and the application of multimodal large language models in this field is limited by scarce instructions following data sets.
A multimodal instruction follows the data set construction method. Through three stages of control signal processing, meta-action rule formulation and behavioral interpretation text generation, a data set is constructed and integrated into a multi-view scene image + multiple rounds of question and answer pairs, which simulates the thinking chain and dynamic time reasoning of human drivers.
It effectively alleviates the explanatory problem of the end-to-end autonomous driving decision-making process, provides high-quality data set support for multimodal large language models, and promotes its training and implementation in autonomous driving tasks.