Vehicle-Cloud Multimodal Data Mining for Scalable Autonomous Driving
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current autonomous driving data mining methods require large amounts of training data and labor resources, are not extensible, and lack scalability, leading to inefficiencies in optimizing autonomous driving algorithms.
Innovation Solution
A device-cloud collaborative data mining method using a large image-text multimodal model, where the cloud encodes target text and the vehicle encodes image data, with model compression and fine-tuning to reduce complexity and resource requirements, enabling efficient data mining and algorithm optimization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a large image-text multimodal model is deployed on the vehicle for data mining, then the precision and generalization performance are improved, but the device complexity and resource requirements increase
Solution Approach 1:
The patent divides the large image-text multimodal model into two parts: the text encoder is deployed on the cloud server, while only the picture encoder is deployed on the vehicle. This segmentation allows the vehicle to perform data mining with reduced model complexity while maintaining precision through cloud-based text processing support.
Solution Approach 2:
The patent introduces a cloud server as an intermediary between the vehicle and the full multimodal model. The cloud server hosts the text encoder and provides text features to the vehicle, allowing the vehicle to perform data mining without running the complete complex model locally.
2Measurement precision
If the full large image-text multimodal model is run on the vehicle, then the data mining precision is improved, but the use of energy and computational resources increase
Solution Approach 1:
The patent segments the computational workload by deploying only the picture encoder on the vehicle and the text encoder on the cloud. This reduces the computational resource consumption on the vehicle while maintaining data mining precision through collaborative processing.
Solution Approach 2:
The cloud server acts as an intermediary that handles the computationally intensive text encoding tasks, allowing the vehicle to conserve energy by not running the complete multimodal model locally while still achieving accurate data mining results.
3Ease of operation
If traditional task models are designed and developed for specific requirements, then the ease of operation is improved, but the adaptability and scalability are reduced
Solution Approach 1:
The patent employs a universal picture encoder model that can handle multiple data mining tasks through a single deployed module on the vehicle. The model's generalization capability allows it to adapt to different requirements without needing separate task-specific models, thereby improving scalability while maintaining ease of operation.
Data Source
AI summary
A device-cloud collaborative data mining method includes: A cloud determines a target text and a task configuration file based on a service requirement; encodes the target text by using a text encoder to obtain a text feature; and places the text feature in the task configuration file and delivers the task configuration file to a vehicle. The vehicle encodes image data by using a first picture encoder to obtain an image feature; calculates a value of a similarity between the text feature and the image feature; determines a target picture based on the value of the similarity and the task configuration file; and uploads the target picture to the cloud. The first picture encoder is obtained by compressing and tuning a second picture encoder, and the text encoder and the second picture encoder are two modules of a large image-text multimodal model.