Multimodal Data Transmission Using Modality Identifiers
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing agent interactions in the 6G era are passive and unimodal, leading to errors in data processing due to the inability to determine the modality of input data, which affects the accuracy of processing results.
Innovation Solution
A method for sending multimodal data that includes identifying the modality of each data piece using an identifier and sending it to a centralized processing device, ensuring accurate processing by determining the correct processing mode for each modality.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If multimodal data is sent without modality identifiers, then the transmission process is simpler, but the centralized processing device cannot determine the modality of input data leading to processing errors
Solution Approach 1:
A modality identifier is introduced as an intermediary element between the terminal and centralized processing device. This identifier acts as a mediator that carries modality information without being part of the core data payload, enabling the processing device to identify data types while maintaining transmission efficiency. The identifier serves as a metadata layer that resolves the contradiction by adding minimal overhead for reliable processing.
2Adaptability or versatility
If unimodal data processing is used, then the processing system is simpler, but it cannot handle diverse input types from multiple devices in the 6G era
Solution Approach 1:
The patent implements universality by creating a single standardized data transmission format that can handle multiple modalities (audio, video, text, etc.) through a common framework. The modality identifier enables the processing system to universally accept and correctly process different data types without requiring separate processing paths for each modality, thus achieving multi-device compatibility while controlling system complexity.
3Measurement precision
If modality identifiers are added to each data piece, then the centralized processing device can accurately determine data modality, but the data transmission overhead increases
Solution Approach 1:
The modality identifier is designed with local quality optimization by using a compact, standardized format that provides sufficient modality information with minimal bits. Rather than transmitting complete modality descriptions, the system uses concise identifiers that convey the necessary information for correct processing while minimizing overhead. This localized optimization ensures high identification accuracy with low transmission cost.
Data Source
AI summary
A multimodal data sending method includes: acquiring at least one piece of data, wherein each of the at least one piece of data has a different modality; determining the identifier corresponding to the modality of each piece of data; and sending at least one piece of data and at least one identifier to a centralized processing device. The identifier corresponding to the modality of each piece of data can be determined, such that not only can at least one piece of data be sent to the centralized processing device, but the identifier corresponding to the modality of each piece of data can be sent to the centralized processing device, so as to enable the centralized processing device to determine the modality of each piece of data according to the identifier.