Cloud-Rendered Multimodal Interaction for Lower User Access Costs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current intelligent agents primarily use voice and text interaction modes, requiring users to jump through multiple steps and integrate various media tool plug-ins, increasing user access costs and disrupting continuous interaction experiences.
Innovation Solution
A multimodal information interaction method that includes intention recognition, calling a first multimodal processing module to determine media resource addresses, and a second module to render media resources directly in the cloud, outputting media streams to the terminal.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If intelligent agents use voice and text interaction modes with media tool plug-ins, then interaction flexibility is improved, but user access costs increase and continuous interaction experience is disrupted
Solution Approach 1:
The patent combines multiple media processing functions (image, video, audio, document rendering) into a unified cloud-based multimodal processing system. Instead of requiring users to integrate multiple media tool plug-ins locally, the system merges these capabilities into a single accessible interface that handles diverse media formats through centralized cloud processing, thereby reducing user access costs while maintaining interaction flexibility.
Solution Approach 2:
The patent introduces a cloud-based multimodal processing system as an intermediary between users and various media resources. This intermediary handles media rendering and processing remotely, eliminating the need for users to directly interact with multiple media tool plug-ins. The system mediates between user requests and media resources, providing a streamlined access path that reduces complexity and improves ease of operation.
2Speed
If media resources are rendered locally on terminal devices, then response speed is improved, but terminal application volume increases
Solution Approach 1:
The patent extracts media rendering and processing functions from terminal devices and relocates them to a cloud-based multimodal processing system. By taking out these resource-intensive operations from local terminals, the system reduces terminal application volume while maintaining fast response speeds through optimized cloud processing and efficient media stream transmission back to users.
Solution Approach 2:
The patent shifts media processing from the local terminal dimension to the cloud infrastructure dimension. Instead of rendering media resources locally on terminal devices, the system processes media in the cloud and transmits only the essential media streams to terminals, effectively moving the computational burden to another dimension (cloud infrastructure) while preserving fast user experience through optimized transmission.
3Adaptability or versatility
If multiple media tool plug-ins are integrated on terminal devices, then media format support is improved, but device complexity increases
Solution Approach 1:
The patent implements a universal cloud-based multimodal processing system that handles multiple media formats (images, videos, audio, documents) through a single multi-functional platform. Instead of requiring terminals to integrate multiple specialized media tool plug-ins, the system provides universal media processing capabilities in the cloud, supporting diverse formats without increasing terminal device complexity.
Data Source
AI summary
A multimodal information interaction method, an intelligent agent, an electronic device, and a storage medium are provided, which relate to a field of artificial intelligence technology, and in particular, to fields of large model and human-computer interaction technology. The method includes: performing intention recognition on a media resource request from a terminal to obtain an intention recognition result, where the intention recognition result represents whether the media resource request hits a predetermined processing mode; in response to the media resource request hitting the predetermined processing mode, determining a media resource address corresponding to the media resource request; and rendering a media resource in the media resource address, and outputting the rendered media stream to the terminal.