System and method for realizing cross-language communication based on multimodal assistance

A multi-modal, cross-language technology, applied in the field of multimedia analysis and network communication, can solve the problems of slow processing speed and difficult use, achieve smooth semantic communication, meet individual needs, and improve efficiency and quality

Inactive Publication Date: 2011-11-30
INST OF AUTOMATION CHINESE ACAD OF SCI
View PDF3 Cites 44 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] The purpose of the present invention is to solve the technical defects of slow processing speed and difficult to use in

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • System and method for realizing cross-language communication based on multimodal assistance
  • System and method for realizing cross-language communication based on multimodal assistance
  • System and method for realizing cross-language communication based on multimodal assistance

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0028] In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be described in further detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0029] The present invention proposes a cross-language communication system based on multimodal assistance and a method for realizing cross-language communication. The method utilizes a front-end interaction module 1, a data management module 2 and a semantic association module 3 to analyze conversation content and use natural language processing tools It can automatically extract the central topic and keywords in the conversation, and the semantic association module 3 automatically searches for relevant pictures and video clips based on the detected central topic and keyword information, and provides them to both parties in an appropriate way, so as to achieve Promote mutual understanding and communication. Here, the pic...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The present invention proposes a cross-language communication system and method based on multi-modal assistance. The method utilizes the foreground interaction module, data management module and semantic association module in the cross-language communication system to analyze conversation content and use natural language processing tools It can automatically extract the central topic and keywords in the conversation, and the semantic association module can automatically search for relevant pictures and video clips based on the detected central topic and keyword information, and provide them to both parties in an appropriate way, so as to promote mutual understanding and communication. Here, the pictures and videos used as aids in understanding can be automatically picked up from the Internet by searching, or directly obtained from a pre-marked multimedia library. Finally, the system generates a multimodal conversation summary based on the text chat information of the two parties in the conversation and the corresponding picture and video content.

Description

technical field [0001] The invention belongs to the fields of multimedia analysis and network communication, and relates to a method for realizing cross-language communication based on multi-mode assistance. Background technique [0002] With the rapid development of communication technology and Internet technology, a network instant messaging system completely different from traditional communication methods such as mail, telephone, and telegram has emerged, such as MSN and QQ. Traditional emails and telegrams mainly use text, and telephone calls mainly use voice, while instant messaging can not only use text and voice, but also can assist rich multimedia methods such as video and pictures. Through the instant messaging system, people across oceans can realize face-to-face real-time conversation. The whole earth has become a veritable global village. [0003] Language issues remain an insurmountable barrier in instant messaging for interlocutors who speak different langua...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F17/28G06F17/30
Inventor 徐常胜程健梁超张歆明
Owner INST OF AUTOMATION CHINESE ACAD OF SCI
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap