Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

A multi-modal intent fusion method and application

A fusion method and multi-modal technology, applied in the field of information fusion, can solve problems such as difficulty in obtaining the user's true intention, achieve the effect of reducing the operational burden and cognitive burden, and reducing the number of misoperations

Active Publication Date: 2021-09-28
UNIV OF JINAN
View PDF5 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] However, in the real operation process of the user, when an intention is generated, the user will not operate immediately, and the operator may generate a new intention during the period, so it is difficult to obtain the real intention of the user

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A multi-modal intent fusion method and application
  • A multi-modal intent fusion method and application
  • A multi-modal intent fusion method and application

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0072] In order to clearly illustrate the technical features of this solution, the present invention will be described in detail below through specific implementation modes and in conjunction with the accompanying drawings. The following disclosure provides many different embodiments or examples for implementing different structures of the present invention. To simplify the disclosure of the present invention, components and arrangements of specific examples are described below. Furthermore, the present invention may repeat reference numerals and / or letters in different instances. This repetition is for the purpose of simplicity and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed. It should be noted that components illustrated in the figures are not necessarily drawn to scale. Descriptions of well-known components and processing techniques and processes are omitted herein to avoid unnecessarily limiting the...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a multi-modal intention fusion method, which uses a sensor to acquire the user's sound information and visual information; uses an intention perception algorithm to convert the acquired sound information into several voice intentions, and converts the visual information into operation intentions; through the voice intention Competitively determine the user's real voice intention; apply the operation intention to the sample image, and present the operation result on the screen; judge the user's real operation intention; build a system feedback rule base, query and output the corresponding System feedback to guide users to operate. The invention also discloses a virtual microscope, which uses the above-mentioned multi-modal intention fusion method, including an input module, a control module and an output module, so that the device can perceive the user's real intention and give corresponding feedback guidance, effectively reducing the user's The number of misoperations is convenient for users to better complete microscope operation experiments.

Description

technical field [0001] The present invention relates to the field of information fusion, in particular to a multimodal intention fusion method and application. Background technique [0002] Virtual experiment is a typical application scenario of human-computer interaction, but in most interactive scenarios of virtual experiment, the operation command cannot be expanded, because the single-pass interaction method cannot fully reflect the user's interaction intention. The same limitation is also reflected in the traditional microscope operation experiment. The traditional method has the problem of being unable to perceive the operator's intention and interact with the user. [0003] The purpose of the fusion of multi-channel information (or multi-modal information fusion) is to build a human-computer interaction method that is more in line with the human cognitive model, so as to perform more efficient and natural human-computer interaction. [0004] However, in the actual op...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06K9/62
CPCG06F18/25
Inventor 冯志全李健冯仕昌徐涛
Owner UNIV OF JINAN
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com