Multi-modal intention fusion method and application

A fusion method, multi-modal technology, applied in the field of information fusion, can solve the problem of difficulty in obtaining the real intention of users

Active Publication Date: 2019-09-27
UNIV OF JINAN
View PDF5 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] However, in the real operation process of the user, when an intention is generated, the user will not operate immediatel

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Multi-modal intention fusion method and application
  • Multi-modal intention fusion method and application
  • Multi-modal intention fusion method and application

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0072] In order to clearly illustrate the technical features of this solution, the present invention will be described in detail below through specific implementation modes and in conjunction with the accompanying drawings. The following disclosure provides many different embodiments or examples for implementing different structures of the present invention. To simplify the disclosure of the present invention, components and arrangements of specific examples are described below. Furthermore, the present invention may repeat reference numerals and / or letters in different instances. This repetition is for the purpose of simplicity and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed. It should be noted that components illustrated in the figures are not necessarily drawn to scale. Descriptions of well-known components and processing techniques and processes are omitted herein to avoid unnecessarily limiting the...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a multi-modal intention fusion method. The method comprises steps of acquiring sound information and visual information of a user through a sensor; converting the acquired sound information into a plurality of voice intentions by utilizing an intention perception algorithm, and converting the visual information into operation intentions; determining a real voice intention of the user through voice intention competition; enabling the operation intention to act on the sample image, and presenting an operation result on a screen; judging the real operation intention of the user; and constructing a system feedback rule base, inquiring and outputting corresponding system feedback according to the real operation intention and the real voice intention of the user, and guiding the user to operate. The invention also discloses a virtual microscope, and the virtual microscope comprises an input module, a control module and an output module, so that the device can sense the real intention of a user, gives corresponding feedback guidance, effectively reduces the number of times of misoperation of the user, and is convenient for the user to better complete a microscope operation experiment.

Description

technical field [0001] The present invention relates to the field of information fusion, in particular to a multimodal intention fusion method and application. Background technique [0002] Virtual experiment is a typical application scenario of human-computer interaction, but in most interactive scenarios of virtual experiment, the operation command cannot be expanded, because the single-pass interaction method cannot fully reflect the user's interaction intention. The same limitation is also reflected in the traditional microscope operation experiment. The traditional method has the problem of being unable to perceive the operator's intention and interact with the user. [0003] The purpose of the fusion of multi-channel information (or multi-modal information fusion) is to build a human-computer interaction method that is more in line with the human cognitive model, so as to perform more efficient and natural human-computer interaction. [0004] However, in the actual op...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06K9/62
CPCG06F18/25
Inventor 冯志全李健冯仕昌徐涛
Owner UNIV OF JINAN
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap