Robot man-machine interaction method and device based on gesture and voice recognition

A technology of speech recognition and gesture recognition, applied in the field of human-computer interaction and robotics, to improve the effect of human-computer interaction

Active Publication Date: 2016-04-06
BEIHANG UNIV
View PDF3 Cites 66 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0008] The purpose of the present invention is to provide a robot human-computer interaction method and device based on gesture and voice

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Robot man-machine interaction method and device based on gesture and voice recognition
  • Robot man-machine interaction method and device based on gesture and voice recognition
  • Robot man-machine interaction method and device based on gesture and voice recognition

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0065] The following is the implementation process of the human-computer interaction method based on gesture and voice recognition between the user and the hexapod leg / arm compound mobile robot in the interactive mode of the present invention:

[0066] (1) Determine whether there is an interactive object near the robot. When the robot detects voice information through the microphone, such as "hello" and "good morning", or detects specific gesture information through the camera, such as waving, it thinks that there is an interactive object near the robot, turns on the interactive mode, and broadcasts through the speaker Play voice information in the form of "Hello, what's your order?".

[0067] (2) In the interactive mode, the user can interact with the robot with predefined gestures or voice. Implementation examples are given below for the gesture mode, voice mode and joint mode in the interaction mode.

[0068] a. Gesture mode, the implementation example of which the user c...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a robot man-machine interaction method and device based on gesture and voice recognition belonging to the technical field of the man-machine interaction and robot. The device comprises a sensor module, a gesture recognition module, a voice recognition module, an information fusion module, an information confirmation module, a robot control module and an emergency help module; the method is: fusing a gesture recognition result and a voice recognition result by the information fusion module, sending the obtained final fusion result to the information confirmation module or the robot control module for realizing emergency help and action execution of the robot. The invention combines two channels of the gesture and voice, overcomes the defect of the single channel and can controls the robot better. The invention can be applied to remotely control the robot, enables the robot to work in fields such as space station, hazardous article treatment and public security instead of human beings, also can realize face to face direct interaction of the man and the robot and can be applied in fields such as medical treatment and domestic service.

Description

technical field [0001] The invention belongs to the field of human-computer interaction and robot technology, and in particular relates to a robot human-computer interaction method and device based on gesture recognition and voice recognition. Background technique [0002] Our country is entering an aging society. The elderly need to be taken care of, while the young and middle-aged need to work to make ends meet and have no time to take care of the elderly. Robots can act as part of the labor force, such as helping the elderly, the young, the sick, and the disabled to conceive. In this process, good human-computer interaction plays an important role. [0003] Human-computer interaction usually has three different descriptions according to different research fields (see reference [1]: Arvin Agah, Humaninteractionswithintelligentsystems: research taxonomy, Computers and Electrical Engineering, 27, 2001, pp71-107). In a broad sense, human-computer interaction refers to the i...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06F3/01
CPCG06F3/011G06F3/017
Inventor 丁希仑齐静
Owner BEIHANG UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap