Interactive identity authenticating and tracking method and system based on multi-modal intelligent robot

An intelligent robot and identity authentication technology, applied in the field of intelligent robots, can solve the problems of lack of single-modal information, inability to recognize and track interactive identities, etc., and achieve the effect of improving recognition accuracy and efficiency

Active Publication Date: 2018-03-16
HENAN UNIVERSITY
View PDF5 Cites 22 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0004] The purpose of the present invention is to solve the problem of multi-modal identity recognition of intelligent robots, and provide a method and system for multi-modal identification and tracking of interactor identities of dialogue intelligent robots, which can solve the lack of single-modal information and cannot identify and track interactions identity issue

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Interactive identity authenticating and tracking method and system based on multi-modal intelligent robot
  • Interactive identity authenticating and tracking method and system based on multi-modal intelligent robot
  • Interactive identity authenticating and tracking method and system based on multi-modal intelligent robot

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0056] like figure 1 As shown, a kind of interactive identification and tracking method based on multimodal intelligent robot of the present invention comprises the following steps:

[0057] Step S101: collecting multi-channel information of interactive objects;

[0058] Step S102: using a deep neural network to extract activation features to form multi-modal features of multi-channel interactive information;

[0059] Step S103: using integrated learning of cross-modal probability cognition to identify the face image features and voiceprint features;

[0060] Step S104: Register and track the identity of the interactor identified by the cross-modal attribute.

Embodiment 2

[0062] like figure 2 As shown, another kind of interactive identification and tracking method based on multimodal intelligent robot of the present invention comprises the following steps:

[0063] Step S201: collecting multi-channel information MI of an interactive object, including:

[0064] As a possible implementation, the intelligent robot is an intelligent dialogue robot, including: a microphone array, a binocular camera, an infrared camera and a depth sensor;

[0065] Step S2011: collect the voiceprint and voice information VA of the interactor through the microphone array of the intelligent dialogue robot, and collect information for identifying the interaction identity and understanding the interaction intention;

[0066] Step S2012: Collect the face RGB image FV and body information of the interactor through the binocular camera of the intelligent dialogue robot, and collect information for identifying the interaction identity and understanding the interaction inten...

Embodiment 3

[0086] like image 3 As shown, a kind of interactive identification and tracking system based on multimodal intelligent robot of the present invention comprises:

[0087] A multi-channel information collection module 301, configured to collect multi-channel information of interactive objects;

[0088] The multimodal feature extraction module 302 is used to extract activation features using a deep neural network to form multimodal features of multi-channel interactive information;

[0089] The cross-modal identity identification module 303 is used to identify the face image features and voiceprint features by adopting integrated learning of cross-modal probability cognition;

[0090] The interactive object registration tracking module 304 is configured to register and track the identity of the interactor identified by the cross-modal attribute.

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention belongs to the field of intelligent robots, particularly relates to a method for authenticating and tracking the identity of an interaction person for an intelligent dialogue robot, andespecially particularly relates to an interactive identity authenticating and tracking method and system based on a multi-modal intelligent robot. The interactive identity authenticating and trackingmethod based on the multi-modal intelligent robot comprises the following steps that: collecting the multichannel information of an interaction object; adopting a deep neural network to extract activation characteristics, and forming the multimodal characteristics of the multichannel interaction information; adopting ensemble learning of cross-modal probability cognition, and carrying out identityauthentication on face image characteristics and vocal print characteristics; and carrying out registration and tracking by the identity, which is authenticated by the cross-modal attribute, of the interaction person. The interactive identity authenticating and tracking system based on the multi-modal intelligent robot comprises a multichannel information collection module, a multi-modal characteristic extraction module, a cross-modal identity authentication module and an interaction object registering and tracking module. By use of the method, the problem that single modal information is inshortage and the interaction identity can not be identified and tracked is solved.

Description

technical field [0001] The invention belongs to the field of intelligent robots, and in particular relates to a method for multimodal identification and tracking of interactor identities for intelligent dialogue robots, and in particular to a method and system for identifying and tracking interactive identities based on multimodal intelligent robots. Background technique [0002] With the development of computer technology and artificial intelligence, the research of intelligent robots has gradually entered into many service fields. Human-computer interaction is an important technical issue in the research of intelligent dialogue robots. When users interact with dialogue machines, only when intelligent dialogue robots recognize and memorize the information of the interlocutors participating in the interaction can they effectively understand the meaning of dialogue or interaction. At the same time, the historical information of the interactive participants records the user's ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/00G10L17/22G10L17/18G10L17/02G06F3/0484G06F21/32
CPCG06F3/0484G06F21/32G10L17/02G10L17/18G10L17/22G06V40/166G06V40/168
Inventor 刘扬李佩琛邵阳雪
Owner HENAN UNIVERSITY
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap