Multimodal man-machine interaction method based on reinforcement learning

Abstract

In order to solve the problem of performance bottleneck caused by data mismatching in a traditional man-machine interaction method, the invention discloses a multi-modal man-machine interaction method based on reinforcement learning, which comprises the following steps: collecting user data, requiring a user to wear a corresponding wearable sensor, recording the user data by the wearable sensor, and recording the user data by the wearable sensor, wherein the recorded data form a training set and a test set; constructing a classification algorithm model on the streaming data set in an off-line manner; and applying the constructed classification algorithm model to carry out man-machine interaction. For a synchronous man-machine interaction method, data are segmented according to an instruction synchronization label, and the data are sent to a classification algorithm model for classification; and for an asynchronous man-machine interaction method, data is cut according to a synchronization time starting point, and the cut data is used as an input sample of a classification model. According to the method, the model is directly constructed from the streaming data, the problems that a traditional man-machine interaction method is complex in development process and low in performance upper limit are solved, and better stability is achieved.

Description

technical field [0001] The invention relates to the fields of human-computer interaction and wearable sensors, and is a human-computer interaction method based on reinforcement learning. Background technique [0002] Human-computer interaction (HCI for short) is the study of communication and communication between humans and computers through mutual understanding, to complete information management, service and processing functions for people to the greatest extent, and to make computers truly a part of people's work and study. A technical science of the harmonious assistant. [0003] In recent years, with the development of integrated electronic technology, electronic sensors have become smaller and smaller in size and stronger in function. Human-computer interaction methods based on wearable sensors are gradually being applied. According to the type of information captured by the sensor, human-computer interaction methods can be divided into: gestures, eye movements and ...

AI Technical Summary

Problems solved by technology

[0006] Aiming at the performance bottleneck problem caused by the mismatch between segment data and streaming data in the traditional interactive system design method, the present invention discloses a multi-modal human-computer interaction method based on reinforcement learning, which includes the following steps:

Images