Distributed Internet of Things equipment collaboration method and system based on multimodal interaction

Abstract

The invention discloses a distributed Internet of Things equipment collaboration method and system based on multimodal interaction. The method comprises the steps as follows: each piece of distributedInternet of Things sub-equipment respectively collects a voice signal in real time via a microphone to perform voice wakeup judgment; on equipment waken up via voice, a camera is started to collect ahuman face image in real time to perform face detection, and a face detection result is sent to an interaction center control via network communication; the interaction center control performs arbitration and collaboration according to a voice wakeup result and the face detection result reported by each piece of sub-equipment; the equipment really responding to wakeup of a user is determined, voice commands are continuously monitored, and meanwhile wakeup information of the other sub-equipment is eliminated; and after the voice commands of the user are processed in real time, the corresponding control commands and voice reply content are issued to the Internet of Things sub-equipment responding to the wakeup. According to the method and system provided by the invention, arbitration and collaboration are performed according to a multimodal result via the distributed Internet of Things equipment and the interaction center control, and accuracy of collaborative interaction and response of the distributed Internet of Things equipment is improved.

Description

technical field

[0001] The invention relates to the technical field of artificial intelligence, in particular to a multi-modal interaction-based distributed Internet of Things device collaboration method and system thereof. Background technique

[0002] With the continuous development of technology in the field of artificial intelligence, the accuracy of speech recognition and face detection has been continuously improved, which has led to the emergence of many intelligent voice devices in our daily life. The built-in microphone or microphone array in the smart voice device can realize the close or far-field interaction between the user and the smart device at a certain distance, but beyond this distance range, the accuracy of voice interaction will decrease or even cannot be realized. Now many smart devices with voice interaction are distributed in our home environment, such as smart voice speakers in the living room, smart desk lamps in the bedroom, etc., and the devices a...

Images