Sound separation method, sound separation device and computer readable storage medium

Abstract

The invention discloses a sound separation method, which comprises the following steps of: dividing an original audio and video sample into a plurality of audio and video segments, and extracting a video stream and an audio stream of each audio and video segment; determining face features in the video stream of each audio and video segment; obtaining audio features in the audio stream of each audio / video segment by using an audio conversion compression method; combining the face features and the audio features of each audio and video segment to generate audio-visual features of each audio and video segment; taking the audio-visual characteristics of each audio-video segment as the input of a sound separation model, and training the sound separation model to obtain a trained sound separation model; and taking the target audio and video data as input of the trained sound separation model, and outputting audio data of a person in the target audio and video data. The invention further provides a sound separation device and a computer readable storage medium. According to the invention, the accurate mapping of the sound and the speaker can be realized, and the voice separation quality is obviously improved.

Description

technical field [0001] The present invention relates to the field of computer technology, in particular to a sound separation method, device and computer-readable storage medium. Background technique [0002] Defects or deficiencies or problems in the existing industry or products: In a noisy indoor environment, such as in a cocktail party, there are many different sound sources at the same time, and it is very easy for human hearing to focus on a certain sound from a noisy environment. "Block" other voices, and the existing computer speech recognition intelligent system cannot accurately identify the target sentence in a noisy environment, cannot strengthen the voice of the selected person, and at the same time weaken the volume of other people at the same time, the existing system fails Solve the "cocktail party effect". Contents of the invention [0003] The present invention provides a sound separation method, device and computer-readable storage medium, the main purp...

AI Technical Summary

Problems solved by technology

[0002] Defects or deficiencies or problems in the existing industry or products: In a noisy indoor environment, such as a cocktail party, there are many different sound sources at the same time, and it is very easy for human hearing to focus on a certain sound from a noisy environment. "Block" other voices, and the existing computer speech recognition intelligent system cannot accurately identify the target sentence in a noisy environment, cannot strengthen the voice of the selected person, and at the same time weaken the volume of other people at the same time, the existing system fails Addressing the "cocktail party effect"

Images