Looking for breakthrough ideas for innovation challenges? Try Patsnap Eureka!

Microphone array speech enhancement system and method based on multi-task network

A microphone array and voice enhancement technology, which is applied in voice analysis, instruments, etc., can solve problems such as difficult training, achieve voice enhancement, strong noise reduction performance, and overcome performance deficiencies

Pending Publication Date: 2022-07-01
SOUTH CHINA UNIV OF TECH
View PDF0 Cites 2 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The system and method aim at the shortcomings of existing network systems such as difficulty in training and easy to fall into local minimum, refine the structure of the deep neural network according to the function of the speech enhancement system, and provide an echo cancellation sub-network, de-aliasing The multi-task speech enhancement network model composed of ring sub-network and noise reduction sub-network can effectively reduce the difficulty of network training, make up for the defects caused by a single network target, and can significantly improve the effect of speech enhancement

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Microphone array speech enhancement system and method based on multi-task network
  • Microphone array speech enhancement system and method based on multi-task network
  • Microphone array speech enhancement system and method based on multi-task network

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0066] This embodiment discloses a microphone array speech enhancement system based on a multi-task network, the system structure is as follows figure 1 As shown, the system consists of a speech preprocessing module, a multi-task network module, a multi-task loss statistics module, a network weight calculation module and a speech reconstruction module. The speech preprocessing module is connected with the multi-task network module and the multi-task loss statistics module. This module obtains the array speech, the reference echo speech and the target speech of each task as the input speech, and preprocesses these input speeches. The preprocessing work includes speech signals. The logarithmic amplitude spectrum of each channel speech and the reference echo speech is extracted; the multi-task network module is connected with the speech preprocessing module, the multi-task loss statistics module and the network weight calculation module to complete the removal of each channel of t...

Embodiment 2

[0070] Based on the multi-task network-based microphone array speech enhancement system disclosed in the above embodiment, this embodiment continues to disclose a multi-task network-based microphone array speech enhancement method. The method adopts the following steps to complete training and testing, training and testing Process such as image 3 shown:

[0071] S1. Construct an array speech training set, preprocess the speech, and obtain the input features of each channel and the labels of the de-reverberation task, the echo cancellation task, the noise reduction task, and the fusion task; the process is as follows:

[0072] S1.1. Construct noisy array speech and corresponding de-reverberated array speech, de-reverberated and de-echoed array speech and noise-free array speech:

[0073] The noisy array speech is x(n)=[x 1 (n),x 2 (n),...,x m (n),...,x M (n)] T ,m∈[1,M], where M=4 is the total number of array elements, the generation of noisy speech is as follows Figur...

Embodiment 3

[0112] Based on the multi-task network-based microphone array speech enhancement system disclosed in the above embodiment, this embodiment continues to disclose a multi-task network-based microphone array speech enhancement method. The method adopts the following steps to complete training and testing, training and testing Process such as image 3 shown:

[0113] S1. Construct an array speech training set, preprocess the speech, and obtain the input features of each channel and the labels of the de-reverberation task, the echo cancellation task, the noise reduction task, and the fusion task; the process is as follows:

[0114] S1.1. Construct noisy array speech and corresponding de-reverberated array speech, de-reverberated and de-echoed array speech and noise-free array speech:

[0115] The noisy array speech is x(n)=[x 1 (n),x 2 (n),...,x m (n),...,x M (n)] T ,m∈[1,M], where M=4 is the total number of array elements, the generation of noisy speech is as follows Figur...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention discloses a microphone array voice enhancement system and method based on a multi-task network. The system is composed of a voice preprocessing module, a multi-task network module, a multi-task loss statistics module, a network weight calculation module and a voice reconstruction module. Wherein the voice preprocessing module acquires array voice, reference echo voice and target voice of each task as input voice and preprocesses the input voice; the multi-task network module completes reverberation removal, echo cancellation and noise reduction tasks of each sound channel of the array voice, fuses the multi-sound-channel voice and outputs the multi-sound-channel voice as enhanced voice; the multi-task loss statistics module is used for calculating the loss value of each task in the multi-task network module and counting the total loss of the network; the network weight calculation module calculates a gradient according to the total loss of the network, carries out back propagation on the gradient, and calculates the weight of the updated network; and the voice reconstruction module completes mapping from the frequency domain features to the time domain voice to obtain enhanced clean voice.

Description

technical field [0001] The present invention relates to the technical field of speech enhancement, in particular to a microphone array speech enhancement system and method based on a multi-task network. Background technique [0002] Speech enhancement based on microphone array is one of the effective methods to suppress interference in speech communication system. Existing microphone array speech enhancement techniques can be mainly divided into two categories: traditional enhancement techniques and enhancement techniques based on deep neural networks. Traditional enhancement technologies usually design corresponding filters for the related characteristics of echo cancellation, spatial reverberation, and environmental noise, which require less hardware computing power and can achieve better real-time performance. However, traditional enhancement technologies still have the following: Disadvantages: (1) The nonlinear noise components cannot be eliminated well, so that the ou...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
IPC IPC(8): G10L21/0208G10L21/0224G10L21/0232
CPCG10L21/0208G10L21/0224G10L21/0232G10L2021/02082G10L2021/02166
Inventor 张军赖志鹏宁更新冯义志余华陈芳炯温淼文季飞
Owner SOUTH CHINA UNIV OF TECH
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Patsnap Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Patsnap Eureka Blog
Learn More
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic, Popular Technical Reports.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap|About US| Contact US: help@patsnap.com