Neural Network Signal Extraction Using Reconstruction Mask Estimation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing signal extraction methods, such as those using deep learning techniques, face limitations in accurately separating target speaker signals from mixed audio due to the lack of true values for reconstruction masks and the difficulty in replicating real-world noise factors like conversation and reverberation in artificial data.
Innovation Solution
A neural network system that combines a first network for estimating a reconstruction mask and a second network for classifying the target signal, using an anchor signal and mixed signal to improve the accuracy of signal extraction by calculating a loss function and updating network parameters based on the classification of the extracted target signal into its true class.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a reconstruction mask is learned based on an anchor signal and mixed audio signal using a simple loss function, then the learning process is simple and fast, but the accuracy of signal extraction is insufficient due to lack of true values and inability to handle complex noise environments
Solution Approach 1:
The patent divides the signal extraction task into two separate networks: a first network for estimating the reconstruction mask and a second network for classifying the target signal. This segmentation allows each network to specialize in one function, improving overall accuracy while maintaining manageable complexity through modular design
Solution Approach 2:
The patent introduces an intermediary classification step between mask estimation and final signal extraction. The second network classifies the extracted signal to provide feedback about extraction quality, acting as a mediator that guides the optimization process and improves accuracy without requiring true value labels
2Reliability
If artificial learning data is generated to replicate real-world noise factors, then the training data can be controlled and prepared in advance, but it is difficult to sufficiently reflect factors like conversation exchange and reverberation present in actual data
Solution Approach 1:
The system uses the second network to perform self-evaluation of the extracted signals through classification. This self-service mechanism allows the system to automatically assess extraction quality and guide learning without requiring manually prepared labeled data, thereby improving reliability while maintaining ease of data preparation
Solution Approach 2:
The patent implements a feedback mechanism where the classification results from the second network are used to optimize the loss function for the first network. This feedback loop allows the system to learn from its own performance and continuously improve extraction accuracy without requiring external labeled training data
3Measurement precision
If clustering methods are used to specify speaker segments from segmented audio signals, then the method is simple to implement, but temporarily overlapping audio signals of speakers cannot be separated due to noise influence
Solution Approach 1:
The patent replaces the mechanical clustering approach with a neural network-based signal processing system. The first network estimates a reconstruction mask through learned features rather than simple distance-based clustering, enabling accurate separation of overlapping speaker signals while maintaining ease of operation through automated processing
Data Source
AI summary
A neural network input unit 81 inputs a neural network in which a first network having a layer for inputting an anchor signal belonging to a predetermined class and a mixed signal including a target signal belonging to the class and a layer for outputting, as an estimation result, a reconstruction mask indicating a time-frequency domain in which the target signal is present in the mixed signal, and a second network having a layer for inputting the target signal extracted by applying the mixed signal to the reconstruction mask and a layer for outputting a result obtained by classifying the input target signal into a predetermined class are combined. A reconstruction mask estimation unit 82 applies the anchor signal and mixed signal to the first network to estimate the reconstruction mask of the class to which the anchor signal belongs. A signal classification unit 83 applies the mixed signal to the estimated reconstruction mask to extract the target signal, and applies the extracted target signal to the second network to classify the target signal into the class.