Microphone array sound source signal extraction method based on covariance residual error prior guidance

Abstract

The application discloses a microphone array sound source signal extraction method based on covariance residual prior guidance, first, based on the covariance matrix of array observation data, the eigenvalue decomposition and iterative optimization algorithm are used to construct the coherent matrix residual model of unknown sound sources, then the coherent matrix is spliced as prior features and observation signals, input into a neural network, and the target sound source signal after extraction is output by the network. Through the covariance residual model, the spatial coherence characteristics of unknown sound sources can be automatically inverted from observation data without manually presetting unknown sound source parameters, and automatic perception of new sound sources is realized.

Description

Technical Field

[0001] This invention belongs to the field of array signal processing technology, specifically relating to a method for extracting microphone array sound source signals based on covariance residual prior guidance. Background Technology

[0002] In complex acoustic environments, microphone array-based sound source extraction is a core technology in fields such as speech enhancement, sound source localization, and speaker separation. Existing sound source extraction schemes are mainly divided into two categories: 1. Traditional signal processing methods: such as those based on beamforming or Wiener filtering. These methods typically rely on known sound source locations or noise covariance matrices. Given the prior knowledge of the number and locations of sound sources, extraction can be achieved by estimating the source variance and coherence matrix. However, when new, unknown, sudden sound sources appear in the environment, traditional methods, lacking prior knowledge of the spatial information of these unknown sound sources (such as the coherence matrix), struggle to effectively separate them, often resulting in interference or distortion in the extracted signal.

[0003] 2. Deep Learning Methods: In recent years, significant progress has been made in sound source separation methods based on Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), or Transformers. These methods typically learn spatiotemporal features through training on large-scale data. However, standard neural network models rely on end-to-end data-driven learning when dealing with unknown sound sources, lacking explicit spatial and physical constraints. When encountering sound source locations not present in the training set or sudden unknown sound sources, their generalization ability is limited, and it is difficult to achieve true "blind" extraction, i.e., automatically identifying and extracting new sound sources without relying on manually labeled location information.

[0004] The existing technology mainly suffers from the following technical pain points: 1. Poor adaptability to unknown sound sources: When an unknown sound source (Emerging Source) appears, traditional methods cannot accurately estimate its covariance matrix or coherence matrix, resulting in a sharp decline in signal extraction performance.

[0005] 2. Lack of physical priors: Most existing neural network models are black-box structures and do not effectively utilize spatial physical models (such as covariance residual structures) in array signal processing as constraints, resulting in slow model convergence and weak robustness to unknown scenarios.

[0006] 3. Low level of automation: Most systems require manual preset of the number or location of sound sources and cannot automatically sense and extract newly emerging sound sources in a dynamically changing acoustic environment.

[0007] Therefore, there is an urgent need for a technical solution that can combine the advantages of the physical model of array signal processing with the feature learning capabilities of neural networks to achieve automatic detection and high-precision extraction of unknown sound sources. Summary of the Invention

[0008] To overcome the shortcomings of existing technologies, this invention provides a microphone array sound source signal extraction method based on covariance residual prior guidance. First, based on the covariance matrix of the array observation data, an eigenvalue decomposition and iterative optimization algorithm is used to construct a coherence matrix residual model of the unknown sound source. Then, this coherence matrix is ​​used as a priori features and concatenated with the observed signal, input into a neural network, which outputs the extracted target sound source signal. This invention, through the covariance residual model, can automatically infer the spatial coherence characteristics of the unknown sound source from the observation data without manually pre-setting the unknown sound source parameters, thus achieving automatic perception of new sound sources.

[0009] The technical solution adopted by this invention to solve its technical problem is as follows: Step 1: Acquisition of observed signals and estimation of covariance; The multi-channel time-domain signals from the microphone array are acquired, transformed to the Short Time Fourier Transform (STFT) domain, and the covariance matrix of the array's observed signals is calculated. ; Step 2: Solve for the coherence matrix of the unknown sound source; Based on known noise or background sound source coherence matrix information, the coherence matrix of unknown sound sources is automatically solved using a residual model. ,include: Obtain the identity matrix by performing eigenvalue decomposition on a known coherence matrix. ; The phase matrix can be directly estimated using the transformed observation covariance matrix. ; Based on the Csiszar I-divergence minimization criterion, the magnitude vector is updated iteratively. ; Reconstructed and post-processed to obtain The processing includes removing negative eigenvalues ​​and trace normalization; Step 3: Embed prior information into the neural network; The coherence matrix obtained by the solution Converted into eigenvectors or tensors, and compared with the original observed signals. Perform splicing or fusion in the frequency domain or feature layer; Step 4: Signal extraction and output; The fused features are input into a trained neural network, which then outputs the extracted target unknown sound source signal based on spatial prior constraints. .

[0010] Preferably, step 1 specifically comprises: Step 1-1: Signal Acquisition and Preprocessing; take over The time-domain signal acquired by each microphone channel is framed and subjected to a Short-Time Fourier Transform (STFT) to obtain the frequency-domain signal. , m =1…. M， Indicates the frequency index; constructs a vector from all channel signals. ; Step 1-2: Estimation of the observation covariance matrix; The covariance matrix of the array observation signal is calculated using a recursive formula. :

[0011] in, It is a forgetting factor.

[0012] Preferably, step 2 specifically comprises: Step 2-1: Modeling the known background; Obtain the coherence matrix of a known background noise or a known sound source. If there is no known sound source, then it is a pure noise coherence matrix; Step 2-2: Eigenvalue decomposition; right Perform eigenvalue decomposition to obtain ,in It is the identity matrix. It is an eigenvalue diagonal matrix; Steps 2-3: Phase Matrix estimate; Calculate the transformation matrix Extract the phase angle as the phase matrix of the unknown source coherence matrix. :

[0013] Steps 2-4: Amplitude Vector Iterative optimization; Initialize the amplitude vector ,in ,and Updated via iterative formula until convergence:

[0014] Steps 2-5: Coherence matrix reconstruction; According to the model Preliminary reconstruction of the coherence matrix; Steps 2-6: Post-processing of physical constraints; For the reconstructed Perform eigenvalue testing, remove negative eigenvalues, and adjust the matrix trace to equal it. This yields the final valid coherence matrix. .

[0015] Preferably, step 3 specifically comprises: Constructing a signal enhancement neural network Its input layer receives two parts: (1) Original observation signal Spectral characteristics; (2) Coherence matrix ; Will Flattened into a vector, it serves as a conditional prior in the network's input layer. Channel splicing can be performed, or fusion can be achieved through an attention mechanism in the intermediate hidden layer.

[0016] Preferably, step 4 specifically comprises: The fused features are input into a trained neural network. After undergoing multiple nonlinear transformations, the trained neural network outputs a time-frequency mask of the unknown target sound source or directly estimates the time-frequency components of the target sound source. The time-frequency components are converted into time-domain signals to complete the extraction.

[0017] Preferably, the trained neural network is a U-Net or a Transformer.

[0018] Preferably, the forgetting factor The range of values ​​is to .

[0019] Preferably, the forgetting factor Values .

[0020] Preferably, the spectral feature is an amplitude spectrum or a phase spectrum.

[0021] Preferably, the Set as to aisle.

[0022] The beneficial effects of this invention are as follows: 1. Adaptive unknown sound source detection: By using the covariance residual model, the spatial coherence characteristics of unknown sound sources can be automatically derived from the observation data without the need for manual preset of unknown sound source parameters, thus realizing the automatic perception of new sound sources.

[0023] 2. Enhancement effect of physical guidance: Using the coherence matrix calculated by the physical model as a priori input to the neural network provides the network with clear spatial direction information, which makes up for the generalization defect of the pure data-driven model in unknown scenarios and significantly improves the signal-to-noise ratio and separation.

[0024] 3. Strong algorithm robustness: The solution process based on matrix decomposition and iterative optimization is introduced, which has strong robustness to noisy environments, and the post-processing step (negative eigenvalue correction) ensures the physical validity of the coherence matrix.

[0025] 4. High architectural flexibility: The neural network part is not limited by a specific structure, and only needs to support conditional input, making it easy to replace different network backbones according to actual application scenarios (such as real-time requirements and computing resources). Detailed Implementation

[0026] The present invention will be further described below with reference to embodiments.

[0027] To address the problems of difficulty in automatically separating unknown sound sources and the lack of explicit spatial prior guidance in existing technologies, this invention provides a microphone array sound source signal extraction method based on covariance matrix residual prior guidance. By constructing a covariance matrix residual model, the coherence matrix of the unknown sound source is automatically solved from the observed signal, and this matrix is ​​embedded into the neural network as a spatial prior, achieving blind / semi-blind sound source signal extraction without manual intervention.

[0028] This invention proposes a two-stage processing architecture. In the first stage, based on the covariance matrix of the array observation data, an eigenvalue decomposition and iterative optimization algorithm is used to construct a coherence matrix residual model of the unknown sound source. In the second stage, this coherence matrix is ​​used as a priori features and concatenated with the observed signal, then input into a neural network, which outputs the extracted target sound source signal. The specific steps include: 1. Acquisition of Observation Signals and Covariance Estimation: Acquire multi-channel time-domain signals from the microphone array, transform them to the Short-Time Fourier Transform (STFT) domain, and calculate the covariance matrix of the array's observation signals. .

[0029] 2. Solving for the coherence matrix of unknown sound sources: Based on the known coherence matrix information of noise or background sound sources, the coherence matrix of unknown sound sources is automatically solved using a residual model. The process includes: Obtain the identity matrix by performing eigenvalue decomposition on a known coherence matrix. ; The phase matrix can be directly estimated using the transformed observation covariance matrix. ; Based on the Csiszar I-divergence minimization criterion, the magnitude vector is updated iteratively. ; Reconstructed and post-processed to obtain (Removal of negative eigenvalues, trace normalization).

[0030] 3. Embedding prior information into a neural network: The coherence matrix obtained from the solution is... Converted into eigenvectors or tensors, and compared with the original observed signals. Splicing or fusion is performed in the frequency domain or feature layer.

[0031] 4. Signal Extraction and Output: The fused features are input into a trained neural network (such as U-Net, Transformer, etc.). The network outputs the extracted target unknown sound source signal based on spatial prior constraints. .

[0032] Example: This embodiment provides a method for extracting signals from unknown sound sources based on a microphone array, applicable to applications including... A microphone array system with multiple sensors. The specific execution steps are as follows: Step S1: Signal acquisition and preprocessing; take over The time-domain signal acquired by each microphone channel is framed and subjected to Short-Time Fourier Transform (STFT) to obtain the frequency-domain signal. Construct a vector from all channel signals. .

[0033] Step S2: Estimation of the observation covariance matrix; The covariance matrix of the array observation signal is calculated using a recursive formula. :

[0034] in, This is the forgetting factor, and its value range is usually [value range missing]. to .

[0035] Step S3: Unknown sound source coherence matrix Calculation; This step involves solving the covariance matrix residual model, and the specific sub-steps include: 1) Known background modeling: Obtain the coherence matrix of known background noise or known sound sources. (If there is no known sound source, it is a pure noise coherence matrix, which is usually modeled as a diffusion field model).

[0036] 2) Eigenvalue decomposition: For Perform eigenvalue decomposition to obtain ,in It is the identity matrix. It is a diagonal matrix of eigenvalues.

[0037] 3) Phase matrix Estimation: Calculate the transformation matrix Extract its phase angle as the phase matrix of the unknown source coherence matrix. :

[0038] 4) Amplitude vector Iterative optimization: Initialize the magnitude vector ,in ,and Updated via iterative formula until convergence:

[0039] 5) Coherence matrix reconstruction: Based on the model Preliminary reconstruction of the coherence matrix.

[0040] 6) Post-processing of physical constraints: This involves processing the reconstructed physical constraints. Perform eigenvalue testing, remove negative eigenvalues ​​(correct them to 0), and adjust the matrix trace to equal it. This yields the final valid coherence matrix. .

[0041] Step S4: Neural Network Construction and Prior Fusion; Constructing a signal enhancement neural network The network's input layer receives two parts: 1) Original observation signal Spectral characteristics (such as amplitude spectrum, phase spectrum); 2) The coherence matrix obtained in step S3 .

[0042] Will Flattened into a vector, it serves as a conditional prior in the network's input layer. Perform channel concatenation, or fuse the channels using an attention mechanism in the intermediate hidden layer.

[0043] Step S5: Signal extraction; The fused features are input into a neural network. After undergoing multiple nonlinear transformations, the network outputs a time-frequency mask of the unknown target sound source or a direct estimate of the time-frequency components of the target sound source. The time-frequency components are converted into time-domain signals to complete the extraction.

[0044] Example Parameter Settings: In specific experimental verification, the following parameter settings can be used to ensure the effect: Number of microphones Set as to Channels ensure the spatial resolution capability of the array.

[0045] Forgetting factor Set as To balance tracking speed and stability.

[0046] Number of iterations: magnitude vector The number of iterations is set to to Experience shows that convergence is possible within this range.

[0047] Neural network architecture: U-Net architecture can be used, which includes encoder-decoder modules and intermediate layer dimensionality adaptation. The input dimension.

[0048] Extended implementation method: Processing of multiple unknown sources: If there are multiple unknown sound sources in the environment, step S3 can be repeated to extract the coherence matrix of different unknown sound sources through multiple rounds of residual subtraction, and then send them into the network or into different branches of the same network for processing.

[0049] Mixed scenarios: When both exist simultaneously in the environment One known source and When there are unknown sources, first synthesize an equivalent background coherence matrix using known source information. Then substitute the values ​​into step S3 to calculate the unknown source. The logic remains unchanged.

[0050] Device configuration: This method can be deployed on microphone array hardware devices with memory, mobile terminals, or cloud servers, and is implemented by a processor executing computer-readable instructions.

Claims

1. A microphone array sound source signal extraction method based on covariance residual prior guidance, characterized in that, Includes the following steps: Step 1: Acquisition of observed signals and estimation of covariance; A multi-channel time domain signal of a microphone array is collected, converted to a short-time Fourier transform (STFT) domain, and a covariance matrix of an array observation signal is calculated ; Step 2: Solve for the coherence matrix of the unknown sound source; Based on known noise or background sound source coherence matrix information, the coherence matrix of unknown sound sources is automatically solved by a residual model , comprising: Eigenvalue decomposition of known coherence matrix to obtain identity matrix ; The phase matrix can be directly estimated using the transformed observation covariance matrix. ; Based on the Csiszar I-divergence minimization criterion, the amplitude vector is iteratively updated ; Reconstruction and post-processing are performed to obtain The processing includes removing negative eigenvalues and trace normalization. Step 3: Embed prior information into the neural network; The obtained coherence matrix is converted into eigenvectors or tensors, and the original observation signals are spliced or fused in the frequency domain or eigen layer; Step 4: Signal extraction and output; The fused features are input into the trained neural network, and the trained neural network outputs the extracted target unknown sound source signal according to the spatial prior constraint .

2. The method of claim 1, wherein, Step 1 specifically involves: Step 1-1: Signal Acquisition and Preprocessing; take over The time-domain signal acquired by each microphone channel is framed and subjected to a Short-Time Fourier Transform (STFT) to obtain the frequency-domain signal. , m =1…. M， Indicates the frequency index; constructs a vector from all channel signals. ; Step 1-2: Estimation of the observation covariance matrix; Recursive formulae are used to calculate the covariance matrix of the array observation signal : wherein is a forgetting factor.

3. The method of claim 2, wherein, Step 2 specifically involves: Step 2-1: Modeling the known background; Obtaining a coherence matrix of a known background noise or a known sound source ; if no known sound source, then a pure noise coherence matrix; Step 2-2: Eigenvalue decomposition; right Perform eigenvalue decomposition to obtain ,in It is the identity matrix. It is an eigenvalue diagonal matrix; Steps 2-3: Phase Matrix estimate; Calculate the transformation matrix Extract the phase angle as the phase matrix of the unknown source coherence matrix. : Step 2-4: Amplitude vector Iterative optimization; Initializing an amplitude vector wherein and ; updating by iteration formula until convergence: Steps 2-5: Coherence matrix reconstruction; According to the model Reconstructing the coherence matrix initially; Steps 2-6: Post-processing of physical constraints; The reconstructed Eigenvalue check is performed, negative eigenvalues are removed, and the trace of the matrix is adjusted to equal , to obtain the final legitimate coherence matrix .

4. The method of claim 3, wherein, Step 3 specifically involves: Constructing signal-enhanced neural networks whose input layer receives two parts: (1) Original observation signal Spectral characteristics; (2) Coherence matrix ; Will be flattened into vectors as conditional priors at the input layer of the network Channel concatenation is performed, or fusion through attention mechanisms at intermediate hidden layers.

5. The method of claim 4, wherein, Step 4 specifically involves: The fused features are input into a trained neural network. After undergoing multiple nonlinear transformations, the trained neural network outputs a time-frequency mask of the unknown target sound source or directly estimates the time-frequency components of the target sound source. ; The time-frequency components are converted into time-domain signals to complete the extraction.

6. The method for extracting microphone array sound source signals based on covariance residual prior guidance according to claim 5, characterized in that, The trained neural network is either U-Net or Transformer.

7. The method of claim 5, wherein, the forgetting factor ranging from to .

8. The method for extracting microphone array sound source signals based on covariance residual prior guidance according to claim 5, characterized in that, the forgetting factor is set to .

9. The method of claim 5, wherein, The spectral characteristics are amplitude spectrum or phase spectrum.

10. The method for extracting microphone array sound source signals based on covariance residual prior guidance according to claim 5, characterized in that, The configured to to passage.

Images