A multi-channel speech enhancement method and system based on adaptive spatial feature compression

Abstract

The application discloses a kind of multi-channel speech enhancement method and system based on adaptive spatial feature compression, belong to speech enhancement technical field.It includes: using lightweight steering vector estimation network to generate steering vector in real time, combined with linear constrained minimum variance algorithm to obtain preliminary enhanced single-channel signal;With the complex spectrum of the enhanced signal as a query, the complex spectrum of the original multi-channel signal as a key and a value, generate fixed-dimension spatial context feature map through multi-head spatial attention compression layer;Constructing a back-end auditory perception neural network to process the spatial context feature map, and designing a dynamic auditory perception cost function based on the SNR estimation subnetwork to guide the network training;A global width multiplier is introduced for the back-end network to achieve scalable computing power.The application effectively solves the problems of traditional methods, such as dependence on sound source distance, multi-channel input calculation expansion, and cost function mismatch with auditory perception, etc., and has the advantages of strong universality, efficient calculation and adaptive optimization of auditory perception.

Description

Technical Field

[0001] This invention belongs to the field of speech enhancement technology, and in particular relates to a multi-channel speech enhancement method and system based on adaptive spatial feature compression. It is especially suitable for solving technical problems such as the dependence of traditional methods on the distance of the sound source, the expansion of the computational load of multi-channel input, and the mismatch between the cost function and the auditory perception. Background Technology

[0002] In practical voice communication and voice interaction scenarios, environmental noise severely interferes with voice quality and intelligibility. Multi-channel voice enhancement using microphone arrays can leverage spatial information to create directionality, effectively suppressing interference noise. In traditional methods, Linear Constrained Minimum Variance (LCMV) beamforming is a commonly used spatial filtering technique, but its performance is highly dependent on accurate steering vectors. Typically, steering vectors need to be obtained by offline measurement of the transfer function from the sound source to the microphone. This not only limits the application to the "near field" and the assumption of a fixed sound source position, but also makes it extremely sensitive to environmental changes. Once the microphone or sound source position changes, the preset steering vector becomes invalid, leading to a sharp decline in enhancement performance.

[0003] In recent years, deep neural networks have been introduced into the field of speech enhancement, and their powerful nonlinear mapping capabilities can effectively handle non-stationary noise. However, when constructing a cascaded system of beamforming and neural networks, a common practice is to stack the beamforming output along with the spectra of all the original channels as network input. As the number of microphones increases, the number of input feature channels expands linearly, causing an explosion in network computation. Furthermore, the stacked features are disordered, making it difficult for the network to directly utilize the spatial structure information, and requiring a large amount of computational power to implicitly learn spatial relationships.

[0004] Furthermore, the mean squared error (MSE) cost function commonly used in network training does not correspond to human subjective auditory perception. Tiny errors in subtle speech components crucial to hearing can easily be buried in the averaging of large, insensible regions, resulting in residual noise or speech impairments still perceptible after noise reduction. While some methods introduce auditory perception cost functions with fixed parameters, their fixed weights cannot adapt to complex and varied noise environments, lacking true adaptability.

[0005] Therefore, there is an urgent need for a multi-channel speech enhancement method and system solution based on adaptive spatial feature compression. Summary of the Invention

[0006] The main objective of this invention is to provide a multi-channel speech enhancement method and system based on adaptive spatial feature compression, which aims to solve the technical problems of traditional methods such as dependence on sound source distance, expansion of multi-channel input computation, and mismatch between cost function and auditory perception.

[0007] To achieve the above objectives, this invention provides a multi-channel speech enhancement method based on adaptive spatial feature compression, comprising the following steps:

[0008] Step S1. For the received raw multi-channel speech signal, a lightweight steering vector estimation network is used to generate steering vectors in real time, and combined with a linearly constrained minimum variance algorithm, a preliminary enhanced single-channel signal is generated. ;

[0009] Step S2. Using the complex spectrum of the single-channel enhanced signal as a query and the complex spectrum of the original multi-channel speech signal as a key and value, the signal is processed through a multi-head spatial attention compression layer to generate a fixed-dimensional spatial context feature map.

[0010] Step S3. Construct a back-end auditory perception neural network and input the spatial context feature map into the back-end auditory perception neural network; and design a dynamic auditory perception cost function for training the back-end auditory perception neural network, the dynamic auditory perception cost function being based on the dynamic weighted amplitude spectrum error of the perception weight coefficients output in real time by a signal-to-noise ratio estimation subnetwork;

[0011] Step S4. Introduce a global width multiplier for the backend auditory perception neural network. By adjusting the value of the global width multiplier, uniformly scale the number of channels in each layer of the network to adapt to deployment platforms with different computing power.

[0012] Step S5. Perform end-to-end joint training of the lightweight guided vector estimation network, the multi-head spatial attention compression layer, the signal-to-noise ratio estimation sub-network, and the back-end auditory perception neural network. The optimized model receives multi-channel noisy speech and outputs an enhanced clean single-channel speech signal.

[0013] Furthermore, step S1 specifically includes:

[0014] The received raw multi-channel speech signal is subjected to a short-time Fourier transform to obtain a time-frequency domain signal, and normalized phase difference features are extracted from all channels of the time-frequency domain signal. The normalized phase difference feature The lightweight steering vector estimation network, consisting of two fully connected layers and an intermediate ELU activation function, is input to generate a steering vector estimate for that time-frequency point in real time. The output dimension is equal to the number of microphones M; the constraint matrix is ​​updated online by the estimated steering vector of the current frame. Combined with the covariance matrix calculated from the input signal The expected response vector consisting of all 1s Calculate the filter weight vector for the linearly constrained minimum variance beamforming. : The time-frequency domain signal is weighted with the filter weight vector, and then restored to the time domain via inverse short-time Fourier transform to obtain the initially enhanced single-channel enhanced signal. .

[0015] Furthermore, the multi-head spatial attention compression layer described in step S2 is used to compress the observation results of any number M microphones into a fixed number K spatial context vectors, where K is a preset constant between 4 and 16, independent of the number of microphones M.

[0016] Furthermore, step S2 specifically includes: obtaining the enhanced signal in step S1. The complex spectrum is used as the query Q;

[0017] Complex spectrum of all original channels As a key and "value" For the h-th attention head, a learnable linear projection matrix is ​​used. and Map queries and keys to a common space and calculate relevance scores. ,in The projected dimension; the correlation score is used to evaluate the linearly projected matrix. Mapped value Perform a weighted summation to generate the spatial context vector of the head. H attention heads (H=K) are used in parallel, and the outputs are concatenated and fused through a linear layer to generate a total spatial context feature map C with a fixed dimension. It has absolutely nothing to do with M.

[0018] Furthermore, the design of the dynamic auditory perception cost function in step S3 specifically includes: introducing a signal-to-noise ratio estimation subnetwork consisting of several convolutional layers, with the beamforming enhancement signal as the input. The amplitude spectrum and the spatial context feature map C compressed in step S2 are output as a real-time signal-to-noise ratio estimation map. ; through a monotonically decreasing differentiable function The signal-to-noise ratio estimation map is mapped to perceptual weight coefficients. : ,in and For learnable parameters; the amplitude spectrum loss function is constructed as follows ;The mean square error of the complex spectrum and the dynamically weighted amplitude spectrum error By combining proportionally, the overall cost function is obtained. .

[0019] Furthermore, the global width multiplier ω mentioned in step S4 has a value range of (0,1]. The number of channels in each layer of the network is obtained by multiplying the basic number of channels by ω and rounding it down. When ω=1, the number of basic convolutional channels in the five layers of the encoding module is {16, 32, 64, 128, 256}, which is the maximum computing power version. When ω takes a value less than 1, the number of channels is reduced proportionally, the number of hidden nodes of GRU is also reduced proportionally, while the convolutional kernel size, stride, number of layers, and number of self-attention heads of the entire network remain unchanged.

[0020] Furthermore, step S5 specifically includes: during the training phase, unifying all parameters of the lightweight guided vector estimation network, the attention projection matrix in the multi-head spatial attention compression layer, the signal-to-noise ratio estimation sub-network, and the back-end auditory perception neural network into a single computation graph, using the same composite cost function. Backpropagation and gradient updates are performed. During the inference phase, multi-channel noisy speech segments of arbitrary length are received and sequentially passed through adaptive beamforming, spatial feature compression, and a scalable enhancement network, ultimately outputting the real part of the complex spectrum of clean speech. and the virtual part The enhanced speech signal in the time domain was reconstructed using the inverse short-time Fourier transform and the overlap-addition method. .

[0021] Furthermore, this invention also proposes a multi-channel speech enhancement system based on adaptive spatial feature compression to implement any of the above-mentioned methods, comprising: a general adaptive beamforming module for generating steering vectors in real time using a lightweight steering vector estimation network and combining it with a linearly constrained minimum variance algorithm to generate a preliminarily enhanced single-channel enhanced signal from the received original multi-channel speech signal; a multi-head spatial attention compression module for using the complex spectrum of the single-channel enhanced signal as a query and the complex spectrum of the original multi-channel speech signal as a key and value, and generating a fixed-dimensional spatial context feature map through a multi-head attention mechanism; an auditory perception neural network module for receiving the spatial context feature map and outputting the real and imaginary parts of the complex spectrum of clean speech, wherein the training cost function includes the amplitude spectrum error dynamically weighted based on the perceptual weight coefficients output in real time by the signal-to-noise ratio estimation sub-network, and the network width can be uniformly scaled by a global width multiplier to adapt to different computing power platforms; and an end-to-end training and output module for jointly training all network modules and outputting the final enhanced clean single-channel speech signal during inference. Attached Figure Description

[0022] To more clearly illustrate the specific embodiments of the present invention, the accompanying drawings used in the description of the specific embodiments will be briefly introduced below.

[0023] Figure 1This is a flowchart of the overall technical solution of a multi-channel speech enhancement method and system based on adaptive spatial feature compression according to the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] Reference Figure 1 , Figure 1 This is a flowchart of the overall technical solution of the present invention. The information flow starts from the adaptive beamforming at the front end, passes through the spatial feature compression in the middle, enters the scalable neural network at the back end, and finally completes the enhancement and output under the guidance of the dynamic auditory perception cost function. The dashed arrows in the figure represent the backpropagation path in the training phase, showing the overall picture of the end-to-end joint optimization of the system. Below, this paper follows... Figure 1 The information flow follows a main thread, gradually introducing each technical aspect.

[0026] This embodiment provides a multi-channel speech enhancement method and system based on adaptive spatial feature compression, including the following steps:

[0027] S1. Universal Adaptive Beamforming

[0028] This invention employs the mathematical framework of Linear Constrained Minimum Variance (LCMV) beamforming, but focuses on the core steering vector. The generation is handled in real-time by a lightweight steering vector estimation network. This network is extremely streamlined, consisting of only two fully connected layers with an ELU activation function in between. The network's input is not the original signal, but rather normalized phase difference features extracted from the short-time Fourier transform results of all M channels. For any pair of microphones and At time and frequency points phase difference It is entirely determined by the path difference between the sound source and the two microphones, and is therefore the essential physical quantity for describing spatial orientation, and is independent of the signal amplitude.

[0029] This invention concatenates the phase differences of all microphone pairs into a single vector, which is then fed into the network. The network directly outputs the steering vector estimate for that time-frequency point. .

[0030] Here, MLP represents a multilayer perceptron, whose output dimension is equal to the number of microphones M. Thus, an online-updating constraint matrix is ​​obtained. It consists of the estimated steering vector of the current frame. Subsequent calculation of the optimal weight vector for LCMV follows the conventional method, using the input signal to calculate the covariance matrix. And then according to and expected response vector Calculate the filter weight vector using (all-1 vector) : .

[0031] Ultimately, the initially enhanced single-channel signal Depend on The time domain is restored by inverse short-time Fourier transform.

[0032] This step trains the network to learn the mapping from phase difference to the optimal steering vector. It no longer needs to know whether the sound source is near or far, nor does it require any offline calibration. Given paired noisy-clean speech data, the network can learn autonomously, applicable to all scenarios, including fixed and moving sound sources. Furthermore, this steering vector is no longer simply a "physically distortion-free constraint," but is optimized end-to-end throughout the entire enhancement chain. It may be trained to be a "perceptually optimal" constraint, slightly sacrificing physical fidelity in exchange for greater noise reduction potential and better sound quality from the backend neural network. The device requires no calibration after leaving the factory and is ready to use immediately, greatly improving product convenience.

[0033] S2. Multi-head adaptive spatial feature compression

[0034] This invention designs a multi-head spatial attention compression layer, which compresses the observations from any number of microphones into a fixed number of K refined "spatial context vectors." This solves the problem that as the number of microphones M increases, the number of input feature channels 2(M+1) expands linearly, leading to an explosion in network computation. It also addresses the issue that stacked channels are unordered for the network, meaning the network doesn't know which physical location a channel corresponds to, requiring significant computational effort to implicitly learn spatial relationships. K is a small constant, typically between 4 and 16, and is independent of the number of microphones.

[0035] The multi-head spatial attention compression layer enhances the beamforming signal obtained in step S1. The complex spectrum is used as a "query" Q, which carries information about "what the sound source is saying and its general characteristics." Simultaneously, the complex spectra of all original channels are... As a "key" and "value" For the h-th attention head, this invention maps the query and the key of each channel to a common space through a learnable linear transformation and calculates the relevance score between them: .

[0036] in, and It is the linear projection matrix of the h-th head. This is the projected dimension, used for normalization. Score It is a scalar or matrix representing the correlation between the information captured by the m-th microphone and the content of the sound source at the current time and frequency point.

[0037] Next, the present invention uses the same attention weights to apply the same linear transformation to the same area. Mapped channel values Perform a weighted summation to generate the spatial context vector of the head. This invention uses H such attention heads in parallel (H=K), each head learning... Since they are all different, they will spontaneously focus on different patterns in space—for example, some heads focus on the direct path of the sound source, some on early reflections, and others may be responsible for suppressing strong interference in a fixed direction. Finally, the outputs of all the heads are concatenated and fused through a linear layer to obtain a total spatial context feature map C, with a fixed dimension. It has absolutely nothing to do with M.

[0038] This step decouples computing power from the number of channels, meaning that the feature map size processed by the backend network remains consistent regardless of whether there are 2 or 20 microphones, greatly improving the system's scalability. From an information processing perspective, guided spatial information fusion is achieved. Instead of feeding raw data to the network indiscriminately, the sound source content is used as a "searchlight" to actively retrieve relevant acoustic cues from various directions and organize them into a structured, concise report. This allows the backend network to utilize spatial information more directly and efficiently, significantly reducing the learning difficulty.

[0039] S3. Dynamic auditory perception cost function based on real-time signal-to-noise ratio estimation

[0040] The main drawback of mean squared error (MSE) as a cost function is that its numerical value often contradicts human subjective hearing. Tiny errors in subtle speech components that are crucial to hearing, such as weak fricatives and codas, may be submerged in the averaging of a large number of imperceptible regions. Although perceptual cost functions such as weighted Euclidean distance (WE) with fixed parameters have improved this, the fixed weights cannot adapt to varying noise environments.

[0041] To endow the system with true adaptive auditory perception capabilities, this invention abandons the use of any pre-weighted cost function and instead introduces a signal-to-noise ratio (SNR) estimation subnetwork, allowing the network to determine the "auditory importance" of each time-frequency point. This subnetwork is extremely lightweight, consisting of several convolutional layers. Its input comes from two information sources: first, the amplitude spectrum of the beamforming enhancement signal Y, which provides a coarse estimate of the current signal envelope; and second, the compressed spatial context features C, which implicitly contain information about the spatial distribution of noise. The output of the subnetwork is a real-time SNR estimation graph with the same dimension as the input spectrum. .

[0042] Subsequently, this invention utilizes a monotonically decreasing differentiable function The signal-to-noise ratio estimate is mapped to the perceptual weight coefficient. : .

[0043] Here, α and β are learnable parameters that control the shape of the mapping curve. The design of this function ensures that when the estimated signal-to-noise ratio is very low at a certain time-frequency point, i.e., noise dominates, When the signal-to-noise ratio is very small, the γ value will approach a pre-set maximum value, such as 2.0, which means that the error at that point will be significantly penalized, forcing the network to strongly reduce noise. Conversely, when the signal-to-noise ratio is very high (speech is dominant), the γ value will approach 0 or 1, and the network will only need to maintain fidelity and avoid damaging the speech.

[0044] Finally, the amplitude spectrum loss function is constructed as follows: .

[0045] The overall cost function of the network Still due to the mean square error of the complex spectrum And the above dynamically weighted amplitude spectrum error proportionally Composed of: .

[0046] This design significantly improves the simulation of auditory masking effects. It does not rely on any pre-assumed energy masking rules, but instead uses an SNR estimator trained end-to-end with the main network to dynamically and adaptively generate penalty coefficients that are most beneficial to perceived quality for each frame and each frequency component. This results in not only further improvements in objective metrics but also in subjective listening experience.

[0047] S4. Lightweight network architecture with scalable computing power

[0048] A good algorithm ultimately needs to be deployed on a wide variety of devices, from cloud servers with abundant computing power to wireless earbuds with stringent requirements for power consumption and real-time performance. Fixed architectures cannot meet such universal needs. Therefore, this invention designs a unified yet computationally adjustable scheme for the backend auditory perception neural network, introducing a global width multiplier ω to the entire encoding, decoding, and temporal modeling modules, with a value range of (0,1). The number of channels in each layer of the network is obtained by multiplying the "base number of channels" by this ω and rounding down. For example, the base number of convolutional channels in the five layers of the encoding module can be designed as {16, 32, 64, 128, 256}. When ω=1, it is the maximum computing power version, pursuing ultimate performance; when ω=0.25, the number of channels is reduced to {4, 8, 16, 32, 64}, GRU The number of hidden nodes is also reduced proportionally, resulting in an ultra-lightweight version that can run smoothly on embedded devices. The entire network structure—kernel size, stride, number of layers, and number of self-attention heads—remains unchanged. This allows developers to freely balance model size, inference speed, and enhancement effects by simply changing a single configuration parameter, generating a series of models adapted to different platforms such as mobile phones, smart speakers, hearing aids, and in-vehicle systems.

[0049] S5. End-to-end joint training and speech output

[0050] During the training phase, this invention integrates all parameters of the steering vector estimation network, the attention projection matrix in the spatial compression layer, the SNR estimation sub-network, and the backend main network into a single computation graph, using the same composite cost function. Backpropagation and gradient updates are performed. The optimizer chosen is the convergent and stable AdamW, with an initial learning rate of [value missing]. And an annealing strategy is used on the validation set.

[0051] Once the model training is complete and the actual deployment (inference phase) begins, the system's workflow is as follows: Figure 1 Completely identical. A noisy, multi-channel speech segment of arbitrary length is input, sequentially passing through adaptive beamforming, spatial feature compression, and a scalable enhancement network, ultimately outputting the real part of the complex spectrum of clean speech. and the virtual part By utilizing the inverse short-time Fourier transform (ISTFT) that is strictly matched to the front-end analysis window and the overlapping addition method, this invention reconstructs the enhanced speech signal in the time domain. Throughout the entire process, there was no human intervention or manual parameter adjustment. All modules worked closely together to deliver a clean, natural, and highly intelligible auditory experience to the listener.

[0052] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A multi-channel speech enhancement method based on adaptive spatial feature compression, characterized in that... Includes the following steps: Step S1. For the received raw multi-channel speech signal, a lightweight steering vector estimation network is used to generate steering vectors in real time, and combined with a linearly constrained minimum variance algorithm, a preliminary enhanced single-channel signal is generated. ; Step S2. Using the complex spectrum of the single-channel enhanced signal as a query and the complex spectrum of the original multi-channel speech signal as a key and value, the signal is processed through a multi-head spatial attention compression layer to generate a fixed-dimensional spatial context feature map. Step S3. Construct a back-end auditory perception neural network and input the spatial context feature map into the back-end auditory perception neural network; and design a dynamic auditory perception cost function for training the back-end auditory perception neural network, the dynamic auditory perception cost function being based on the dynamic weighted amplitude spectrum error of the perception weight coefficients output in real time by a signal-to-noise ratio estimation subnetwork; Step S4. Introduce a global width multiplier for the backend auditory perception neural network. By adjusting the value of the global width multiplier, uniformly scale the number of channels in each layer of the network to adapt to deployment platforms with different computing power. Step S5. Perform end-to-end joint training of the lightweight guided vector estimation network, the multi-head spatial attention compression layer, the signal-to-noise ratio estimation sub-network, and the back-end auditory perception neural network. The optimized model receives multi-channel noisy speech and outputs an enhanced clean single-channel speech signal.

2. The method according to claim 1, characterized in that, Step S1 specifically includes: The received raw multi-channel speech signal is subjected to a short-time Fourier transform to obtain a time-frequency domain signal, and normalized phase difference features are extracted from all channels of the time-frequency domain signal. ; The normalized phase difference feature The lightweight steering vector estimation network, consisting of two fully connected layers and an intermediate ELU activation function, is input to generate a steering vector estimate for that time-frequency point in real time. The output dimension is equal to the number of microphones, M. The constraint matrix is ​​updated online by constructing the estimated steering vector of the current frame. Combined with the covariance matrix calculated from the input signal The expected response vector consisting of all 1s Calculate the filter weight vector for the linearly constrained minimum variance beamforming. : ; The time-frequency domain signal is weighted with the filter weight vector, and then restored to the time domain via inverse short-time Fourier transform to obtain the initially enhanced single-channel enhanced signal. .

3. The method according to claim 1, characterized in that, The multi-head spatial attention compression layer described in step S2 is used to compress the observation results of any number M microphones into a fixed number K spatial context vectors, where K is a preset constant between 4 and 16, independent of the number of microphones M.

4. The method according to claim 3, characterized in that, Step S2 specifically includes: The enhanced signal is obtained in step S1. The complex spectrum is used as the query Q; Complex spectrum of all original channels As a key and "value" ; For the h-th attention head, a learnable linear projection matrix is ​​used. and Map queries and keys to a common space and calculate relevance scores. ,in The dimensions after projection; Use the correlation score to apply the linear projection matrix Mapped value Perform a weighted summation to generate the spatial context vector of the head. ; H attention heads are used in parallel (H=K), and the outputs are concatenated and fused through a linear layer to generate a total spatial context feature map C with a fixed dimension. It has absolutely nothing to do with M.

5. The method according to claim 1, characterized in that, The design of the dynamic auditory perception cost function in step S3 specifically includes: A signal-to-noise ratio estimation subnetwork consisting of several convolutional layers is introduced, with the beamforming enhancement signal as input. The amplitude spectrum and the spatial context feature map C compressed in step S2 are output as a real-time signal-to-noise ratio estimation map. ; Through a monotonically decreasing differentiable function The signal-to-noise ratio estimation map is mapped to perceptual weight coefficients. : ,in and These are learnable parameters; The amplitude spectrum loss function is constructed as follows: ; Mean square error of complex spectrum and the dynamically weighted amplitude spectrum error By combining proportionally, the overall cost function is obtained. .

6. The method according to claim 1, characterized in that, The global width multiplier ω mentioned in step S4 has a value range of (0,1]. The number of channels in each layer of the network is obtained by multiplying the basic number of channels by ω and rounding it down. When ω=1, the basic number of convolutional channels in the five layers of the encoding module is {16, 32, 64, 128, 256}, which is the maximum computing power version. When ω takes a value less than 1, the number of channels is reduced proportionally, the number of hidden nodes of GRU is also reduced proportionally, while the convolutional kernel size, stride, number of layers, and number of self-attention heads of the entire network remain unchanged.

7. The method according to claim 1, characterized in that, Step S5 specifically includes: During the training phase, all parameters of the lightweight steering vector estimation network, the attention projection matrix in the multi-head spatial attention compression layer, the signal-to-noise ratio estimation sub-network, and the back-end auditory perception neural network are integrated into a single computation graph and use the same composite cost function. Perform backpropagation and gradient update; During the inference phase, noisy multi-channel speech segments of arbitrary length are received and sequentially passed through adaptive beamforming, spatial feature compression, and a scalable enhancement network, ultimately outputting the real part of the complex spectrum of clean speech. and the virtual part The enhanced speech signal in the time domain was reconstructed using the inverse short-time Fourier transform and the overlap-addition method. .

8. A multi-channel speech enhancement system based on adaptive spatial feature compression, characterized in that, To implement the method according to any one of claims 1 to 7, comprising: A general adaptive beamforming module is used to generate steering vectors in real time using a lightweight steering vector estimation network, and combined with a linearly constrained minimum variance algorithm to generate a preliminarily enhanced single-channel enhanced signal from the received raw multi-channel speech signal. The multi-head spatial attention compression module is used to take the complex spectrum of the single-channel enhanced signal as a query and the complex spectrum of the original multi-channel speech signal as a key and value, and generate a fixed-dimensional spatial context feature map through a multi-head attention mechanism. The auditory perception neural network module is used to receive the spatial context feature map and output the real and imaginary parts of the complex spectrum of clean speech. The training cost function includes the amplitude spectrum error dynamically weighted by the perceptual weight coefficients output in real time based on the signal-to-noise ratio estimation subnetwork. The network width can be uniformly scaled by a global width multiplier to adapt to different computing power platforms. The end-to-end training and output module is used to jointly train all network modules and output the final enhanced clean single-channel speech signal during inference.

Images