Multi-microphone array beamforming signal enhancement method and device

Abstract

The invention relates to a multi-microphone array beam forming signal enhancement method and device, and the method comprises the steps: obtaining a multi-channel time domain signal collected by a multi-microphone array, and carrying out the time-frequency conversion, and obtaining a plurality of time-frequency domain signals; acquiring a spatial distribution vector of the multi-microphone array, and performing deformation compensation processing on the complex number of time-frequency domain signals to obtain sound field consistency signals; performing sound source direction feature extraction on the sound field consistency signal, and performing complex time-space attention beam forming network processing to obtain a beam forming signal; performing beam direction and gain parameter optimization on the beam forming signal through a preset adversarial environment simulator to obtain a dynamic frequency band enhanced signal; and performing inverse time-frequency conversion on the dynamic frequency band enhanced signal, and performing low-delay optimization to obtain a target quality voice output signal. According to the invention, signal deformation compensation can be carried out more accurately.

Description

Technical Field

[0001] The present invention relates to the technical field of signal enhancement, and in particular to a multi-microphone array beamforming signal enhancement method and device. Background Art

[0002] As a technology to improve the quality of speech signals and enhance speech recognition capabilities, the multi-microphone array beamforming signal enhancement method has been widely used in the fields of modern communications and speech processing. With the popularization of smart devices and voice interaction systems, how to accurately extract and enhance the target speech signal in a complex acoustic environment has become one of the research focuses. Existing signal enhancement methods usually rely on a single processing technology, such as a simple beamforming or noise suppression algorithm, while ignoring the spatial information of the multi-microphone array and its influence on the consistency of the sound field. The application of this single technology may lead to unsatisfactory signal enhancement effects, thereby affecting the performance of the speech recognition system. In addition, current methods often lack the ability to adapt to dynamic acoustic environments, and cannot effectively optimize the beam direction and gain parameters, so that ideal signal enhancement effects cannot be achieved in complex environments. The existence of these problems urgently requires a signal enhancement method that comprehensively considers the spatial distribution of multi-microphone arrays, dynamic sound field characteristics, and adaptability to complex environments to improve the quality of target speech signals and the robustness of the system. Summary of the invention

[0003] The main purpose of the present invention is to provide a multi-microphone array beamforming signal enhancement method and device, which can more accurately compensate for signal deformation and improve the quality of the sound field consistency signal.

[0004] To achieve the above object, the present invention provides a multi-microphone array beamforming signal enhancement method, comprising: Acquire a multi-channel time domain signal collected by a multi-microphone array, perform time-frequency conversion, and obtain a complex time-frequency domain signal; Acquire the spatial distribution vector of the multi-microphone array, perform deformation compensation processing on the complex time-frequency domain signal, and obtain a sound field consistency signal; Extracting the sound source direction feature of the sound field consistency signal, and performing complex spatiotemporal attention beamforming network processing to obtain a beamforming signal; Optimizing the beam direction and gain parameters of the beamforming signal by using a preset adversarial environment simulator to obtain a dynamic frequency band enhancement signal; The dynamic frequency band enhancement signal is subjected to inverse time-frequency conversion and low-delay optimization to obtain a target quality speech output signal.

[0005] Furthermore, the method of acquiring a multi-channel time domain signal collected by a multi-microphone array and performing time-frequency conversion to obtain a complex time-frequency domain signal includes: Performing endpoint detection and noise power estimation on the multi-channel time domain signal to obtain a preprocessed signal; The pre-processed signal is framed and windowed according to a preset frame length and frame shift parameter to obtain an initial complex frequency domain signal; Performing spectrum smoothing on the initial complex frequency domain signal to obtain a smoothed frequency domain signal; Performing nonlinear sub-band division on the smoothed frequency domain signal to obtain a sub-band signal; Performing geometric phase compensation on the sub-band signal to obtain a sub-band correction signal; The sub-band correction signal is subjected to spectral subtraction denoising based on a preset noise power spectrum, and is subjected to complex time-frequency domain feature representation conversion to obtain the complex time-frequency domain signal.

[0006] Furthermore, the acquiring of the spatial distribution vector of the multi-microphone array and the performing of deformation compensation processing on the complex time-frequency domain signal to obtain a sound field consistency signal includes: Acquiring the physical coordinates and orientation parameters of the multi-microphone array, performing spatial construction, and obtaining the spatial distribution vector; Performing spatial transfer function mapping calculation on the complex time-frequency domain signal according to the spatial distribution vector to obtain array acoustic characteristic mapping information; Performing singular value decomposition on the array acoustic characteristic mapping information, and performing phase and amplitude nonlinear correction to obtain a preliminary compensation signal; Calculating a cross-power spectral density matrix based on the preliminary compensation signal, and constructing a sound field geometric consistency constraint to obtain a sound field topology representation; Performing complex domain tensor decomposition on the acoustic field topology representation to obtain a deformation compensation coefficient; The phase and amplitude of the complex time-frequency domain signal are aligned according to the deformation compensation coefficient to obtain the sound field consistency signal.

[0007] Furthermore, the sound source direction feature extraction is performed on the sound field consistency signal, and a complex spatiotemporal attention beamforming network is performed to obtain a beamforming signal, including: Performing adaptive wavelet packet transform decomposition on the sound field consistency signal to obtain a multi-level sound field time-frequency representation; Performing complex domain spatial covariance calculation on the multi-level sound field time-frequency representation to obtain a frequency band phase difference spectrum; Performing high-order singular value decomposition on the frequency band phase difference spectrum to obtain an orthogonal projection structure; Constructing a hyperbolic direction estimation function according to the orthogonal projection structure, and performing a gradient polarization search to obtain a sound source direction vector; Phase-aligning the sound source direction vector with the sound field consistency signal to obtain an initial beam gain coefficient; A complex domain attention mask is constructed according to the initial beam gain coefficient, and channel weighting is performed on the sound field consistency signal to obtain a beamforming signal.

[0008] Furthermore, the optimizing the beam direction and gain parameters of the beamforming signal by using a preset adversarial environment simulator to obtain a dynamic frequency band enhancement signal includes: Based on the adversarial environment simulator, a variety of noise interference and reverberation condition simulations are performed on the beamforming signal to obtain diversified environmental interference samples; Performing gradient descent optimization calculation on the diverse environmental interference samples, and adjusting beam direction parameters and frequency band gain coefficients according to a preset signal-to-noise ratio objective function to obtain a parameter optimization matrix; Reconstructing the directionality of the beamforming signal according to the parameter optimization matrix to obtain a directionally enhanced signal; According to the directional enhancement signal, acoustic feature matching is performed with preset voice data to obtain corresponding target actual short-term voice data; Based on a preset meta-learning online calibration algorithm, feature extraction is performed on the target actual short-term speech data to obtain a target feature adaptation matrix; Performing a convolution operation on the target feature adaptation matrix and the direction enhancement signal to obtain a spectrum correction signal; The spectrum correction signal is subjected to cross-band consistency processing and phase band gain to obtain the dynamic band enhanced signal.

[0009] Furthermore, the simulating various noise interference and reverberation conditions of the beamforming signal based on the adversarial environment simulator to obtain diversified environmental interference samples includes: Performing time-frequency transformation on the beamforming signal through the adversarial environment simulator to obtain a signal matrix represented in the time-frequency domain; Constructing a three-dimensional acoustic propagation structure according to the signal matrix represented in the time-frequency domain, and setting a random reflection coefficient to obtain a spatial reverberation parameter set; Randomly extract various types of noise sources from the preset environmental noise library, and perform time domain stretching and frequency domain shifting to obtain a deformed noise source set; Position mapping is performed on the deformation noise source set according to a preset spatial directional distribution function to obtain a multi-directional noise distribution matrix; Calculating acoustic propagation delay and energy attenuation according to the multi-directional noise distribution matrix and the spatial reverberation parameter set to obtain a noise propagation characteristic vector; The time-frequency domain representation signal matrix is ​​subjected to reverberation processing based on the spatial reverberation parameter set, and interference superposition is performed with the noise propagation characteristic vector to obtain the diversified environmental interference samples.

[0010] Furthermore, the preset meta-learning online calibration algorithm is used to extract features from the target actual short-term speech data to obtain a target feature adaptation matrix, including: Performing short-time window framing and windowing processing on the target actual short-time speech data to obtain an overlapping segmented signal sequence; Calculating Mel-frequency cepstrum coefficients of the overlapping segmented signal sequence to obtain a segmented feature description vector; Constructing a meta-learning task sampling set for the segmented feature description vector to obtain a feature support set and a feature query set; Based on the feature support set, a meta-learning inner loop optimization function is constructed to obtain environment adaptation parameters; Performing MAML gradient update on the feature query set according to the environmental adaptation parameters to obtain task model parameters; Performing meta-learning outer loop optimization and online learning rate adjustment on the task model parameters to obtain a dynamic learning rate matrix; Performing prototype network mapping on the segmented feature description vector according to the dynamic learning rate matrix to obtain an environmental perception feature representation; Performing metric learning embedding space construction on the environmental perception feature representation to obtain a speech feature embedding matrix; The speech feature embedding matrix is ​​subjected to Bayesian credible interval constraints to obtain a target feature adaptation matrix.

[0011] Further, the performing inverse time-frequency conversion on the dynamic frequency band enhancement signal and performing low-delay optimization to obtain a target quality speech output signal includes: Performing overlap-add inverse transform processing on the dynamic frequency band enhancement signal to obtain an initial time domain signal; Performing inter-frame phase consistency compensation on the initial time domain signal to obtain a phase continuous intermediate signal; Dynamically adjust the buffer according to the phase-continuous intermediate signal to obtain a variable signal buffer; Performing nonlinear amplitude normalization on the variable signal buffer to obtain a dynamic range compression signal; Calculating a time domain transient feature vector according to the dynamic range compression signal, and performing time domain jitter elimination to obtain a jitter correction signal; Performing multi-stage cascade frequency band reconstruction on the jitter correction signal to obtain a frequency response equalization signal; Performing device pre-compensation processing on the frequency response equalization signal to obtain a device matching signal; Noise and artifact elimination is performed on the device matching signal according to a preset frame selective discarding algorithm to obtain the target quality speech output signal.

[0012] The present invention also provides a multi-microphone array beamforming signal enhancement device, which is applied to any of the multi-microphone array beamforming signal enhancement methods described above, comprising: An acquisition module, the acquisition module is used to acquire a multi-channel time domain signal acquired by a multi-microphone array, perform time-frequency conversion, and obtain a complex time-frequency domain signal; An analysis module, the analysis module is used to obtain the spatial distribution vector of the multi-microphone array, perform deformation compensation processing on the complex time-frequency domain signal, and obtain a sound field consistency signal; An association module, the association module is used to extract the sound source direction feature of the sound field consistency signal, and perform complex spatiotemporal attention beamforming network processing to obtain a beamforming signal; A processing module, the processing module is used to optimize the beam direction and gain parameters of the beamforming signal through a preset confrontation environment simulator to obtain a dynamic frequency band enhancement signal; A control module is used to perform inverse time-frequency conversion on the dynamic frequency band enhancement signal and perform low-delay optimization to obtain a target quality speech output signal.

[0013] The present invention provides a multi-microphone array beamforming signal enhancement method and device, which has the following beneficial effects: By performing time-frequency conversion on the time domain signal collected by the multi-microphone array and combining the spatial distribution vector information of the array, the deformation compensation of the signal can be performed more accurately, and the quality of the sound field consistency signal can be improved. This method can effectively improve the extraction accuracy of the target speech signal by extracting the directional characteristics of the sound source and applying the complex spatiotemporal attention beamforming network, avoiding the occurrence of signal attenuation or distortion in traditional signal enhancement methods. By introducing a preset adversarial environment simulator, the beam direction and gain parameters are optimized, which can adapt to the dynamic and complex acoustic environment, realize the adaptive adjustment of the beam direction and gain parameters, and further improve the signal clarity and the accuracy of speech recognition. Based on this technology, noise interference and echo effects can be effectively reduced, thereby ensuring that high-quality speech output signals can be maintained in different environments, reducing the sound quality degradation and speech distortion problems in traditional methods. In addition, the low-latency optimization design of the dynamic frequency band enhancement signal contributes to the smooth operation of the real-time voice interaction system and enhances the robustness and user experience of the system. By comprehensively considering the spatiotemporal information and dynamic sound field characteristics, this method can flexibly adapt to complex and changeable acoustic environments, and improve the overall performance and application effect of the multi-microphone array system. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a flow chart of a multi-microphone array beamforming signal enhancement method provided by the present invention; Figure 2 It is a structural diagram of a multi-microphone array beamforming signal enhancement device provided by the present invention.

[0015] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0017] The present invention is further described below in conjunction with the accompanying drawings and specific implementation methods.

[0018] Reference Figure 1 As shown, 1. A multi-microphone array beamforming signal enhancement method, comprising: Step S1: Acquire a multi-channel time domain signal collected by a multi-microphone array, perform time-frequency conversion, and obtain a complex time-frequency domain signal; Step S2: obtaining a spatial distribution vector of a multi-microphone array, performing deformation compensation processing on complex time-frequency domain signals, and obtaining a sound field consistency signal; Step S3: extracting the sound source direction feature of the sound field consistency signal, and performing complex spatiotemporal attention beamforming network processing to obtain a beamforming signal; Step S4: optimizing the beam direction and gain parameters of the beamforming signal through a preset adversarial environment simulator to obtain a dynamic frequency band enhancement signal; Step S5: Perform inverse time-frequency conversion on the dynamic frequency band enhancement signal and perform low-delay optimization to obtain a target quality speech output signal.

[0019] Based on the above steps, the detailed steps are as follows: Step S1: The first step of multi-microphone array beamforming is to obtain the original acoustic signal and convert it into a representation suitable for processing. The microphone array consists of multiple microphones arranged in a specific geometry, and each microphone captures the time domain representation of the sound wave. These time domain signals contain the sound source information as well as background noise and reverberation. In order to effectively process these signals, it is necessary to convert them to the time-frequency domain. The time-frequency conversion usually uses the short-time Fourier transform (STFT), which divides the time domain signal into short time frames and applies the Fourier transform to each frame. In specific implementation, a windowing function (such as a Hamming window or a Hanning window) is applied to the time domain signal of each microphone channel to reduce spectral leakage. After the window function is applied, a fast Fourier transform (FFT) is performed to generate a complex time-frequency domain representation. This complex representation contains amplitude and phase information, which is crucial for subsequent beamforming. The result of the time-frequency conversion is a three-dimensional tensor with dimensions [number of microphones × number of time frames × number of frequency points], and each element is a complex number. The choice of frame length and frame shift step size affects the trade-off between time resolution and frequency resolution and needs to be adjusted according to the application scenario. Typical speech processing may use a frame length of 20-30ms and a frame shift of 10-15ms.

[0020] Step S2: The spatial distribution vector of the multi-microphone array describes the relative position relationship of each microphone in three-dimensional space. This spatial distribution information is crucial to understanding how sound waves reach different parts of the array. In practical applications, microphone arrays may have problems such as shape deformation, installation errors, or inconsistent microphone characteristics, which will affect the consistent capture of the sound field. The deformation compensation process estimates the calibration coefficients of each microphone through the spatial distribution vector to compensate for these errors. The compensation process applies phase and amplitude correction to the complex time-frequency domain signal based on the acoustic propagation model. The correction factor usually includes a frequency-dependent complex coefficient matrix that is used to adjust the phase and amplitude relationship between each microphone channel. Deformation compensation may also involve microphone sensitivity difference calibration and frequency response equalization. The sound field consistency signal obtained after processing should ideally be: if the sound source comes from a specific direction, the signals of all microphones should be completely coherent at the corresponding phase delay. This consistency provides a key basis for sound source direction estimation and beamforming in the next step.

[0021] Step S3: Use the calibrated sound field consistency signal to locate and enhance the sound source. The sound source direction feature extraction process is based on the phase information analysis of the sound field consistency signal, and the possible direction of the sound source is calculated by the phase difference between the microphone signals. Common methods include generalized cross correlation (GCC), multi-channel cross power spectral density (CPSD) analysis, or a directional feature extraction network based on deep learning. The extracted directional features are usually expressed as probability distributions or feature vectors of azimuth and elevation angles. The complex spatiotemporal attention beamforming network is an innovative signal processing architecture that combines the advantages of traditional beamforming technology and deep learning. The network applies an attention mechanism to both the time dimension and the spatial dimension, and can dynamically adjust the attention to different time-frequency points and spatial directions. The network processing includes the calculation of a complex masking matrix, which assigns different complex weights to each time-frequency point according to the sound source direction and time-frequency characteristics. By applying the masking matrix to the sound field consistency signal, the signal from the target direction can be enhanced and interference and noise from other directions can be suppressed. The quality of the beamforming signal depends on the accuracy of the directional features and the degree of optimization of the network parameters.

[0022] Step S4: The adversarial environment simulator is a specially designed module for simulating various acoustic interference situations that may occur in real-world environments, such as reverberation, non-stationary noise, and competing speakers. The simulator adopts an adversarial training framework, in which the generator is responsible for creating various complex acoustic scenes, and the discriminator evaluates the performance of the beamforming algorithm in these scenes. Based on this adversarial mechanism, the system can discover potential weaknesses in the beamforming process. The beam direction optimization process fine-tunes the width and direction of the main lobe of the beam to maximize the suppression of interference sources while maintaining the target signal enhancement. The gain parameter optimization adjusts the gain distribution of different frequency bands to balance the relationship between signal amplification and noise suppression. The optimization process uses gradient descent or evolutionary algorithms to iteratively optimize based on objective indicators such as signal-to-noise ratio and speech clarity. The dynamic band enhancement signal is the result of optimization, which has the characteristics of adaptive gain adjustment for different frequency components, and can improve its intelligibility while maintaining the naturalness of the speech. This step enables the beamforming technology to shift from static optimization to dynamic adaptation, greatly improving the robustness of the system in a changing environment.

[0023] Step S5: Inverse time-frequency conversion uses inverse short-time Fourier transform (ISTFT) to convert the complex time-frequency domain signal back to the time domain. The conversion process includes performing an inverse Fourier transform on each time-frequency frame, and then using the overlap-add method to synthesize a continuous time domain signal. To ensure signal quality, spectrum smoothing technology is usually used to reduce artifacts caused by phase discontinuity. Low-latency optimization is a key requirement for real-time voice interaction systems, and the traditional STFT-ISTFT framework may introduce large processing delays. Low-latency optimization strategies include: using shorter analysis windows and frame shift steps, implementing a streaming processing architecture to reduce buffering delays, and using frequency domain prediction technology to estimate the content of subsequent frames in advance. In addition, computing resource optimization is also an important aspect of reducing system latency, including algorithm parallelization, GPU acceleration and other technical means. The target quality speech signal outputted in the end maintains good speech clarity and naturalness while meeting the low latency requirements. Quality assessment is usually combined with subjective listening tests (such as MOS scores) and objective indicators (such as PESQ, STOI) for comprehensive judgment. This step enables the entire system to meet the needs of application scenarios with high real-time requirements, such as teleconferencing and smart speakers.

[0024] Step S6: As the final step of the entire multi-microphone array beamforming signal enhancement method, the output signal is finally polished and quality verified. Post-processing includes noise suppression residual artifact processing, dynamic range compression and dereverberation enhancement. Artifact processing eliminates music noise and spectral discontinuity that may be introduced in the beamforming process through spectrum smoothing and transient detection technology. Dynamic range compression adjusts the dynamic characteristics of the signal according to the application scenario so that the output maintains consistent audibility on different playback devices. De-dereverberation enhancement technology supplements the shortcomings of beamforming in reverberation suppression, and further improves the clarity of speech through blind room impulse response estimation and inverse filtering. A multi-dimensional evaluation system is established in the quality assessment link, including two categories: objective evaluation and subjective evaluation. Objective evaluation uses standardized indicators such as PESQ, STOI, SNR, etc., and subjective evaluation uses methods such as MUSHRA test or ABX comparison, and invites professional listeners and ordinary users to participate in the evaluation. The evaluation results are fed back to the previous steps to form a closed-loop optimization mechanism. This step ensures that the final output voice signal can meet the quality standards expected by users in different application scenarios, and provides a scientific basis for the continuous optimization of the system.

[0025] The present invention provides a multi-microphone array beamforming signal enhancement method, which can more accurately compensate for signal deformation by performing time-frequency conversion on the time domain signal collected by the multi-microphone array and combining the spatial distribution vector information of the array, thereby improving the quality of the sound field consistency signal. This method can effectively improve the extraction accuracy of the target voice signal by extracting the directional characteristics of the sound source and applying the complex spatiotemporal attention beamforming network, avoiding the occurrence of signal attenuation or distortion in the traditional signal enhancement method. By introducing a preset adversarial environment simulator, the beam direction and gain parameters are optimized, which can adapt to the dynamic and complex acoustic environment, realize the adaptive adjustment of the beam direction and gain parameters, and further improve the signal clarity and the accuracy of speech recognition. Based on this technology, noise interference and echo effects can be effectively reduced, thereby ensuring that high-quality voice output signals can be maintained in different environments, reducing the sound quality degradation and voice distortion problems in traditional methods. In addition, the low-delay optimization design of the dynamic frequency band enhancement signal contributes to the smooth operation of the real-time voice interaction system and enhances the robustness and user experience of the system. By comprehensively considering the spatiotemporal information and dynamic sound field characteristics, this method can flexibly adapt to complex and changeable acoustic environments, thereby improving the overall performance and application effect of the multi-microphone array system.

[0026] In one embodiment, a multi-channel time domain signal collected by a multi-microphone array is obtained, and a time-frequency conversion is performed to obtain a complex time-frequency domain signal, including: After acquiring the multi-channel time domain signal collected by the multi-microphone array, endpoint detection and noise power estimation are performed on it to obtain the preprocessed signal. Endpoint detection refers to identifying the starting and ending points of the valid speech segment in the speech signal to distinguish the valid speech from the background noise. Endpoint detection is usually based on features such as short-time energy and zero-crossing rate, and determines whether the current frame contains valid speech by setting an appropriate threshold. Noise power estimation refers to estimating the power spectral density of background noise during the period of no speech activity, providing a reference for subsequent signal enhancement. Noise power estimation uses the minimum statistical method or the method based on speech activity detection to update the noise power during the period of no speech activity. The preprocessed signal is the signal after endpoint detection and noise power estimation. The signal retains the valid speech part and is accompanied by the noise power estimation result.

[0027] The pre-processed signal is framed and windowed according to the preset frame length and frame shift parameters to obtain the initial complex frequency domain signal. Framing is to divide the long speech signal into short time frames, and the signal in each frame has relative stability. The preset frame length is usually 20-30 milliseconds (corresponding to a signal with a sampling rate of 16kHz, and a frame length of 320-480 sampling points), and the frame shift is half of the frame length or less (such as 10-15 milliseconds, corresponding to 160-240 sampling points) to ensure that there is a certain degree of overlap between adjacent frames. Windowing is to multiply each frame signal by a window function (such as Hamming window, Hanning window, etc.) to reduce spectrum leakage. After framing and windowing, each frame signal is subjected to a fast Fourier transform (FFT) and converted to the frequency domain to obtain the initial complex frequency domain signal. The initial complex frequency domain signal contains amplitude information and phase information, which provides a basis for subsequent spectrum processing.

[0028] The initial complex frequency domain signal is spectrally smoothed to obtain a smoothed frequency domain signal. Spectral smoothing is to reduce the volatility of the spectrum and improve the stability of spectrum estimation by weighted averaging the energy spectra of adjacent frequency points in the frequency domain. Spectral smoothing uses a sliding window smoothing method in the time-frequency domain to perform a weighted average on each frequency point, taking into account the corresponding frequency points of the previous and subsequent frames. The smoothed frequency domain signal has better smoothness and continuity than the initial complex frequency domain signal, reduces the influence of random noise, and is beneficial to subsequent signal processing.

[0029] The smooth frequency domain signal is divided into nonlinear subbands to obtain subband signals. Nonlinear subband division refers to dividing the frequency domain signal into multiple subbands according to the auditory characteristics of the human ear. The subband division of the low-frequency part is finer, and the subband division of the high-frequency part is coarser. This division method conforms to the human ear's perception characteristics of sounds of different frequencies. The low-frequency part contains more voice information and requires more sophisticated processing. Nonlinear subband division uses the Mel frequency scale or the Bark frequency scale to convert the linear frequency scale into a perceptual frequency scale. For a signal with a sampling rate of 16kHz, it is usually divided into 20-30 subbands. The subband signal is the signal after nonlinear subband division, and each subband contains signal components within a specific frequency range.

[0030] Perform geometric phase compensation on the sub-band signal to obtain a sub-band correction signal. Geometric phase compensation refers to the phase adjustment of the signal received by each microphone according to the geometric layout of the microphone array and the direction of the sound source, so that the sound source signal in the target direction is aligned in phase, thereby enhancing the target sound source signal. Geometric phase compensation calculates the corresponding phase difference based on the time delay difference from the sound source to each microphone, and performs phase correction on each microphone signal. For linear microphone arrays, phase compensation is usually based on the plane wave assumption; for annular or other shaped microphone arrays, phase compensation takes into account more complex geometric relationships. The sub-band correction signal is a signal after geometric phase compensation, and the sound source signal in the target direction received by each microphone is aligned in phase.

[0031] Based on the preset noise power spectrum, the subband correction signal is denoised by spectral subtraction, and the complex time-frequency domain feature representation conversion is performed to obtain a complex time-frequency domain signal. Spectral subtraction denoising is a commonly used speech enhancement technology. The enhanced speech signal power spectrum is obtained by subtracting the estimated noise power spectrum from the observed signal power spectrum. The preset noise power spectrum is the background noise power spectrum estimated during the period without speech activity. Spectral subtraction denoising adopts an over-subtraction or under-subtraction strategy and sets a spectral floor to avoid generating music noise. The complex time-frequency domain feature representation conversion converts the enhanced signal from the power spectrum domain back to the complex time-frequency domain to facilitate subsequent beamforming and signal reconstruction. The complex time-frequency domain signal retains the amplitude and phase information of the enhanced signal, providing the necessary information for beamforming and inverse transformation.

[0032] This embodiment achieves efficient target sound source extraction in a complex acoustic environment through a systematic signal processing process. Through a preprocessing method combining endpoint detection and noise power estimation, effective speech and background noise are accurately distinguished, laying the foundation for subsequent processing. The nonlinear sub-band division technology is adopted to finely divide the frequency domain signal according to the auditory characteristics of the human ear, the low-frequency part is more refined, and the high-frequency part is relatively rough, which effectively improves the quality of speech perception. The geometric structure phase compensation makes full use of the spatial information of the microphone array to achieve phase alignment of the sound source signal in the target direction, and significantly enhances the spatial selectivity of the target sound source. The spectrum smoothing technology reduces spectrum fluctuations and improves the stability of spectrum estimation, while the spectrum subtraction denoising based on the preset noise power spectrum further suppresses background noise interference. The organic combination of these technologies forms an integrated signal enhancement solution, which improves signal clarity and intelligibility while maintaining the naturalness of the voice, and is particularly suitable for voice interaction scenarios in complex acoustic environments such as conference systems and smart speakers.

[0033] In one embodiment, obtaining a spatial distribution vector of a multi-microphone array and performing deformation compensation processing on complex time-frequency domain signals to obtain a sound field consistency signal includes: By measuring or setting the position coordinates of each microphone element in three-dimensional space and its pickup angle parameters, a complete spatial geometric structure model is constructed. These physical coordinates and orientation parameters are integrated into a unified spatial coordinate system to generate a spatial distribution vector. The spatial distribution vector contains the precise spatial position and directionality information of each pickup element in the microphone array, providing basic data for subsequent acoustic characteristic mapping.

[0034] After obtaining the spatial distribution vector, the vector is used to perform spatial transfer function mapping calculation on the converted complex time-frequency domain signal. This mapping process associates the complex time-frequency domain signal received by each microphone with its spatial position and direction characteristics to generate array acoustic characteristic mapping information. The array acoustic characteristic mapping information characterizes the characteristics of the interaction between the sound wave and the microphone array when propagating in space, including information such as the time difference of the sound wave reaching each microphone, amplitude attenuation, and phase change.

[0035] The array acoustic characteristic mapping information is subjected to singular value decomposition (SVD) operation to decompose the complex mapping information into several orthogonal characteristic components. Through this decomposition, the system identifies the main acoustic characteristic patterns and secondary interference components. Subsequently, the system performs nonlinear correction of the phase and amplitude of the decomposition results to eliminate or reduce the distortion caused by uneven microphone spacing, sensitivity differences and hardware errors, thereby obtaining a preliminary compensation signal. The preliminary compensation signal has higher spatial consistency than the original signal, but is not yet fully aligned.

[0036] Based on the preliminary compensation signal, the system calculates the cross-power spectral density matrix (CPSD), which describes the correlation and energy distribution between different microphone signals. Combining the physical characteristics and propagation laws of the sound field, the system constructs the geometric consistency constraints of the sound field, which reflect the geometric relationship that the sound field should present at each microphone position under ideal conditions. By applying these constraints, the system generates a topological representation of the sound field, which describes the geometric relationship between the sound source, the spatial structure of the sound field, and the microphone array.

[0037] The topological representation of the sound field is processed by tensor decomposition in the complex domain, and the sound field characteristics are decomposed into amplitude components and phase components in the complex domain to obtain the deformation compensation coefficient. The deformation compensation coefficient contains fine correction parameters for each frequency point and each microphone to offset the non-ideal deformation factors in the sound field.

[0038] The deformation compensation coefficient is used to perform precise phase and amplitude alignment operations on the complex time-frequency domain signals. Phase alignment ensures that the phase difference between the signals from the same sound source and different microphones is properly compensated, while amplitude alignment makes the signal strength received by each microphone balanced and consistent. After this processing, the system obtains a sound field consistency signal, which is highly consistent in space and provides ideal basic data for subsequent beamforming processing.

[0039] Compared with the original signal, the spatial coherence of the sound field consistency signal is significantly improved, which reduces the impact of environmental noise and reverberation and enhances the quality of the target sound source signal, thereby achieving more accurate sound source localization and signal enhancement in complex acoustic environments.

[0040] This embodiment obtains the spatial distribution vector of the multi-microphone array and performs deformation compensation processing. The method achieves high-precision spatial consistency alignment of the microphone array signal and significantly improves the accuracy and stability of beamforming. The spatial transfer function mapping calculation combined with the singular value decomposition technology effectively solves the problem of acoustic characteristic distortion caused by component position errors, sensitivity differences and hardware inconsistencies in the actual deployment of the microphone array. By constructing the geometric consistency constraint of the sound field and performing complex domain tensor decomposition, this method can accurately identify and compensate for nonlinear deformations in the sound field, overcoming the limitations of traditional methods that are difficult to cope with complex acoustic environments. The precise alignment of phase and amplitude makes the signals from the same sound source highly consistent between different microphones, thereby significantly suppressing environmental noise and reverberation interference while maintaining the integrity of the target sound source signal.

[0041] In one embodiment, the sound source direction feature is extracted from the sound field consistency signal, and a complex spatiotemporal attention beamforming network is processed to obtain a beamforming signal, including: The sound field consistency signal collected by the multi-microphone array is decomposed by adaptive wavelet packet transform to generate a multi-level time-frequency representation of the sound field. Adaptive wavelet packet transform is a time-frequency analysis method that decomposes the signal at different frequency and time scales and is suitable for the processing of non-stationary signals. The transform automatically adjusts the decomposition level and wavelet basis function according to the local characteristics of the signal to obtain the best time-frequency resolution. During the transformation process, each microphone signal is mapped to multiple frequency sub-bands to form a time-frequency atomic structure. These time-frequency atoms retain the time and frequency characteristics of the original signal and effectively capture the energy distribution and phase information of the sound field in each frequency band. The multi-level time-frequency representation of the sound field contains the complete time-frequency characteristics of the sound field in different frequency ranges, providing a basis for subsequent spatial feature analysis.

[0042] The multi-level time-frequency representation of the sound field is sent to the complex domain spatial covariance calculation unit to calculate the phase difference spectrum of each frequency band. Complex domain spatial covariance is a statistical description of the phase relationship between different microphone signals, reflecting the characteristics of sound wave propagation in space. During the calculation process, for each frequency band, a complex covariance matrix is ​​formed between the multi-microphone signals, which contains the phase difference information between the microphone pairs. The band phase difference spectrum is a three-dimensional tensor structure with dimensions of frequency, space and time. It records the spatial phase change patterns in the sound field, which are closely related to the direction and distance of the sound source. The band phase difference spectrum has a high signal-to-noise ratio and can maintain relatively stable directional information even in a noisy environment.

[0043] The frequency band phase difference spectrum is subjected to high-order singular value decomposition to obtain an orthogonal projection structure. High-order singular value decomposition is a multidimensional data analysis technology suitable for processing three-dimensional and higher tensor data. This decomposition decomposes the frequency band phase difference spectrum tensor into a series of orthogonal subspace projections, each of which represents an independent spatial pattern in the sound field. The orthogonal projection structure consists of a main singular matrix and corresponding singular values. The main singular matrix reflects the main spatial features in the frequency band phase difference spectrum, while the singular values ​​indicate the significance of each feature. The orthogonal projection structure eliminates redundant information, highlights the main directional features in the sound field, and effectively improves the accuracy and robustness of subsequent directional estimation.

[0044] Based on the orthogonal projection structure, a hyperbolic direction estimation function is constructed and a gradient polarization search is performed to obtain the sound source direction vector. The hyperbolic direction estimation function is a nonlinear mapping that maps spatial coordinates to directional similarity scores to form a hyperbolic surface. This function uses the orthogonal projection structure of the band phase difference spectrum to form a directional response surface in three-dimensional space. Gradient polarization search is an algorithm for finding the optimal point on this response surface. It iteratively calculates the gradient direction and moves along the gradient direction, and finally converges to the extreme point of the directional response surface. The sound source direction vector is a three-dimensional unit vector that points to the spatial position of the sound source and contains azimuth and pitch angle information. The calculation of the direction vector takes into account the fusion of multi-band information, ensuring the stability of direction estimation in various acoustic environments.

[0045] The sound source direction vector is phase-aligned with the sound field consistency signal to obtain the initial beam gain coefficient. Phase alignment is the core step of beamforming, which makes the phases of the microphone signals consistent in a specific direction, thereby enhancing the sound source signal in that direction. During the alignment process, the time delay difference from the sound source to each microphone is calculated based on the sound source direction vector and the geometric layout of the microphone array, and the phase of each microphone signal is adjusted accordingly. The initial beam gain coefficient is a set of complex weights that contain amplitude and phase information, which are used to weighted synthesize the signals of each microphone. These coefficients make the signals in the target direction overlap in phase, while the signals in the non-target direction cancel each other out due to phase inconsistency, thereby forming a spatial selective gain.

[0046] The complex domain attention mask is constructed according to the initial beam gain coefficient, and the sound field consistency signal is channel-weighted to obtain the beamforming signal. The complex domain attention mask is a spatial filter operating on the complex plane, which assigns different weights to signals of different directions and frequencies. The mask construction process uses the initial beam gain coefficient and further optimizes the spatial filtering characteristics through an adaptive learning mechanism. The mask not only considers the amplitude of the signal, but also retains the phase information, ensuring phase consistency during the beamforming process. Channel weighting is the process of applying the complex domain attention mask to the sound field consistency signal. The signals of each channel are weighted and superimposed in the complex domain to form the final beamforming output. The beamforming signal has a high signal-to-noise ratio and clarity, effectively suppresses environmental noise and reverberation interference, and enhances the sound source signal in the target direction. The beamforming signal is used in subsequent applications such as speech recognition, sound source localization, and acoustic monitoring, significantly improving the performance of these applications in complex acoustic environments.

[0047] This embodiment obtains a multi-level time-frequency representation of the sound field by performing adaptive wavelet packet transform decomposition on the sound field consistency signal, effectively capturing the energy distribution and phase information of the sound field in each frequency band, and providing a solid foundation for accurate sound source positioning. The frequency band phase difference spectrum obtained by complex domain spatial covariance calculation can maintain stable directional information in a high noise environment, significantly improving the anti-interference ability of the system. The orthogonal projection structure generated by high-order singular value decomposition eliminates redundant information, highlights the main directional features in the sound field, and enhances the accuracy of direction estimation. Based on the method of hyperbolic direction estimation function and gradient polarization search, high-precision sound source direction vector calculation is achieved, adapting to changes in complex acoustic environments. The initial beam gain coefficient obtained by phase alignment makes the signal phase in the target direction consistent and superimposed, while the signals in the non-target direction cancel each other out. Finally, the construction and application of the complex domain attention mask optimizes the spatial filtering characteristics while retaining the phase information, produces a beamforming signal with high signal-to-noise ratio and clarity, and effectively suppresses environmental noise and reverberation interference.

[0048] In one embodiment, the beam direction and gain parameters of the beamforming signal are optimized by a preset adversarial environment simulator to obtain a dynamic frequency band enhancement signal, including: The simulator contains a noise generation module, a reverberation simulation module and an environmental condition parameter library to create diverse environmental interference samples.

[0049] The adversarial environment simulator simulates various noise interference and reverberation conditions on the beamforming signal to generate diverse environmental interference samples. These samples include various noise types such as white noise, pink noise, mechanical noise, and human voice, as well as reverberation conditions under different room sizes, materials, and shapes. The environmental interference samples form an interference matrix by locating the interference source and mapping the interference intensity. The matrix contains interference information in both spatial and frequency dimensions.

[0050] When performing gradient descent optimization calculations on samples of diverse environmental interference, the stochastic gradient descent algorithm is used to adjust the beam direction parameters and the frequency band gain coefficients according to the preset signal-to-noise ratio objective function. The signal-to-noise ratio objective function is defined as the logarithmic value of the ratio of the expected signal power to the noise power. The function sets the threshold to 15 dB to achieve adaptive adjustment of the beam direction. During the optimization process, the adjustment range of the beam direction parameters is ±30 degrees, and the adjustment range of the frequency band gain coefficient is 0.5 to 2.0. The parameter optimization matrix is ​​obtained through 300 iterative calculations. The parameter optimization matrix contains the beam direction angle correction value and the adjustment value of each frequency band gain coefficient, which is used for subsequent signal reconstruction.

[0051] The beamforming signal is reconstructed according to the parameter optimization matrix to obtain a directional enhanced signal. During the directional reconstruction process, the subband decomposition technology is used to decompose the signal into 32 sub-bands, and each sub-band is independently corrected by the corresponding directional parameters. The directional enhanced signal has a more accurate sound source localization capability and a stronger noise suppression effect than the original beamforming signal, especially in a low signal-to-noise ratio environment.

[0052] According to the direction enhancement signal, the acoustic feature matching is performed with the preset speech data to obtain the corresponding target actual short-time speech data. The acoustic feature matching process uses Mel Frequency Cepstral Coefficient (MFCC) feature extraction and Dynamic Time Warping (DTW) distance calculation to retrieve the most similar speech segment from the preset speech database. The preset speech database contains 8,000 standard speech samples covering speakers of different genders, ages and languages. The target actual short-time speech data represents the ideal speech signal characteristics and is used to guide subsequent signal enhancement processing.

[0053] Based on the preset meta-learning online calibration algorithm, the target actual short-term speech data is feature extracted to obtain the target feature adaptation matrix. The meta-learning online calibration algorithm adopts a model-independent adaptive learning mechanism, which includes three core components: feature extraction layer, feature mapping layer and adaptation layer. The feature extraction layer uses short-time Fourier transform to extract time-frequency features, the feature mapping layer identifies key feature points through the attention mechanism, and the adaptation layer generates a 16×16 dimensional adaptation matrix. The target feature adaptation matrix contains the gain and phase adjustment parameters required for spectrum correction, and has high-dimensional feature expression capabilities.

[0054] The target feature adaptation matrix is ​​convolved with the directional enhancement signal to obtain the spectrum correction signal. The convolution operation adopts a two-dimensional convolution method with a convolution kernel size of 5×5, a step size of 1, and a fill mode of "same". During the convolution process, each element of the adaptation matrix is ​​multiplied and accumulated with the corresponding frequency-time point of the signal to achieve fine adjustment of the spectrum. The spectrum correction signal has a clearer performance in the harmonic structure and resonance peak, effectively eliminating the spectrum distortion caused by environmental interference.

[0055] The spectrum correction signal is subjected to cross-band consistency processing and phase band gain to obtain a dynamic band enhancement signal. The cross-band consistency processing adopts an adjacent band smoothing algorithm and uses a 5-point smoothing window to eliminate discontinuities between bands. The phase band gain adopts group delay correction technology to maintain the continuity of the signal phase, and the phase adjustment range is ±0.2π. The dynamic band enhancement signal is manifested in a low-frequency enhancement of 3-6dB, a mid-frequency enhancement of 1-3dB, and a high-frequency enhancement of 4-8dB in the spectrum. The overall enhancement characteristics are dynamically adjusted as the environment changes.

[0056] This embodiment optimizes the beam direction and gain parameters of the beamforming signal through an adversarial environment simulator, which can significantly improve the quality and clarity of the signal in various complex noise and reverberation environments. This method effectively adjusts the beam direction and frequency band gain by simulating a variety of environmental interference samples and combining the gradient descent optimization algorithm, so that the beamforming signal can be adaptively adjusted when facing different interferences, thereby improving the robustness and adaptability of the system. Based on the actual short-term speech data of the target, acoustic feature matching is performed, so that the system can more accurately restore the time-frequency characteristics of the speech signal, further enhancing the intelligibility of the speech. The target features are extracted and the adaptation matrix is ​​generated through the meta-learning online calibration algorithm, which effectively improves the spectral quality of the signal, making the effect of the spectrum correction signal more significant. The final dynamic frequency band enhancement signal can provide a stronger enhancement effect in different frequency bands, especially in low signal-to-noise ratio environments.

[0057] In one embodiment, a variety of noise interference and reverberation conditions are simulated on the beamforming signal based on a confrontation environment simulator to obtain diverse environmental interference samples, including: The adversarial environment simulator performs a time-frequency transform operation on the beamforming signal. Time-frequency transform is the process of converting a time-domain signal into a time-frequency domain representation. The short-time Fourier transform (STFT) is used to decompose the beamforming signal into a representation in two dimensions: time and frequency. The result of the time-frequency transform is a time-frequency domain representation signal matrix, which contains the energy distribution information of the signal at different time points and different frequencies.

[0058] A three-dimensional acoustic propagation structure is constructed based on the signal matrix represented in the time-frequency domain. The three-dimensional acoustic propagation structure is a mathematical model that simulates the propagation characteristics of sound waves in three-dimensional space. The structure takes into account factors such as spatial geometry, obstacle locations, and surface materials. During the construction process, the adversarial environment simulator sets random reflection coefficients to simulate the reflection characteristics of sound waves on surfaces of different materials. The randomization of reflection coefficients ensures the diversity and authenticity of the simulated environment. Through this step, the system generates a set of spatial reverberation parameters, which includes key parameters such as room impulse response (RIR), early reflections, and late reverberation.

[0059] The adversarial environment simulator randomly extracts various types of noise sources from a preset environmental noise library. The environmental noise library is a data set containing various real environmental noise recordings, such as traffic noise, crowd noise, machine noise, etc. These noise sources are stretched in the time domain to change the duration and time characteristics of the noise; at the same time, frequency domain offset processing is performed to adjust the frequency characteristics of the noise. Time domain stretching uses phase vocoder technology to keep the noise timbre characteristics unchanged; frequency domain offset is achieved through spectrum shifting. After these processes, the system obtains a set of deformed noise sources, which has higher diversity and unpredictability.

[0060] The adversarial environment simulator performs position mapping on the set of deformed noise sources based on the preset spatial directional distribution function. The spatial directional distribution function describes the distribution law of noise sources in space and defines the possibility of noise sources appearing in a specific direction based on the probability density function. The position mapping process distributes the deformed noise sources to various directions in three-dimensional space and generates a multi-directional noise distribution matrix. This matrix represents the spatial distribution of noise sources from different directions, increasing the spatial diversity of noise interference.

[0061] The specific process of position mapping the deformation noise source set according to the preset spatial directional distribution function is as follows: The spatial directional distribution function is a mathematical description of the distribution law of noise sources in three-dimensional space. The position mapping process first assigns spatial position coordinates to each deformed noise source. , where A represents the horizontal angle (azimuth), C represents the vertical angle (elevation), and r represents the distance.

[0062] For each noise source ni, the system is distributed from the spatial directionality function The azimuth and elevation angles are sampled from the ADC. This distribution function can be a uniform distribution, a Gaussian distribution, or a custom distribution based on actual environment measurements. The distance parameter r is sampled based on a preset distance distribution function D(r), which defines the possible distance range between the noise source and the microphone array.

[0063] The mathematical expression of position mapping is: ; Where M represents the mapping function, represents the i-th deformation noise source, represents the angle value sampled from the directional distribution function, Represents the distance value sampled from the distance distribution function.

[0064] After completing the position mapping of all deformed noise sources, the system constructs a multi-directional noise distribution matrix N. Each row of the matrix contains the location information and noise characteristic parameters of a noise source, forming a complete spatial distribution representation. This matrix will be used for subsequent acoustic propagation delay and energy attenuation calculations to ensure that the simulated noise interference has real spatial characteristics.

[0065] The adversarial environment simulator calculates acoustic propagation delay and energy attenuation based on the multi-directional noise distribution matrix and the spatial reverberation parameter set. Acoustic propagation delay refers to the time required for sound waves to propagate from the noise source to the receiving point, which is related to the distance from the sound source and the propagation speed of the medium; energy attenuation describes the degree of energy loss of sound waves during propagation, which is related to the propagation distance and the environmental absorption coefficient. By calculating the propagation path characteristics from each noise source to the microphone array, the system generates a noise propagation characteristic vector, which contains the delay, attenuation and direction information of each noise source from propagating to the receiving point.

[0066] The adversarial environment simulator performs reverberation processing on the signal matrix represented in the time-frequency domain based on the spatial reverberation parameter set. Reverberation processing simulates the acoustic effect produced by multiple reflections of sound waves in a closed space, and is achieved by convolution operation with the room impulse response. The processed signal is interfered and superimposed with the noise propagation characteristic vector to simulate the situation in which the target signal in a real environment is interfered with by multiple noise sources and reverberation at the same time. The superposition process takes into account the spatial position, propagation characteristics and energy size of the noise source to ensure that the generated interference pattern is more realistic. Finally, a variety of environmental interference samples are obtained, which reflect the signal characteristics in various complex acoustic environments.

[0067] The generation of diverse environmental interference samples greatly enriches the diversity of training data, enabling the beamforming algorithm to learn to adapt to various complex environmental conditions, thereby improving the robustness and generalization ability of the system in practical applications. Through these samples generated by the adversarial environment simulator, the beamforming system can maintain good signal enhancement effects under unseen noise and reverberation conditions.

[0068] This embodiment can effectively simulate a variety of noise interference and reverberation conditions and generate diversified environmental interference samples by introducing an adversarial environment simulator into the multi-microphone array beamforming signal enhancement method. This method achieves accurate simulation of complex acoustic environments through time-frequency transformation, construction of three-dimensional acoustic propagation structure, random extraction of deformed noise sources, position mapping, and acoustic propagation delay and energy attenuation calculation, thereby greatly improving the robustness and adaptability of beamforming signal enhancement. By stretching the real environmental noise in the time domain and shifting the frequency domain, the generated noise source has a higher diversity, which can effectively improve the system's ability to handle unexpected noise interference. The introduction of spatial reverberation parameter sets and multi-directional noise distribution matrices makes the generated interference samples more realistic and reflects the reverberation and noise characteristics in the real environment. Finally, by performing reverberation processing and interference superposition on the signal matrix represented in the time-frequency domain, the generated diversified environmental interference samples greatly enrich the training data, which helps the performance of the beamforming algorithm in complex environments, thereby improving the signal enhancement effect and generalization ability of the system.

[0069] In one embodiment, feature extraction is performed on the target actual short-term speech data based on a preset meta-learning online calibration algorithm to obtain a target feature adaptation matrix, including: The target actual short-term speech data is subjected to short-term window framing and windowing processing to obtain an overlapping segmented signal sequence. Specifically, a 25-millisecond Hamming window is used to frame the original speech signal, and the overlap between adjacent frames is 10 milliseconds. This processing is based on the assumption that the speech signal has quasi-steady-state characteristics in a short period of time, and the continuity and smooth transition of the signal are ensured by overlapping framing. After the windowing processing, the edge of each speech frame is attenuated, reducing the spectrum leakage phenomenon. The overlapping segmented signal sequence is a time series set composed of multiple overlapping short-term speech segments, which retains the timing characteristics and local spectrum characteristics of the original speech.

[0070] Mel-frequency cepstral coefficients are calculated for overlapping segmented signal sequences to obtain segmented feature description vectors. The Mel-frequency cepstral coefficient calculation process includes performing fast Fourier transform on each segmented signal to obtain the power spectrum, converting the linear frequency scale to the Mel-frequency scale, and applying discrete cosine transform to obtain the cepstral coefficients. Usually, 13-26 dimensional Mel-frequency cepstral coefficients are extracted, and then combined with the first-order difference and second-order difference coefficients to form a 39-78 dimensional feature vector. The segmented feature description vector is a set of multidimensional vectors that can effectively characterize the acoustic characteristics of speech and capture the key information of speech in the spectral domain.

[0071] The meta-learning task sampling set is constructed for the segmented feature description vector to obtain the feature support set and feature query set. The meta-learning task sampling set is constructed using the N-way K-shot strategy. N categories are randomly selected from the segmented feature description vector. Each category contains K samples to form the feature support set. In addition, some samples are selected to form the feature query set. The feature support set is used for model parameter initialization and inner loop optimization, and the feature query set is used to evaluate the model generalization ability and outer loop optimization. The "category" here refers to different acoustic environments or speaker feature patterns.

[0072] Based on the feature support set, a meta-learning inner loop optimization function is constructed to obtain the environmental adaptation parameters. The meta-learning inner loop optimization function uses a gradient descent algorithm to perform local optimization on the feature support set. The loss function is set to the prototype network metric loss. The optimization goal is to minimize the Euclidean distance between the samples in the feature support set and the corresponding category prototype. The environmental adaptation parameters include the embedding function parameter matrix and the metric space transformation matrix under a specific environment, which reflect the adaptive adjustment results of the model to the current acoustic environment.

[0073] The feature query set is updated with MAML (Model-Agnostic Meta-Learning) gradients according to the environment adaptation parameters to obtain the task model parameters. MAML gradient update calculates the gradient of the loss function on the feature query set to the environment adaptation parameters and uses the second-order derivative optimization method to adjust the parameters. The update process includes three steps: forward propagation to calculate the loss, back propagation to calculate the gradient, and parameter update. The task model parameters are the set of model weights after environment adaptation and have the ability to quickly adapt to new environments.

[0074] Meta-learning outer loop optimization integrates and optimizes the meta-learning results of multiple tasks to improve the generalization ability and adaptability of the model. The specific process is as follows: The tasks in the task pool are sampled to form multiple tasks, each of which contains a feature support set and a feature query set. For each task, the loss function on the feature query set is calculated based on the current task model parameters. The loss function usually uses cross entropy loss or mean square error to reflect the performance of the model on the task.

[0075] The gradient of the loss function of each task is calculated to obtain the gradient of the loss function to the model parameters. The gradients of all tasks are averaged to obtain the global gradient. Based on the global gradient, the global parameters of the model are updated through the gradient descent algorithm. The updated model parameters can better adapt to different tasks and improve the generalization ability of the model.

[0076] The above task sampling, loss calculation, gradient calculation and parameter update process is repeated in the outer loop until the loss function converges or reaches the preset number of iterations. Through multiple iterations, the model parameters are gradually optimized to better adapt to different acoustic environments and signal characteristics.

[0077] The purpose of online learning rate adjustment is to dynamically adjust the learning rate of the model so that it has the optimal learning rate under different environments and tasks. The specific process is as follows: The online learning rate adjustment adopts the Bayesian optimization framework. Bayesian optimization is a global optimization method based on probability model. It selects the optimal learning rate parameter by constructing the probability distribution model of the objective function. The Bayesian optimization framework includes three main parts: prior distribution, likelihood function and posterior distribution.

[0078] According to the historical optimization trajectory and current performance indicators, the prior distribution of the learning rate parameters is constructed. The prior distribution reflects the initial estimate of the learning rate parameters and can be described by Gaussian process or other probability distribution models.

[0079] Based on the loss function and gradient information of the current task, the posterior distribution of the learning rate parameter is updated. The posterior distribution combines the prior information and the observed data of the current task to more accurately reflect the optimal value of the learning rate parameter.

[0080] By maximizing the likelihood function of the posterior distribution, the optimal learning rate parameter is selected. The optimal learning rate parameter is the learning rate value that minimizes the loss of the model on the current task.

[0081] Generate a dynamic learning rate matrix based on the optimal learning rate parameters. The dynamic learning rate matrix contains adaptive learning rate values ​​for different parameter layers and different tasks. The learning rate of each parameter layer is adjusted according to its importance and degree of variation on different tasks, so that the model can learn at the optimal rate in different environments and tasks.

[0082] The segmented feature description vector is mapped to a prototype network according to the dynamic learning rate matrix to obtain the environmental perception feature representation. The prototype network mapping process includes two main steps: feature embedding and prototype calculation. Feature embedding uses a deep neural network to map the segmented feature description vector to a high-dimensional embedding space; prototype calculation obtains the category prototype vector by averaging the embedding vectors of samples of the same category. The environmental perception feature representation is a set of high-dimensional feature vectors with environmental adaptability generated under the prototype network framework, which retains the key features of the speech signal while filtering out the influence of environmental noise.

[0083] The embedding space of environmental perception feature representation is constructed by metric learning to obtain the speech feature embedding matrix. The metric learning embedding space construction adopts the Mahalanobis distance metric framework, and the distance between similar samples is minimized and the distance between heterogeneous samples is maximized by learning the optimal distance metric matrix.

[0084] The process of constructing the embedding space of metric learning includes three main steps: initializing the metric matrix, iteratively optimizing the metric parameters, and verifying the metric performance. In the initialization stage, the covariance matrix is ​​calculated based on the environmental perception feature representation, and the initial distance metric matrix is ​​obtained using the eigendecomposition method. The covariance matrix reflects the correlation between different feature dimensions, and its inverse matrix is ​​used to construct the Mahalanobis distance metric. The initial distance metric matrix serves as the initial parameter of metric learning and provides an initial metric benchmark for subsequent optimization.

[0085] In the iterative optimization stage, the contrast loss function is used to update the distance metric matrix. The optimization goal is to minimize the Mahalanobis distance between samples of the same category and maximize the Mahalanobis distance between samples of different categories. During the optimization process, positive and negative sample pairs are constructed, where the positive sample pairs come from the same category and the negative sample pairs come from different categories. Multiple positive and negative sample pairs are constructed by random sampling, and the gradient of the loss function is calculated. The distance metric matrix is ​​iteratively adjusted using the gradient descent method so that the optimization goal gradually converges.

[0086] In the stage of verifying the metric performance, the quality of the optimized metric space is evaluated using classification accuracy and clustering metrics. Classification accuracy is evaluated by the performance of the nearest neighbor classifier in the metric space, and clustering metrics include intra-class distance, inter-class distance, and silhouette coefficient. Intra-class distance measures the closeness of samples of the same category, inter-class distance measures the degree of separation of samples of different categories, and silhouette coefficient comprehensively evaluates the clustering effect. The optimized metric learning embedding space is evaluated by the above indicators, and the training strategy is adjusted according to the evaluation results.

[0087] The speech feature embedding matrix is ​​a data structure containing multiple speech feature vectors. These feature vectors have good intra-class aggregation and inter-class separability in the learned metric space, ensuring the ability to distinguish speech features in different environments in the metric space.

[0088] The speech feature embedding matrix is ​​constrained by Bayesian credible intervals to obtain the target feature adaptation matrix. The Bayesian credible interval constraint estimates the uncertainty interval of the parameters by establishing a probability distribution model of the feature parameters, and filters out unreliable feature representations. The constraint process includes four key steps: constructing the prior distribution of feature parameters, calculating the posterior distribution, determining the credible interval, and screening the feature vector. The target feature adaptation matrix is ​​a set of feature representations that have been screened for credibility. It has higher robustness and environmental adaptability, and provides a reliable feature basis for subsequent beamforming processing.

[0089] This embodiment can effectively adapt to different acoustic environments and improve the clarity and comprehensibility of speech signals by adopting a preset meta-learning online calibration algorithm for feature extraction in a multi-microphone array beamforming signal enhancement method. Through short-time window framing and windowing processing, as well as Mel frequency cepstral coefficient calculation, the time and spectral characteristics of the original speech are carefully preserved. By using meta-learning task sampling set construction and inner loop optimization function, the model parameters can be quickly adjusted to adapt to the new acoustic environment, thereby improving the flexibility and adaptability of the model. MAML gradient update and meta-learning outer loop optimization ensure the efficient learning ability of the model on different tasks, and through online learning rate adjustment, the learning rate is adaptively changed, further improving the training efficiency and stability of the model. Through the construction of metric learning embedding space, speech features have good intra-class aggregation and inter-class separability in high-dimensional quantitative space, ensuring the ability to distinguish speech features in different environments. The whole method not only improves the accuracy of speech recognition in noisy environments, but also enhances the robustness and practicality of the system under complex acoustic conditions.

[0090] In one embodiment, the dynamic frequency band enhancement signal is subjected to inverse time-frequency conversion and low-delay optimization to obtain a target quality speech output signal, including: The dynamic band enhancement signal is converted to the original time domain signal through an overlap-add inverse transform process. The overlap-add inverse transform is a method of converting a frequency domain signal back to the time domain. The process uses an inverse short-time Fourier transform (ISTFT). In this process, the frequency domain data is divided into several frames, each frame of data is inverse Fourier transformed, and overlap-added in the time domain. The overlap factor is usually set to 50% or 75% to ensure a smooth transition. The initial time domain signal retains the basic time domain characteristics of the original sound, but there may be phase discontinuities.

[0091] The initial time domain signal is compensated for inter-frame phase consistency to obtain a phase-continuous intermediate signal. This step is designed to solve the problem of phase discontinuity that may occur between adjacent frames to avoid artificial synthesis sound effects and distortion. Phase consistency compensation calculates the phase difference between adjacent frames and applies a phase correction factor to achieve a smooth transition of phase. For each frequency component, the system calculates its phase continuity metric and performs phase correction when a phase jump exceeding a preset threshold (usually π / 2) is detected. The phase-continuous intermediate signal has a more natural sound characteristic and reduces the traces of artificial synthesis.

[0092] The phase-continuous intermediate signal is processed by dynamic buffer adjustment to obtain a variable signal buffer. Dynamic buffer adjustment adaptively adjusts the buffer size based on signal characteristics and processing requirements. Based on voice activity detection (VAD), a smaller buffer is used in silent segments to reduce latency, while a larger buffer is used in active speech segments to ensure processing quality. The buffer size adjustment range is between 10ms and 50ms, with an adjustment step of 5ms. The variable signal buffer effectively balances the relationship between processing latency and signal quality.

[0093] The variable signal buffer is processed by nonlinear amplitude normalization to obtain a dynamic range compressed signal. Nonlinear amplitude normalization uses logarithmic compression or hyperbolic tangent function to nonlinearly map the signal amplitude, compressing high amplitude signals while enhancing low amplitude signals. Normalization parameters are dynamically adjusted according to the statistical characteristics of the signal, including compression ratio (1.5:1 to 3:1) and threshold (-20dB to -30dB). The dynamic range compressed signal has a more balanced sound energy distribution, which enhances the audibility and intelligibility of speech.

[0094] The dynamic range compression signal is subjected to time domain transient feature vector calculation and time domain jitter elimination to obtain a jitter correction signal. The time domain transient feature vector is calculated by analyzing parameters such as the short-term energy change, zero crossing rate, and spectral entropy of the signal. Jitter detection is based on the degree of mutation of the feature vector. When jitter is detected (the feature vector change exceeds the preset threshold of 25%), an adaptive smoothing filter is applied to suppress jitter. The jitter elimination process retains the transient characteristics of the speech signal while removing instability. The jitter correction signal has a more stable speech quality.

[0095] The jitter correction signal is processed by multi-stage cascade band reconstruction to obtain a frequency response equalization signal. Multi-stage cascade band reconstruction uses a segmented multi-resolution filter group to decompose the signal into multiple sub-bands (usually 4 to 8 sub-bands), which are enhanced separately and then synthesized. The low frequency band (0-1kHz) focuses on preserving the fundamental frequency and timbre information, the mid-frequency band (1-4kHz) focuses on improving the clarity of speech, and the high frequency band (4-8kHz) focuses on restoring speech details. The frequency response equalization signal has a more comprehensive and balanced spectrum distribution, and the clarity and naturalness of speech are significantly improved.

[0096] The frequency response equalization signal is processed by device pre-compensation to obtain the device matching signal. Device pre-compensation is based on the frequency response characteristics of the target playback device, and the inverse filtering technology is used for pre-correction to compensate for the distortion that may be introduced by the device. The compensation processing includes frequency response correction, phase correction, and dynamic range adjustment. The correction parameters are obtained through the device characteristic database, and different compensation templates are applied to different categories of devices (such as headphones, speakers, or mobile devices). The device matching signal is optimized for a specific playback device to ensure the best listening experience in the actual usage environment.

[0097] The device matching signal is processed by the frame selective discarding algorithm to eliminate noise and artifacts, and finally obtain the target quality voice output signal. The frame selective discarding algorithm scores the quality of each frame signal based on the signal quality evaluation indicators. These evaluation indicators include signal-to-noise ratio (SNR), voice presence probability (VAD) and spectral distortion (SDR). In the signal-to-noise ratio evaluation, the system calculates the ratio between the energy of each frame signal and the background noise. When the signal-to-noise ratio is lower than the preset threshold (such as 10dB), the frame is considered to have poor quality. The voice presence probability is determined by the voice activity detection (VAD) algorithm. When the voice presence probability is lower than the preset threshold (such as 0.4), the frame is considered to be a non-voice frame. The spectral distortion is evaluated by comparing the spectrum difference between the current frame and the frames before and after it. When the spectral distortion exceeds the preset threshold (such as 0.2), the frame is considered to have artifacts.

[0098] When the signal quality score is lower than the preset threshold, the frame will be discarded or replaced with an interpolated frame. The interpolated frame is generated based on the data of the previous and next frames through linear interpolation or spline interpolation technology. This processing method ensures the continuity and natural transition of the signal.

[0099] The noise artifact elimination process uses wavelet decomposition and reconstruction technology. First, the device matching signal is decomposed by wavelet decomposition, which decomposes the signal into several levels of detail and approximation components. The detail component contains high-frequency noise and artifact information, while the approximation component retains the main characteristics of the signal. In each level of detail component, the system applies threshold processing methods (such as soft threshold or hard threshold) to suppress noise, and the threshold is dynamically set according to the standard deviation of the noise. The processed detail component is recombined with the approximation component, and the denoised signal is obtained through wavelet reconstruction.

[0100] In the process of eliminating noise artifacts, the system also combines directional filtering technology to perform directional processing on the detected artifact area. Based on the transient characteristics and local statistical characteristics of the signal, directional filtering technology adaptively adjusts the filter parameters to maximize the suppression of artifacts while retaining the natural characteristics of the speech. The processed signal is optimized through multiple iterations to obtain the target quality speech output signal.

[0101] This embodiment effectively improves the voice signal quality of the multi-microphone array beamforming system through a series of signal processing steps. Through the frame selective discarding algorithm, the quality of each frame signal can be accurately evaluated, and dynamic adjustments can be made according to the signal quality score to discard or replace low-quality frames, thereby ensuring the continuity and naturalness of the voice signal. This process effectively eliminates noise interference and artifacts, avoids traces of artificial synthesis, and makes the voice signal clearer and more natural. Through wavelet decomposition and reconstruction technology, noise suppression can be refined during the denoising process to improve the fidelity of the signal. In addition, combined with directional filtering technology, the system adaptively suppresses artifacts, effectively retains the original characteristics of the speech, and further improves the intelligibility of the speech.

[0102] Reference Figure 2 As shown, the present invention also provides a multi-microphone array beamforming signal enhancement device, which is applied to any multi-microphone array beamforming signal enhancement method, comprising: An acquisition module is used to acquire multi-channel time domain signals acquired by a multi-microphone array, perform time-frequency conversion, and obtain complex time-frequency domain signals; An analysis module, which is used to obtain the spatial distribution vector of the multi-microphone array, perform deformation compensation processing on the complex time-frequency domain signal, and obtain a sound field consistency signal; An association module is used to extract the sound source direction feature of the sound field consistency signal and perform complex spatiotemporal attention beamforming network processing to obtain a beamforming signal; A processing module, the processing module is used to optimize the beam direction and gain parameters of the beamforming signal through a preset adversarial environment simulator to obtain a dynamic frequency band enhancement signal; The control module is used to perform inverse time-frequency conversion on the dynamic frequency band enhancement signal and perform low-delay optimization to obtain a target quality voice output signal.

[0103] The present invention provides a multi-microphone array beamforming signal enhancement device, which can more accurately compensate for signal deformation by performing time-frequency conversion on the time domain signal collected by the multi-microphone array and combining the spatial distribution vector information of the array, thereby improving the quality of the sound field consistency signal. This method can effectively improve the extraction accuracy of the target speech signal by extracting the directional characteristics of the sound source and applying the complex spatiotemporal attention beamforming network, avoiding the occurrence of signal attenuation or distortion in the traditional signal enhancement method. By introducing a preset adversarial environment simulator, the beam direction and gain parameters are optimized, which can adapt to the dynamic and complex acoustic environment, realize the adaptive adjustment of the beam direction and gain parameters, and further improve the signal clarity and the accuracy of speech recognition. Based on this technology, noise interference and echo effects can be effectively reduced, thereby ensuring that high-quality speech output signals can be maintained in different environments, reducing the sound quality degradation and speech distortion problems in traditional methods. In addition, the low-latency optimization design of the dynamic frequency band enhancement signal contributes to the smooth operation of the real-time voice interaction system and enhances the robustness and user experience of the system. By comprehensively considering the spatiotemporal information and dynamic sound field characteristics, this method can flexibly adapt to complex and changeable acoustic environments, thereby improving the overall performance and application effect of the multi-microphone array system.

[0104] In this embodiment, the processor and the memory may be connected via a bus or other means. The memory may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk, or a solid-state drive. The processor may be a general-purpose processor, such as a central processing unit, a digital signal processor, an application-specific integrated circuit, or one or more integrated circuits configured to implement the embodiments of the present invention.

[0105] It should be noted that technicians in the relevant technical field can clearly understand that for the convenience and conciseness of description, the specific working process of the system and each module described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0106] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A multi-microphone array beamforming signal enhancement method, characterized in that: include: Acquire a multi-channel time domain signal collected by a multi-microphone array, perform time-frequency conversion, and obtain a complex time-frequency domain signal; Acquire the spatial distribution vector of the multi-microphone array, perform deformation compensation processing on the complex time-frequency domain signal, and obtain a sound field consistency signal; Extracting the sound source direction feature of the sound field consistency signal, and performing complex spatiotemporal attention beamforming network processing to obtain a beamforming signal; Optimizing the beam direction and gain parameters of the beamforming signal by using a preset adversarial environment simulator to obtain a dynamic frequency band enhancement signal; The dynamic frequency band enhancement signal is subjected to inverse time-frequency conversion and low-delay optimization to obtain a target quality speech output signal.

2. The multi-microphone array beamforming signal enhancement method according to claim 1, characterized in that: The method of acquiring a multi-channel time domain signal collected by a multi-microphone array and performing time-frequency conversion to obtain a complex time-frequency domain signal includes: Performing endpoint detection and noise power estimation on the multi-channel time domain signal to obtain a preprocessed signal; The pre-processed signal is framed and windowed according to a preset frame length and frame shift parameter to obtain an initial complex frequency domain signal; Performing spectrum smoothing on the initial complex frequency domain signal to obtain a smoothed frequency domain signal; Performing nonlinear sub-band division on the smoothed frequency domain signal to obtain a sub-band signal; Performing geometric phase compensation on the sub-band signal to obtain a sub-band correction signal; The sub-band correction signal is subjected to spectral subtraction denoising based on a preset noise power spectrum, and is subjected to complex time-frequency domain feature representation conversion to obtain the complex time-frequency domain signal.

3. The multi-microphone array beamforming signal enhancement method according to claim 1, characterized in that: The acquiring of the spatial distribution vector of the multi-microphone array and performing deformation compensation processing on the complex time-frequency domain signal to obtain a sound field consistency signal includes: Acquiring the physical coordinates and orientation parameters of the multi-microphone array, performing spatial construction, and obtaining the spatial distribution vector; Performing spatial transfer function mapping calculation on the complex time-frequency domain signal according to the spatial distribution vector to obtain array acoustic characteristic mapping information; Performing singular value decomposition on the array acoustic characteristic mapping information, and performing phase and amplitude nonlinear correction to obtain a preliminary compensation signal; Calculating a cross-power spectral density matrix based on the preliminary compensation signal, and constructing a sound field geometric consistency constraint to obtain a sound field topology representation; Performing complex domain tensor decomposition on the acoustic field topology representation to obtain a deformation compensation coefficient; The phase and amplitude of the complex time-frequency domain signal are aligned according to the deformation compensation coefficient to obtain the sound field consistency signal.

4. The multi-microphone array beamforming signal enhancement method according to claim 1, characterized in that: The sound source direction feature extraction is performed on the sound field consistency signal, and a complex spatiotemporal attention beamforming network is performed to obtain a beamforming signal, including: Performing adaptive wavelet packet transform decomposition on the sound field consistency signal to obtain a multi-level sound field time-frequency representation; Performing complex domain spatial covariance calculation on the multi-level sound field time-frequency representation to obtain a frequency band phase difference spectrum; Performing high-order singular value decomposition on the frequency band phase difference spectrum to obtain an orthogonal projection structure; Constructing a hyperbolic direction estimation function according to the orthogonal projection structure, and performing a gradient polarization search to obtain a sound source direction vector; Phase-aligning the sound source direction vector with the sound field consistency signal to obtain an initial beam gain coefficient; A complex domain attention mask is constructed according to the initial beam gain coefficient, and channel weighting is performed on the sound field consistency signal to obtain a beamforming signal.

5. The multi-microphone array beamforming signal enhancement method according to claim 1, characterized in that: The step of optimizing the beam direction and gain parameters of the beamforming signal by using a preset adversarial environment simulator to obtain a dynamic frequency band enhancement signal includes: Based on the adversarial environment simulator, a variety of noise interference and reverberation condition simulations are performed on the beamforming signal to obtain diversified environmental interference samples; Performing gradient descent optimization calculation on the diverse environmental interference samples, and adjusting beam direction parameters and frequency band gain coefficients according to a preset signal-to-noise ratio objective function to obtain a parameter optimization matrix; Reconstructing the directionality of the beamforming signal according to the parameter optimization matrix to obtain a directionally enhanced signal; According to the directional enhancement signal, acoustic feature matching is performed with preset voice data to obtain corresponding target actual short-term voice data; Based on a preset meta-learning online calibration algorithm, feature extraction is performed on the target actual short-term speech data to obtain a target feature adaptation matrix; Performing a convolution operation on the target feature adaptation matrix and the direction enhancement signal to obtain a spectrum correction signal; The spectrum correction signal is subjected to cross-band consistency processing and phase band gain to obtain the dynamic band enhanced signal.

6. The multi-microphone array beamforming signal enhancement method according to claim 5, characterized in that: The simulating various noise interference and reverberation conditions of the beamforming signal based on the adversarial environment simulator to obtain diversified environmental interference samples includes: Performing time-frequency transformation on the beamforming signal through the adversarial environment simulator to obtain a signal matrix represented in the time-frequency domain; Constructing a three-dimensional acoustic propagation structure according to the signal matrix represented in the time-frequency domain, and setting a random reflection coefficient to obtain a spatial reverberation parameter set; Randomly extract various types of noise sources from the preset environmental noise library, and perform time domain stretching and frequency domain shifting to obtain a set of deformed noise sources; Position mapping is performed on the deformation noise source set according to a preset spatial directional distribution function to obtain a multi-directional noise distribution matrix; Calculating acoustic propagation delay and energy attenuation according to the multi-directional noise distribution matrix and the spatial reverberation parameter set to obtain a noise propagation characteristic vector; The time-frequency domain representation signal matrix is ​​subjected to reverberation processing based on the spatial reverberation parameter set, and interference superposition is performed with the noise propagation characteristic vector to obtain the diversified environmental interference samples.

7. The multi-microphone array beamforming signal enhancement method according to claim 5, characterized in that: The preset meta-learning online calibration algorithm is used to extract features from the target actual short-term speech data to obtain a target feature adaptation matrix, including: Performing short-time window framing and windowing processing on the target actual short-time speech data to obtain an overlapping segmented signal sequence; Calculating Mel-frequency cepstrum coefficients of the overlapping segmented signal sequence to obtain a segmented feature description vector; Constructing a meta-learning task sampling set for the segmented feature description vector to obtain a feature support set and a feature query set; Based on the feature support set, a meta-learning inner loop optimization function is constructed to obtain environment adaptation parameters; Performing MAML gradient update on the feature query set according to the environmental adaptation parameters to obtain task model parameters; Performing meta-learning outer loop optimization and online learning rate adjustment on the task model parameters to obtain a dynamic learning rate matrix; Performing prototype network mapping on the segmented feature description vector according to the dynamic learning rate matrix to obtain an environmental perception feature representation; Performing metric learning embedding space construction on the environmental perception feature representation to obtain a speech feature embedding matrix; The speech feature embedding matrix is ​​subjected to Bayesian credible interval constraints to obtain a target feature adaptation matrix.

8. The multi-microphone array beamforming signal enhancement method according to claim 1, characterized in that: The inverse time-frequency conversion of the dynamic frequency band enhancement signal and low-delay optimization to obtain a target quality speech output signal includes: Performing overlap-add inverse transform processing on the dynamic frequency band enhancement signal to obtain an initial time domain signal; Performing inter-frame phase consistency compensation on the initial time domain signal to obtain a phase continuous intermediate signal; Dynamically adjust the buffer according to the phase-continuous intermediate signal to obtain a variable signal buffer; Performing nonlinear amplitude normalization on the variable signal buffer to obtain a dynamic range compression signal; Calculating a time domain transient feature vector according to the dynamic range compression signal, and performing time domain jitter elimination to obtain a jitter correction signal; Performing multi-stage cascade frequency band reconstruction on the jitter correction signal to obtain a frequency response equalization signal; Performing device pre-compensation processing on the frequency response equalization signal to obtain a device matching signal; Noise and artifact elimination is performed on the device matching signal according to a preset frame selective discarding algorithm to obtain the target quality speech output signal.

9. A multi-microphone array beamforming signal enhancement device, characterized in that: The multi-microphone array beamforming signal enhancement method applied to any one of claims 1 to 8 above comprises: An acquisition module, wherein the acquisition module is used to acquire a multi-channel time domain signal acquired by a multi-microphone array, perform time-frequency conversion, and obtain a complex time-frequency domain signal; An analysis module, the analysis module is used to obtain the spatial distribution vector of the multi-microphone array, perform deformation compensation processing on the complex time-frequency domain signal, and obtain a sound field consistency signal; An association module, the association module is used to extract the sound source direction feature of the sound field consistency signal, and perform complex spatiotemporal attention beamforming network processing to obtain a beamforming signal; A processing module, the processing module is used to optimize the beam direction and gain parameters of the beamforming signal through a preset confrontation environment simulator to obtain a dynamic frequency band enhancement signal; A control module is used to perform inverse time-frequency conversion on the dynamic frequency band enhancement signal and perform low-delay optimization to obtain a target quality speech output signal.

Images