Voice signal processing method and device, electronic equipment and storage medium

Abstract

The invention relates to a voice signal processing method and device, electronic equipment and a storage medium, and relates to the technical field of audio, and the method comprises the steps: obtaining a multi-channel far-field voice signal collected for a target sound source; inputting the multi-channel far-field voice signal into a preset multi-channel filtering model, and performing voice enhancement processing to obtain target speech spectrum information; the preset multi-channel filtering model is a filtering model obtained by adaptively updating an original multi-channel filtering model according to the multi-channel far-field voice signal, or a filtering model obtained by adaptively constructing a prediction result of the multi-channel far-field voice signal according to a first neural network model; and inputting the target speech spectrum information into a preset near-field speech generation model, and performing speech reconstruction processing to obtain a near-field speech signal corresponding to the multi-channel far-field speech signal. By using the technical scheme provided by the embodiment of the invention, the conversion of voice listening feeling can be realized, and the voice quality and the voice intelligibility are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of audio technology, and in particular to a voice signal processing method, device, electronic device and storage medium. Background Art

[0002] The development of communication technology has made it possible to conduct real-time voice communication in large spaces such as conference rooms and classrooms. In related applications, the use of far-field sound pickup technology based on microphone arrays can achieve real-time voice communication in far-field environments.

[0003] However, due to the fact that the speaker is far away from the microphone array and the room transmission characteristics of the far-field environment, the collected far-field speech has a significant difference in hearing from the speech collected in the near-field environment, and the speech quality of the collected far-field speech is low, which can easily cause auditory fatigue and reduce the efficiency of voice communication.

[0004] With the popularization of online real-time voice communication, there is an urgent need for an effective solution to convert voice picked up in a far-field environment into high-quality voice with a near-field effect in hearing. Summary of the invention

[0005] The present disclosure provides a speech signal processing method, device, electronic device and storage medium, which can convert speech picked up in a far-field environment into speech with a near-field effect in terms of auditory perception, improve the quality of speech and the intelligibility of speech. The technical solution of the present disclosure is as follows:

[0006] According to a first aspect of an embodiment of the present disclosure, there is provided a method for processing a speech signal, comprising:

[0007] Acquire multi-channel far-field speech signals collected from target sound sources;

[0008] The multi-channel far-field speech signal is input into a preset multi-channel filter model to perform speech enhancement processing to obtain target spectral information; the preset multi-channel filter model is a filter model obtained by adaptively updating the original multi-channel filter model according to the multi-channel far-field speech signal, or a filter model adaptively constructed according to the prediction result of the multi-channel far-field speech signal by the first neural network model;

[0009] The target spectrum information is input into a preset near-field speech generation model, and speech reconstruction processing is performed to obtain a near-field speech signal corresponding to the multi-channel far-field speech signal.

[0010] In an optional embodiment, the preset near-field speech generation model is a vocoder model obtained by training a generative adversarial network, the preset near-field speech generation model includes a generative network, and the target spectrum information is input into the preset near-field speech generation model to perform speech reconstruction processing to obtain a near-field speech signal corresponding to the multi-channel far-field speech signal, including:

[0011] Determine target Mel spectrum information corresponding to the target spectrum information;

[0012] Input the target Mel spectrum information into the input convolution layer of the generation network, perform upsampling processing, and obtain at least one original sampling point information; the at least one original sampling point information represents the amplitude and phase corresponding to at least one time-frequency point;

[0013] Inputting the at least one original sampling point information into the multi-receptive field fusion layer of the generation network, performing fusion processing based on the multi-receptive field, and obtaining at least one target sampling point information;

[0014] The at least one target sampling point information is input into the output convolution layer of the generation network, and time-frequency conversion processing is performed to obtain the near-field speech signal.

[0015] In an optional embodiment, the preset near-field speech generation model is an audio codec model based on a deep learning network, the preset near-field speech generation model includes an encoder, a quantizer and a decoder, and the target spectrum information is input into the preset near-field speech generation model to perform speech reconstruction processing to obtain the near-field speech signal corresponding to the multi-channel far-field speech signal, including:

[0016] Inputting the target spectral information into the encoder, performing feature encoding processing, and obtaining latent space feature information corresponding to the target spectral information;

[0017] Inputting the latent space feature information into the quantizer to perform feature quantization processing to obtain quantized feature information;

[0018] The quantized feature information is input into the decoder, and feature decoding processing is performed to obtain the near-field speech signal.

[0019] In an optional embodiment, the multi-channel far-field speech signal includes a plurality of far-field speech signals corresponding to a plurality of channels one by one, and the multi-channel far-field speech signal is input into a preset multi-channel filter model to perform speech enhancement processing to obtain target spectrum information, including:

[0020] Determining spectral information corresponding to each far-field speech signal among the multiple far-field speech signals;

[0021] Based on the preset multi-channel filtering model, determining a signal weight corresponding to each far-field speech signal;

[0022] A weighted summation process is performed according to the spectrum information corresponding to each far-field speech signal and the signal weight corresponding to each far-field speech signal to obtain the target spectrum information.

[0023] In an optional embodiment, the multi-channel far-field voice signal includes a plurality of far-field voice signals corresponding one-to-one to the plurality of channels, and the method further includes:

[0024] Determining spectral information corresponding to each far-field speech signal among the multiple far-field speech signals;

[0025] Based on the spectral information corresponding to each far-field speech signal, a model update parameter is calculated, where the model update parameter indicates the importance of each far-field speech signal;

[0026] Based on the model update parameters, the original multi-channel filtering model is adaptively updated to obtain the preset multi-channel filtering model.

[0027] In an optional embodiment, determining the spectral information corresponding to each far-field speech signal in the plurality of far-field speech signals includes:

[0028] Perform short-time Fourier transform processing on each of the multiple far-field speech signals to obtain spectral information corresponding to each far-field speech signal.

[0029] In an optional embodiment, the calculating the model update parameter based on the spectral information corresponding to each far-field speech signal includes:

[0030] Performing feature extraction processing on the spectral information corresponding to each far-field speech signal to obtain time-frequency feature information corresponding to each far-field speech signal;

[0031] Based on the time-frequency feature information corresponding to each far-field speech signal, a first spatial covariance matrix and a second spatial covariance matrix are calculated; the first spatial covariance matrix indicates the degree of correlation between each far-field speech signal; the second spatial covariance matrix indicates the degree of correlation between the noise of each far-field speech signal;

[0032] Based on the first spatial covariance matrix and the second spatial covariance matrix, obtaining a third spatial covariance matrix;

[0033] Performing eigendecomposition processing on the third spatial covariance matrix to obtain a steering vector, wherein the steering vector indicates a direction of a target far-field speech signal in the multi-channel far-field speech signal, wherein the target far-field speech signal is a far-field speech signal least affected by noise;

[0034] The model update parameter is calculated based on the steering vector and the second spatial covariance matrix.

[0035] In an optional embodiment, the calculating the model update parameter based on the spectral information corresponding to each far-field speech signal includes:

[0036] Performing feature extraction processing on the spectral information corresponding to each far-field speech signal to obtain time-frequency feature information corresponding to each far-field speech signal;

[0037] Based on the time-frequency feature information corresponding to each far-field speech signal, a first spatial covariance matrix and a second spatial covariance matrix are calculated; the first spatial covariance matrix indicates the degree of correlation between each far-field speech signal; the second spatial covariance matrix indicates the degree of correlation between the noise components of each far-field speech signal;

[0038] The first spatial covariance matrix and the second spatial covariance matrix are input into a second neural network model, and parameter prediction processing is performed to obtain the model update parameters.

[0039] In an optional embodiment, the first neural network model includes a feature representation network, a feature extraction network and a feature mapping network, and the method further includes:

[0040] Inputting the multi-channel far-field speech signal into the feature representation network, performing feature representation processing, and obtaining first speech feature data;

[0041] Inputting the first speech feature data into the feature extraction network to perform feature extraction processing to obtain second speech feature data;

[0042] Inputting the second speech feature data into the feature mapping network, performing feature mapping processing, and obtaining a prediction result, wherein the prediction result is a parameter for constructing a filtering model;

[0043] The preset multi-channel filtering model is constructed according to the prediction results.

[0044] In an optional embodiment, the method further includes:

[0045] Acquire a clean speech sample signal and a near-field speech reference signal corresponding to the clean speech sample signal;

[0046] Determine sample Mel spectrum information corresponding to the clean speech sample signal;

[0047] Inputting the sample Mel spectrum information into the current generation network, performing speech reconstruction processing, and obtaining the corresponding current near-field speech sample signal;

[0048] Determining a current first reconstruction loss data according to the current near-field speech sample signal and the near-field speech reference signal; the current first reconstruction loss data indicates a Manhattan distance between the current near-field speech sample signal and the near-field speech reference signal in a current training round;

[0049] Inputting the current near-field speech sample signal into the current identification network to perform speech identification processing to obtain a current first identification result, wherein the current first identification result indicates whether the current near-field speech sample signal is identified as a corresponding near-field speech reference signal in the current training round;

[0050] Based on the current first identification result, determining current second reconstruction loss data;

[0051] Determining current third reconstruction loss data according to the current first reconstruction loss data and the current second reconstruction loss data;

[0052] Training the current generation network according to the current third reconstruction loss data to obtain a next generation network;

[0053] Inputting the near-field speech reference signal and the current near-field speech sample signal into the current identification network, performing speech identification processing, and obtaining a current second identification result, wherein the current second identification result indicates the degree of accuracy of distinguishing the near-field speech reference signal and the current near-field speech sample signal in the current training round;

[0054] Based on the current second identification result, determining current first identification loss data;

[0055] Based on the current first identification loss data, training the current identification network to obtain a next identification network;

[0056] Using the next generation network as the current generation network of the next training round, and using the next identification network as the current identification network of the next training round, and looping through the above training process;

[0057] When the current generation network and the current identification network reach a preset convergence condition, the training is terminated to obtain the generation network and the identification network.

[0058] In an optional embodiment, the method further includes:

[0059] Acquire a multi-channel original voice sample signal and a clean voice sample signal corresponding to the multi-channel original voice sample signal;

[0060] Input each of the multi-channel original speech sample signals into an initial multi-channel filtering model to perform speech enhancement processing to obtain first sample spectral information;

[0061] Determining second sample spectrum information corresponding to the clean speech sample signal;

[0062] Based on the first sample spectral information and the second sample spectral information, determining first enhancement loss data; the first enhancement loss data indicates the degree of spectral difference between the first sample spectral information and the second sample spectral information in the norm space;

[0063] The initial multi-channel filtering model is trained based on the first enhancement loss data to obtain the original multi-channel filtering model.

[0064] According to a second aspect of an embodiment of the present disclosure, there is provided a voice signal output device, comprising:

[0065] An acquisition module is configured to acquire a multi-channel far-field speech signal collected from a target sound source;

[0066] The speech enhancement module is configured to input the multi-channel far-field speech signal into a preset multi-channel filter model, perform speech enhancement processing, and obtain target spectral information; the preset multi-channel filter model is a filter model obtained by adaptively updating the original multi-channel filter model according to the multi-channel far-field speech signal, or a filter model adaptively constructed according to the prediction result of the first neural network model for the multi-channel far-field speech signal;

[0067] The speech reconstruction module is configured to input the target speech spectrum information into a preset near-field speech generation model, perform speech reconstruction processing, and obtain a near-field speech signal corresponding to the multi-channel far-field speech signal.

[0068] In an optional embodiment, the preset near-field speech generation model is a vocoder model obtained by training a generative adversarial network, the preset near-field speech generation model includes a generative network, and the speech reconstruction module includes:

[0069] A mel spectrum determination unit is configured to determine target mel spectrum information corresponding to the target spectrum information;

[0070] An upsampling unit is configured to input the target Mel spectrum information into the input convolution layer of the generating network, perform upsampling processing, and obtain at least one original sampling point information; the at least one original sampling point information represents the amplitude and phase corresponding to at least one time-frequency point;

[0071] A multi-receptive field fusion unit is configured to input the at least one original sampling point information into the multi-receptive field fusion layer of the generation network, perform fusion processing based on multiple receptive fields, and obtain at least one target sampling point information;

[0072] The time-frequency conversion unit is configured to input the at least one target sampling point information into the output convolution layer of the generation network, perform time-frequency conversion processing, and obtain the near-field speech signal.

[0073] In an optional embodiment, the preset near-field speech generation model is an audio codec model based on a deep learning network, the preset near-field speech generation model includes an encoder, a quantizer and a decoder, and the speech reconstruction module includes:

[0074] A feature encoding unit is configured to input the target spectral information into the encoder, perform feature encoding processing, and obtain latent space feature information corresponding to the target spectral information;

[0075] A feature quantization unit is configured to input the latent space feature information into the quantizer, perform feature quantization processing, and obtain quantized feature information;

[0076] The feature decoding unit is configured to input the quantized feature information into the decoder, perform feature decoding processing, and obtain the near-field speech signal.

[0077] In an optional embodiment, the multi-channel far-field speech signal includes a plurality of far-field speech signals corresponding one-to-one to a plurality of channels, and the speech enhancement module includes:

[0078] A first signal spectrum determination unit is configured to determine spectrum information corresponding to each far-field speech signal in the plurality of far-field speech signals;

[0079] A signal weight determination unit, configured to determine a signal weight corresponding to each far-field speech signal based on the preset multi-channel filtering model;

[0080] The weighting unit is configured to perform weighted summation processing according to the spectrum information corresponding to each far-field speech signal and the signal weight corresponding to each far-field speech signal to obtain the target spectrum information.

[0081] In an optional embodiment, the multi-channel far-field voice signal includes a plurality of far-field voice signals corresponding one-to-one to the plurality of channels, and the apparatus further includes:

[0082] A second signal spectrum determination unit is configured to determine spectrum information corresponding to each far-field speech signal in the plurality of far-field speech signals;

[0083] A model update parameter calculation unit is configured to calculate a model update parameter based on the spectrum information corresponding to each far-field speech signal, wherein the model update parameter indicates the importance of each far-field speech signal;

[0084] The adaptive updating unit is configured to perform adaptive updating on the original multi-channel filtering model based on the model updating parameters to obtain the preset multi-channel filtering model.

[0085] In an optional embodiment, either the first signal spectrum determining unit or the second signal spectrum determining unit includes:

[0086] The short-time Fourier transform subunit is configured to perform short-time Fourier transform processing on each of the multiple far-field speech signals to obtain spectrum information corresponding to each far-field speech signal.

[0087] In an optional embodiment, the model update parameter calculation unit includes:

[0088] A first feature extraction subunit is configured to perform feature extraction processing on the spectral information corresponding to each far-field speech signal to obtain time-frequency feature information corresponding to each far-field speech signal;

[0089] The first covariance calculation subunit is configured to calculate a first spatial covariance matrix and a second spatial covariance matrix based on the time-frequency feature information corresponding to each far-field speech signal; the first spatial covariance matrix indicates the degree of correlation between each far-field speech signal; the second spatial covariance matrix indicates the degree of correlation between the noise of each far-field speech signal;

[0090] A second covariance calculation subunit is configured to obtain a third spatial covariance matrix based on the first spatial covariance matrix and the second spatial covariance matrix;

[0091] an eigendecomposition subunit, configured to perform eigendecomposition processing on the third spatial covariance matrix to obtain a steering vector, wherein the steering vector indicates a direction of a target far-field speech signal in the multi-channel far-field speech signal, wherein the target far-field speech signal is a far-field speech signal least affected by noise;

[0092] The model update parameter first calculation subunit is configured to calculate the model update parameter based on the steering vector and the second spatial covariance matrix.

[0093] In an optional embodiment, the model update parameter calculation unit includes:

[0094] A first feature extraction subunit is configured to perform feature extraction processing on the spectral information corresponding to each far-field speech signal to obtain time-frequency feature information corresponding to each far-field speech signal;

[0095] The third covariance calculation subunit is configured to calculate a first spatial covariance matrix and a second spatial covariance matrix based on the time-frequency feature information corresponding to each far-field speech signal; the first spatial covariance matrix indicates the degree of correlation between each far-field speech signal; the second spatial covariance matrix indicates the degree of correlation between the noise components of each far-field speech signal;

[0096] The second subunit of the model update parameter is configured to input the first spatial covariance matrix and the second spatial covariance matrix into a second neural network model, perform parameter prediction processing, and obtain the model update parameter.

[0097] In an optional embodiment, the device further comprises:

[0098] A feature representation unit is configured to input the multi-channel far-field speech signal into the feature representation network, perform feature representation processing, and obtain first speech feature data;

[0099] A feature extraction unit is configured to input the first speech feature data into the feature extraction network, perform feature extraction processing, and obtain second speech feature data;

[0100] A feature mapping unit is configured to input the second speech feature data into the feature mapping network, perform feature mapping processing, and obtain a prediction result, wherein the prediction result is a filter model construction parameter;

[0101] The model building unit is configured to build the preset multi-channel filtering model according to the prediction result.

[0102] In an optional embodiment, the device further comprises:

[0103] A first training data acquisition unit is configured to acquire a clean speech sample signal and a near-field speech reference signal corresponding to the clean speech sample signal;

[0104] A sample Mel spectrum determination unit is configured to determine the sample Mel spectrum information corresponding to the clean speech sample signal;

[0105] A sample speech reconstruction unit is configured to input the sample Mel spectrum information into the current generation network, perform speech reconstruction processing, and obtain a corresponding current near-field speech sample signal;

[0106] A first reconstruction loss data determining unit is configured to determine a current first reconstruction loss data according to the current near-field speech sample signal and the near-field speech reference signal; the current first reconstruction loss data indicates a Manhattan distance between the current near-field speech sample signal and the near-field speech reference signal in a current training round;

[0107] A first identification result determination unit is configured to input the current near-field speech sample signal into a current identification network, perform speech identification processing, and obtain a current first identification result, wherein the current first identification result indicates whether the current near-field speech sample signal is identified as a corresponding near-field speech reference signal in a current training round;

[0108] A second reconstruction loss data determining unit is configured to determine current second reconstruction loss data based on the current first identification result;

[0109] A third reconstruction loss data determining unit is configured to determine current third reconstruction loss data according to the current first reconstruction loss data and the current second reconstruction loss data;

[0110] The first training unit is configured to train the current generation network according to the current third reconstruction loss data to obtain a next generation network.

[0111] A second identification result determination unit is configured to input the near-field speech reference signal and the current near-field speech sample signal into the current identification network, perform speech identification processing, and obtain a current second identification result, wherein the current second identification result indicates the accuracy of distinguishing the near-field speech reference signal and the current near-field speech sample signal in the current training round;

[0112] A first identification loss data determining unit, configured to determine current first identification loss data based on the current second identification result;

[0113] A second training unit is configured to perform training on the current identification network based on the current first identification loss data to obtain a next identification network;

[0114] A loop unit, configured to execute the next generation network as the current generation network of the next training round, and the next identification network as the current identification network of the next training round, and cyclically execute the above training process;

[0115] The judgment unit is configured to terminate the training when the current generation network and the current identification network reach a preset convergence condition, thereby obtaining the generation network and the identification network.

[0116] In an optional embodiment, the device further comprises:

[0117] A second training data acquisition unit is configured to acquire a multi-channel original speech sample signal and a clean speech sample signal corresponding to the multi-channel original speech sample signal;

[0118] A sample speech enhancement unit is configured to input each of the multi-channel original speech sample signals into an initial multi-channel filtering model, perform speech enhancement processing, and obtain first sample spectral information;

[0119] A sample signal spectrum determination unit is configured to determine second sample spectrum information corresponding to the clean speech sample signal;

[0120] A first enhancement loss data determining unit is configured to determine first enhancement loss data based on the first sample spectral information and the second sample spectral information; the first enhancement loss data indicates a degree of spectral difference between the first sample spectral information and the second sample spectral information in a norm space;

[0121] The third training unit is configured to perform training on the initial multi-channel filtering model based on the first enhanced loss data to obtain the original multi-channel filtering model.

[0122] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the speech signal processing method described in any one of the first aspect of the embodiments of the present disclosure.

[0123] According to the fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform a speech signal processing method as described in any one of the first aspect of the embodiments of the present disclosure.

[0124] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising computer instructions, which, when executed by a processor, implement a speech signal processing method as described in any one of the first aspect of an embodiment of the present disclosure.

[0125] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0126] The disclosed embodiment inputs the multi-channel far-field speech signal collected from the target sound source into a preset multi-channel filter model, performs speech enhancement processing, and obtains target spectral information; wherein the preset multi-channel filter model is a filter model obtained by adaptively updating the original multi-channel filter model according to the multi-channel far-field speech signal, or is a filter model obtained by adaptively constructing the prediction result of the multi-channel far-field speech signal according to the first neural network model, which can more adaptively perform speech enhancement processing such as noise reduction and reverberation on the multi-channel remote speech signal, thereby improving the speech quality of the target spectral information; the disclosed embodiment inputs the target spectral information into a preset near-field speech generation model, performs speech reconstruction processing, adjusts the amplitude response and / or phase response of different frequencies in the target spectral information, realizes the change in speech hearing perception, and can improve the speech damage that may be caused by the preset multi-channel filter model after speech enhancement processing, thereby obtaining a near-field speech signal corresponding to the multi-channel far-field speech signal, thereby improving the speech quality and speech intelligibility, and meeting the needs of more voice communications and applications.

[0127] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0128] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0129] Figure 1 is a schematic diagram of an application environment according to an exemplary embodiment;

[0130] Figure 2 is a flow chart of a method for processing a speech signal according to an exemplary embodiment;

[0131] Figure 3 is a flow chart showing a speech enhancement process according to an exemplary embodiment;

[0132] Figure 4 is a flow chart showing a method of adaptively updating an original multi-channel filtering model according to an exemplary embodiment;

[0133] Figure 5is a flow chart showing a speech reconstruction process according to an exemplary embodiment;

[0134] Figure 6 is a flowchart of another speech reconstruction process according to an exemplary embodiment;

[0135] Figure 7 is a flowchart of a model training method according to an exemplary embodiment;

[0136] Figure 8 is a flowchart of an adversarial training method according to an exemplary embodiment;

[0137] Fig. 9 is a flow chart of a method for processing a speech signal in a specific application according to an exemplary embodiment;

[0138] Fig.10 is a block diagram of a speech signal processing device according to an exemplary embodiment;

[0139] Fig.11 is a block diagram of a terminal for performing voice signal processing according to an exemplary embodiment;

[0140] Fig.12 The invention is a block diagram of a server for performing speech signal processing according to an exemplary embodiment. DETAILED DESCRIPTION

[0141] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0142] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0143] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0144] In order to facilitate understanding of the technical solutions described in the embodiments of the present disclosure and the technical effects produced therefrom, the embodiments of the present disclosure explain the relevant professional terms involved:

[0145] Reverberation: refers to the phenomenon that voice signals encounter obstacles such as walls, ceilings, and floors to form reflected sounds, which are superimposed on the direct sounds.

[0146] STFT: short-time Fourier transform or short-term Fourier transform, full name short-time Fourier transform, is a mathematical transform closely related to Fourier transform, specifically used to analyze the frequency and phase characteristics of time-varying signals in local areas. This transform converts a long non-stationary signal into a short stationary signal matrix by adding a window function to the time domain of the signal, and then performs a discrete Fourier transform on the signal matrix weighted by the window function to obtain a time-frequency spectrum matrix. STFT can provide the frequency information of the signal at a specific time point, making up for the defect that Fourier transform cannot provide time resolution, making it possible to analyze non-stationary signals. STFT is widely used in audio signal processing, speech recognition and other fields, and its results can be intuitively displayed as a sound spectrum diagram, that is, the square of the modulus value of the STFT energy.

[0147] Beamforming: The beamforming described in the embodiments of the present disclosure specifically refers to digital beamforming. Digital beam forming (DBF) is a type of beamforming technology that uses digital signal processing technology to process the signals received by the array antenna, and achieves in-phase superposition by compensating for the phase difference caused by the propagation path difference caused by the different spatial positions of the sensors, thereby forming a beam with maximum energy reception in that direction. DBF technology is widely used in applications that require high directivity and high resolution, such as radar, sonar, and wireless communications.

[0148] See also Figure 1 , which is a schematic diagram of an application environment of a speech signal processing method according to an exemplary embodiment, the application environment may include a terminal 110 and a microphone array 120, and the terminal 110 and the microphone array 120 may be connected via a wired network or a wireless network.

[0149] The terminal 110 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc. The terminal 110 may be an all-in-one device equipped with a microphone array 120. An application (Application, referred to as App) may be installed in the terminal 110. The application may be an independent application or a subprogram in an independent application. The user of the terminal 110 may log in to the application using pre-registered user information, which may include an account number and a password.

[0150] The microphone array 120 may also be referred to as a microphone array, which refers to an orderly arrangement of multiple microphones. It is composed of a certain number of microphones and is a system used to sample and process the sound field. The microphone array 120 has various configurations, and can be divided into linear arrays, planar arrays, and spatial arrays, etc. according to the geometric configuration. The embodiments of the present disclosure are not limited to this. The multi-channel speech signal collected by the microphone array 120 retains the spatial phase characteristics of the signal, providing more available information for dereverberation and echo cancellation.

[0151] In a specific embodiment, the microphone array 120 can be used to collect multi-channel far-field voice signals of the target sound source in the space. The microphone array 120 transmits the collected multi-channel far-field voice signals to the terminal 110. The terminal 110 inputs the received multi-channel far-field voice signal into a preset multi-channel filter model, performs voice enhancement processing, and obtains the target spectrum information, wherein the preset multi-channel filter model is a filter model obtained by adaptively updating the original multi-pain filter model according to the multi-channel far-field voice signal, or a filter model obtained by adaptively constructing the prediction result of the multi-channel far-field voice signal according to the first neural network model; the terminal 110 inputs the target spectrum information into a preset near-field voice generation model, performs voice reconstruction processing, and obtains the near-field voice signal corresponding to the multi-channel far-field voice signal, thereby realizing a change in the sense of hearing. The terminal 110 can apply the near-field voice signal to voice real-time communication services to achieve close interaction with users outside the space.

[0152] In a specific embodiment, the microphone array 120 transmits the collected multi-channel far-field voice signals to the server, so that the server processes the multi-channel far-field voice signals to obtain corresponding near-field voice signals, and distributes the near-field voice signals to all terminals in the voice communication service.

[0153] Figure 2 is a flow chart of a method for processing a speech signal according to an exemplary embodiment. Figure 2 As shown, the speech signal processing method may include the following steps:

[0154] In step S210, a multi-channel far-field speech signal collected from a target sound source is obtained.

[0155] The voice signal processing method provided in the embodiment of the present disclosure is used to convert far-field voice signals into near-field voice signals, thereby achieving a change in the auditory perception of the voice signals, thereby optimizing the user's auditory experience and voice communication effect, and can provide services for increasingly popular voice applications.

[0156] In the embodiments of the present disclosure, the far-field voice signal refers to the voice signal generated by speaking from a position far away from the position of the sound pickup device. A microphone array can be used as a sound pickup device to perform multi-channel data acquisition of the voice of the target sound source. The acquired multi-channel far-field voice signal includes a far-field voice signal corresponding to each of the multiple channels. The multi-channel far-field voice signal can effectively and accurately determine the relative distance and relative angle of the target sound source, thereby achieving the positioning and tracking of the target sound source and the directional pickup of voice in the far-field environment.

[0157] In a specific embodiment, the target sound source includes at least one sound emitter.

[0158] In step S230, the multi-channel far-field speech signal is input into a preset multi-channel filter model to perform speech enhancement processing to obtain target spectral information.

[0159] In the embodiments of the present disclosure, it is understandable that in a far-field environment, the target sound source is far away from the sound pickup device and is in a movable state, and the collected voice signal is attenuated, scattered and reflected. In addition, the environment is noisy and there are many interference signals, which leads to the reduction of the voice energy reaching the sound pickup device and drowning in the background noise and reverberation, reducing the voice quality and intelligibility, which will cause auditory fatigue for a long time, and the voice recognition results are poor. Therefore, in order to improve the quality of speech, the embodiments of the present disclosure input the multi-channel far-field speech signal into a preset multi-channel filtering model and perform speech enhancement processing to perform time alignment, noise and reverberation removal, and other processing that can improve the voice quality of the multi-channel far-field speech signal.

[0160] In an embodiment of the present disclosure, the preset multi-channel filtering model is a filtering model obtained by adaptively updating the original multi-channel filtering model according to the multi-channel far-field speech signal, or is a filtering model adaptively constructed according to the prediction result of the first neural network model for the multi-channel far-field speech signal.

[0161] In a specific embodiment, a multi-channel far-field speech signal can provide richer spatial information than a single channel. During speech enhancement processing, by analyzing the differences and correlations between the multi-channel far-field speech signals, the speech source can be more accurately located and separated, thereby improving the clarity of the speech.

[0162] In a specific embodiment, multi-channel far-field speech signals from different microphones can be used to reduce the impact of environmental noise. During the speech enhancement process, by reasonably weighting and combining the multi-channel far-field speech signals, noise can be suppressed and the clarity and intelligibility of the speech can be improved.

[0163] In a specific embodiment, the multi-channel far-field speech signal can be subjected to interference suppression through an adaptive algorithm. During the speech enhancement process, the time, frequency, phase and other information of the multi-channel far-field speech signal can be used to model and remove noise and interference, thereby improving the purity and reliability of the speech signal.

[0164] In the disclosed embodiment, the target spectrum information represents the spectrum features of the multi-channel far-field speech signal after speech enhancement processing, and can be represented in the format of a spectrogram, which is also called a spectrogram. A spectrogram is a graph showing the change of the spectrum of a speech signal over time, with time as the horizontal axis and frequency as the vertical axis. The strength of speech at any given frequency component at a given moment is represented by the grayscale or hue of the corresponding point, where the strength of speech corresponds to the energy of speech data, which can be represented by the amplitude of the speech signal or the real and imaginary parts of the complex spectrum of the speech signal.

[0165] In a specific embodiment, the multi-channel far-field speech signal includes a plurality of far-field speech signals corresponding to a plurality of channels one by one, and the speech enhancement processing performed by the preset multi-channel filtering model can be as follows: Figure 3 As shown, including:

[0166] In step S310, spectral information corresponding to each far-field speech signal among a plurality of far-field speech signals is determined.

[0167] Step S310 performs time-frequency conversion on each far-field speech signal represented by a time domain waveform to obtain the spectral information corresponding to each far-field speech signal. The spectral information corresponding to each far-field speech signal can indicate the change of the speech spectrum of the corresponding far-field speech signal over time.

[0168] In a specific embodiment, multiple far-field speech signals may be time-aligned first, and then each of the aligned multiple far-field speech signals may be time-converted to obtain the spectral information corresponding to each far-field speech signal.

[0169] In step S320, based on a preset multi-channel filtering model, a signal weight corresponding to each far-field speech signal is determined.

[0170] The preset multi-channel filter model is a filter model obtained by adaptively updating the original multi-channel filter model according to the multi-channel far-field speech signal, or a filter model obtained by adaptively constructing the prediction result of the multi-channel far-field speech signal according to the first neural network model. The model parameters in the preset multi-channel filter model can indicate the signal weight corresponding to each channel, that is, the signal weight corresponding to each far-field speech signal. The signal weight corresponding to each far-field speech signal indicates the importance of the corresponding far-field speech signal in the multi-channel far-field speech signal.

[0171] In step S330, a weighted summation process is performed based on the spectrum information corresponding to each far-field speech signal and the signal weight corresponding to each far-field speech signal to obtain the target spectrum information.

[0172] In a specific embodiment, a preset multi-channel filtering model performs speech enhancement processing on a multi-channel far-field speech signal based on the principle of beamforming. Beamforming is mainly aimed at the microphone array, and by fusing the speech data of multiple channels, the noise and interference directions are suppressed, and the speech signal in the direction of the target sound source is enhanced. That is, by weighted summing the spectral information corresponding to each far-field speech signal, the speech component corresponding to the target sound source in the obtained target spectral information is enhanced, while the noise component, echo component or reverberation component is suppressed.

[0173] In the above embodiment, by performing weighted summation processing on the spectral information corresponding to each far-field speech signal and the signal weight corresponding to each far-field speech signal, the speech component corresponding to the target sound source is enhanced, and the noise component, echo component or reverberation component can be effectively removed to improve the speech quality corresponding to the target spectral information.

[0174] In a specific embodiment, the first neural network model includes a feature representation network, a feature extraction network, and a feature mapping network, and adaptively constructing the prediction result of the multi-channel far-field speech signal according to the first neural network model may include:

[0175] The multi-channel far-field speech signal is input into a feature representation network, and feature representation processing is performed to obtain first speech feature data.

[0176] The first speech feature data is input into a feature extraction network, and feature extraction processing is performed to obtain second speech feature data.

[0177] The second speech feature data is input into a feature mapping network, and feature mapping processing is performed to obtain a prediction result, where the prediction result is a parameter for constructing a filtering model.

[0178] A preset multi-channel filtering model is constructed based on the prediction results.

[0179] Specifically, the first speech feature data is feature data obtained after sampling, quantization, encoding, and embedding representation of a multi-channel far-field speech signal; the feature extraction network may include multiple hidden layers, and through feature extraction processing, the input first speech feature data can be mapped to a higher-level feature representation, thereby realizing nonlinear processing of the first speech feature data to obtain second speech feature data; the feature mapping network is used to nonlinearly map the input second speech feature data into multiple model parameter prediction values.

[0180] In the above embodiment, the model parameters of the multi-channel far-field speech signal are predicted by a neural network model, so that the preset multi-channel filter model constructed according to the prediction results can perform targeted speech enhancement processing on the multi-channel far-field speech signal, effectively improving the speech quality and also improving the construction efficiency of the preset multi-channel filter model.

[0181] In a specific embodiment, Figure 4 As shown, the adaptive updating of the original multi-channel filtering model may include the following steps:

[0182] In step S410, spectral information corresponding to each far-field speech signal among a plurality of far-field speech signals is determined.

[0183] Step S410 may refer to step S310 in the aforementioned embodiment, and will not be described in detail here.

[0184] In step S420, based on the spectral information corresponding to each far-field speech signal, a model update parameter is calculated, where the model update parameter indicates the importance of each far-field speech signal.

[0185] In a specific embodiment, the spectral information corresponding to each far-field speech signal can be analyzed based on the maximum signal-to-noise ratio criterion, the minimum mean square error criterion, the linear constrained minimum variance criterion, the maximum likelihood criterion or by using a neural network, etc., to determine the signal existence probability corresponding to each far-field speech signal. The signal existence probability can characterize the importance of the corresponding far-field speech signal, so that the model update parameters can be adaptively determined according to the signal existence probability corresponding to each far-field speech signal, so that an ideal speech quality gain can be obtained after speech enhancement processing by a preset multi-channel filtering model.

[0186] In step S430, the original multi-channel filtering model is adaptively updated based on the model update parameters to obtain a preset multi-channel filtering model.

[0187] In a specific embodiment, the preset multi-channel filter model obtained after adaptive updating can directly perform speech enhancement processing on the multi-channel far-field speech signal collected subsequently.

[0188] In the above embodiment, based on the spectral statistical characteristics of each far-field speech signal itself, the parameters of the original multi-channel filtering model are adaptively updated so that the obtained preset multi-channel filtering model can perform targeted speech enhancement processing on the multi-channel far-field speech signal according to the current far-field environment, thereby effectively improving the speech quality.

[0189] In a specific embodiment, in steps S310 and S410, each of the multiple far-field speech signals may be subjected to short-time Fourier transform processing to obtain the spectral information corresponding to each far-field speech signal. The short-time Fourier transform processing includes framing, windowing, and Fourier transforming each far-field speech signal to convert the far-field speech signal in the time domain into an amplitude spectrum or a complex spectrum.

[0190] In the above embodiment, the far-field speech signal in the time domain is converted into an amplitude spectrum or a complex spectrum by means of short-time Fourier transform, thereby retaining the time, frequency, phase, amplitude and other characteristics of the far-field speech signal, and providing a good data basis for speech enhancement and speech reconstruction.

[0191] In a specific embodiment, step S420 may be implemented as follows:

[0192] In step S421, feature extraction processing is performed on the spectral information corresponding to each far-field speech signal to obtain time-frequency feature information corresponding to each far-field speech signal.

[0193] In a specific embodiment, the spectral information corresponding to each far-field speech signal is discretized, and the spectral data corresponding to a plurality of preset time points and a plurality of preset frequency points are extracted as the time-frequency feature information.

[0194] In step S423, based on the time-frequency feature information corresponding to each far-field speech signal, a first spatial covariance matrix and a second spatial covariance matrix are calculated; the first spatial covariance matrix indicates the degree of correlation between each far-field speech signal; the second spatial covariance matrix indicates the degree of correlation between the noise of each far-field speech signal.

[0195] It is feasible to calculate the covariance between the time-frequency feature information corresponding to each far-field speech signal and obtain the first spatial covariance matrix.

[0196] It is feasible to predict the probability of containing only noise for each component in the time-frequency feature information corresponding to each far-field speech signal, and update the time-frequency feature information corresponding to each far-field speech signal according to the predicted probability value; for the updated time-frequency feature information corresponding to each far-field speech signal, calculate the covariance between the two to obtain the second spatial covariance matrix.

[0197] In step S425, a third spatial covariance matrix is ​​obtained based on the first spatial covariance matrix and the second spatial covariance matrix.

[0198] It is feasible that, when the row and column dimensions of the first spatial covariance matrix and the second spatial covariance matrix are the same, the components located in the same row and column in the first spatial covariance matrix and the second spatial covariance matrix are subtracted to obtain a third spatial covariance matrix. The third spatial covariance matrix can indicate the degree of correlation between the pure speech in each far-field speech signal to a certain extent.

[0199] In step S427, the third spatial covariance matrix is ​​subjected to eigendecomposition processing to obtain a steering vector, which indicates the direction of a target far-field speech signal in the multi-channel far-field speech signal, and the target far-field speech signal is the far-field speech signal least affected by noise.

[0200] It is feasible to perform eigendecomposition processing on the third space covariance matrix, and obtain the main eigenvector as a steering vector. Determination of the steering vector helps to perform phase compensation on other far-field speech signals, thereby enhancing the target far-field speech signal.

[0201] In another specific embodiment, the steering vector may also be calculated based on the spatial phase difference between each far-field speech signal.

[0202] In step S429, model update parameters are calculated based on the steering vector and the second spatial covariance matrix.

[0203] In a specific embodiment, matrix calculation is performed based on the steering vector and the second spatial covariance matrix to obtain model update parameters, and the model update parameters can indicate the importance of each far-field speech signal to adaptively update the original multi-channel filter model.

[0204] In the above embodiment, the model update parameters that can indicate the importance of each far-field voice signal are determined according to the time-frequency characteristics of each far-field voice signal itself, so as to realize the adaptive update of the original multi-channel filtering model, thereby improving the adaptability of the preset multi-channel filtering model obtained after the update and the collected multi-channel far-field voice signal, and can effectively improve the voice enhancement effect of the multi-channel far-field voice signal.

[0205] In a specific embodiment, step S420 may be implemented as follows:

[0206] In step S422, feature extraction processing is performed on the spectral information corresponding to each far-field speech signal to obtain time-frequency feature information corresponding to each far-field speech signal.

[0207] In step S424, based on the time-frequency feature information corresponding to each far-field speech signal, a first spatial covariance matrix and a second spatial covariance matrix are calculated; the first spatial covariance matrix indicates the degree of correlation between each far-field speech signal; the second spatial covariance matrix indicates the degree of correlation between the noise components of each far-field speech signal.

[0208] Step S422 and step S424 may refer to the aforementioned embodiment and will not be described in detail here.

[0209] In step S426, the first spatial covariance matrix and the second spatial covariance matrix are input into the second neural network model, and parameter prediction processing is performed to obtain model update parameters.

[0210] In a specific embodiment, the second neural network model can be obtained by training a recurrent convolutional network or a deep learning network using training sample data, and can directly output model update parameters that can indicate the importance of each far-field speech signal based on the input first spatial covariance matrix and the second spatial covariance matrix.

[0211] In the above embodiment, the second neural network model is used to predict the model update parameters that can indicate the importance of each far-field speech signal, which can improve the adaptability of the model update parameters to the multi-channel far-field speech signal, and then improve the adaptability of the preset multi-channel filtering model obtained after the update to the collected multi-channel far-field speech signal, which can effectively improve the speech enhancement effect on the multi-channel far-field speech signal.

[0212] In step S250, the target spectrum information is input into a preset near-field speech generation model, and speech reconstruction processing is performed to obtain a near-field speech signal corresponding to the multi-channel far-field speech signal.

[0213] Generally, near-field voice signals refer to voice signals received from a relatively close distance, such as voice signals received from microphones built into mobile phones, televisions, computers, and other devices. In the disclosed embodiment, the near-field voice signals corresponding to the multi-channel far-field voice signals obtained through voice reconstruction processing are not voice signals collected from a relatively close distance to the target sound source, but they also have a near-field effect in terms of hearing, which can avoid auditory fatigue of the listener and enhance the listener's auditory experience.

[0214] In the disclosed embodiment, the target spectral information is input into a preset near-field speech generation model for speech reconstruction processing. On the one hand, the target spectral information can be converted into a speech signal of a time domain waveform. On the other hand, the speech reconstruction function of the model is used to adjust the amplitude response and / or phase response of different frequencies in the target spectral information, thereby eliminating the influence of the spatial transmission performance of the far-field environment itself on the hearing of the speech signal in the process of collecting multi-channel far-field speech signals, thereby obtaining a speech signal with a near-field hearing sense. At the same time, in the process of adjusting the amplitude response and / or phase response of different frequencies in the target spectral information, the excessive suppression of the amplitude and / or phase corresponding to the time-frequency point with a low signal-to-noise ratio in the speech enhancement stage can also be corrected, or residual noise can also be removed.

[0215] In a specific embodiment, the preset near-field speech generation model can be a vocoder model obtained by targeted training of a generative adversarial network such as HiFiGAN or NSF-HiFiGAN, or it can be an audio codec model (codec) based on a deep learning network, etc.

[0216] In a specific embodiment, Figure 5 As shown, the preset near-field speech generation model is a vocoder model obtained by training a generative adversarial network. The preset near-field speech generation model includes a generation network. Step S250 can be implemented as follows:

[0217] In step S510, target mel-spectrogram information corresponding to target speech spectrum information is determined.

[0218] Considering that the frequencies in the target spectral information are distributed linearly, but the human ear perceives frequencies in a logarithmic manner, that is, it is sensitive to changes in the low-frequency band and insensitive to changes in the high-frequency band, the target spectral information with linear frequency distribution will appear "not useful enough" in the process of feature extraction. Therefore, in order to adapt to the working principle of the vocoder model, the target spectral information needs to be converted into the corresponding target Mel-spectrum spectrum information. Specifically, the frequency values ​​in the target spectral information are logarithmically calculated to obtain the corresponding Mel-frequency.

[0219] In step S520, the target Mel spectrum information is input into the input convolution layer of the generation network and up-sampled to obtain at least one original sampling point information.

[0220] The at least one original sampling point information represents the amplitude and phase corresponding to at least one time-frequency point.

[0221] In step S530, at least one original sampling point information is input into the multi-receptive field fusion layer of the generation network, and a fusion process based on multiple receptive fields is performed to obtain at least one target sampling point information.

[0222] In step S540, at least one target sampling point information is input into the output convolution layer of the generation network, and a time-frequency conversion process is performed to obtain a near-field speech signal.

[0223] The generated network mainly includes input convolution layer, multi-receptive field fusion layer and output convolution layer. The input convolution layer is specifically composed of one-dimensional transposed convolution; the multi-receptive field fusion layer is specifically composed of residual network, which is mainly responsible for optimizing at least one original sampling point information obtained by upsampling. The multi-receptive field fusion layer is a structure that uses dilated convolution and ordinary convolution to improve the receptive field of the generated network. The dilation multiple of dilated convolution gradually increases. After each dilated convolution, it is followed by an ordinary convolution with a convolution kernel greater than 1, thereby realizing the alternating use of dilated convolution and ordinary convolution. The input and output sizes of the dilated convolution and ordinary convolution remain unchanged. After a round of dilated convolution and ordinary convolution is completed, the original input jumps to the result after convolution, thereby realizing a round of "multi-receptive field fusion"; by setting different convolution kernels and expansion multiples of the dilated convolution, the amplitude response and phase response corresponding to the time-frequency points are implicitly adjusted; the output convolution layer is mainly responsible for mapping the information of at least one target sampling point after the fusion of multiple receptive fields in the time domain until the length of the output sequence matches the time resolution of the multi-channel far-field speech signal, thereby obtaining the above-mentioned near-field speech signal.

[0224] In the above embodiment, the preset near-field speech generation model constructed based on the generative adversarial network is used to achieve targeted adjustment of the amplitude response and / or phase response of different frequencies in the target spectral information, thereby obtaining a speech signal with a near-field effect in terms of auditory perception, meeting the application requirements of near-fielding far-field speech signals, thereby improving the quality of speech and the intelligibility of speech. At the same time, the preset near-field speech generation model directly outputs a speech signal of a time domain waveform, which can avoid the problem of insufficient reconstruction performance that may be caused by similar input and output modes.

[0225] In a specific embodiment, Figure 6 As shown, the preset near-field speech generation model is an audio codec model based on a deep learning network. The preset near-field speech generation model includes an encoder, a quantizer, and a decoder. Step S250 can be implemented as follows:

[0226] In step S610, the target spectral information is input into an encoder to perform feature encoding processing to obtain latent space feature information corresponding to the target spectral information.

[0227] The latent space feature information represents the latent vector representation of the target spectral information in the latent space.

[0228] In the feature encoding processing stage, in addition to compressing the target spectral information into the latent space, the features of the target spectral information can also be extracted and the latent vectors can be encoded.

[0229] In step S620, the latent space feature information is input into a quantizer to perform feature quantization processing to obtain quantized feature information.

[0230] In step S630, the quantized feature information is input into a decoder for feature decoding processing to obtain a near-field speech signal.

[0231] The audio codec model based on the deep learning network can adopt the architecture of encoder-quantizer-decoder, where the encoder-quantizer is used to output discrete quantized feature information, and the decoder is used to restore the discrete quantized feature information into a continuous audio signal, that is, a near-field speech signal. It is feasible that the encoder is mainly composed of convolutional layers, and the quantizer uses the RVQ layer to quantize the latent space feature information. The encoder and decoder are mirror images in structure.

[0232] In the above embodiment, an audio codec model based on a deep learning network is used as a preset near-field speech generation model. By encoding, quantizing and decoding the target spectral information in the latent space, the amplitude response and / or phase response of different frequencies in the target spectral information are implicitly adjusted to obtain a speech signal with a near-field effect in terms of auditory perception, thereby meeting the application requirements of bringing far-field speech signals into near-field, thereby improving the quality of speech and the intelligibility of speech.

[0233] Figure 7 is a flow chart of a model training method according to an exemplary embodiment. Figure 7 As shown, the method includes:

[0234] In step S710, a multi-channel original speech sample signal and a clean speech sample signal corresponding to the multi-channel original speech sample signal are obtained.

[0235] In a specific embodiment, the multi-channel original speech sample signal may be obtained by performing random noise addition processing on the basis of the corresponding pure speech sample signal.

[0236] In a specific embodiment, the original speech sample signal and its corresponding clean speech sample signal are not limited to speech signals having a far-field effect in auditory perception.

[0237] In step S720, each of the multi-channel original speech sample signals is input into an initial multi-channel filter model to perform speech enhancement processing to obtain first sample spectral information.

[0238] In a specific embodiment, each original speech sample signal in the multi-channel original speech sample signal is input into the initial multi-channel filtering model, and the original spectral information corresponding to each original speech sample signal is determined. Based on the initial multi-channel filtering model, the sample signal weight corresponding to each original speech sample signal is determined. According to the original spectral information corresponding to each original speech sample signal and the sample signal weight corresponding to each original speech sample signal, weighted summation processing is performed to obtain the above-mentioned first sample spectral information.

[0239] In step S730, second sample spectrum information corresponding to the clean speech sample signal is determined.

[0240] In step S740, first enhancement loss data is determined based on the first sample spectral information and the second sample spectral information.

[0241] The first enhanced loss data indicates the degree of spectral difference between the first sample spectral information and the second sample spectral information in the norm space.

[0242] In a specific embodiment, the first enhancement loss data is calculated according to formula (1), wherein X and Y represent the first sample spectral information and the second sample spectral information respectively, α represents the weighting coefficient, which can be 0.7 as an example, and c represents the compression coefficient, which can be 0.3 as an example.

[0243]

[0244] In step S750, the initial multi-channel filtering model is trained based on the first enhanced loss data to obtain the original multi-channel filtering model.

[0245] Figure 8 is a flow chart of a model training method according to an exemplary embodiment. Figure 8 As shown, the method includes:

[0246] In step S810, a clean speech sample signal and a near-field speech reference signal corresponding to the clean speech sample signal are obtained.

[0247] In a specific embodiment, the clean speech sample signal may be a simulated clean human voice signal, and the near-field speech reference signal may be a human voice signal generated and collected in a near-field environment based on the clean speech sample signal.

[0248] In step S820, the sample Mel-spectrogram information corresponding to the clean speech sample signal is determined.

[0249] In step S830, the sample Mel spectrum information is input into the current generation network to perform speech reconstruction processing to obtain the corresponding current near-field speech sample signal.

[0250] Step S830 may refer to the aforementioned embodiment and will not be described in detail here.

[0251] In step S840, current first reconstruction loss data is determined according to the current near-field speech sample signal and the near-field speech reference signal.

[0252] The current first reconstruction loss data indicates the Manhattan distance between the current near-field speech sample signal and the near-field speech reference signal in the current training round.

[0253] In step S850, the current near-field speech sample signal is input into the current identification network to perform speech identification processing to obtain the current first identification result.

[0254] The current first identification result indicates whether the current near-field speech sample signal is identified as the corresponding near-field speech reference signal in the current training round.

[0255] In a specific embodiment, the current identification network can use a multi-scale discriminator to better improve the identification ability of the generation network.

[0256] In step S860, current second reconstruction loss data is determined based on the current first identification result.

[0257] Feasibly, the current second reconstruction loss data may be determined according to the difference between the current first identification result and the true result (the true result indicates that the signals input into the initial identification network are all near-field speech sample signals).

[0258] In step S870, current third reconstruction loss data is determined according to the current first reconstruction loss data and the current second reconstruction loss data.

[0259] In a specific embodiment, according to formula (2), the current third reconstruction loss data L is calculated as G , where L1 represents the current first reconstruction loss data, y represents the pure speech sample signal, G(y) represents the generated current near-field speech sample signal, and D i (G(y)) represents the identification result given by the i-th discriminator for G(y) in the current first identification result, and the range can be between 0 and 1.

[0260]

[0261] In step S880, the current generation network is trained according to the current third reconstruction loss data to obtain the next generation network.

[0262] In a specific embodiment, the current generation network is trained based on the current third reconstruction loss data corresponding to the multiple groups of samples to obtain the next generation network.

[0263] In step S910, the near-field speech reference signal and the current near-field speech sample signal are input into the current identification network, speech identification processing is performed, and the current second identification result is obtained.

[0264] Among them, the current second identification result indicates whether the signal input into the current identification network is a near-field speech reference signal or a current near-field speech sample signal, that is, it can indicate the accuracy of distinguishing the near-field speech reference signal and the current near-field speech sample signal in the current training round.

[0265] In step S920, current first identification loss data is determined based on the current second identification result.

[0266] In a specific embodiment, according to formula (3), the current first identification loss data L is calculated: D , where x represents the near-field speech reference signal, y represents the pure speech sample signal, G(y) represents the generated current near-field speech sample signal, and D i (G(y)) represents the identification result given by the i-th discriminator for G(y) in the current second identification result, D i (x) represents the identification result given by the i-th discriminator for x in the current second identification result, and the identification result can range from 0 to 1.

[0267]

[0268] In step S930, based on the current first identification loss data, the current identification network is trained to obtain a next identification network.

[0269] In step S940, the next generation network is used as the current generation network of the next training round, and the next identification network is used as the current identification network of the next training round, and the above training process is executed in a loop.

[0270] In step S950, when the current generation network and the current identification network reach a preset convergence condition, the training is terminated to obtain the generation network and the identification network.

[0271] Fig. 9 is a schematic diagram of a speech signal processing method according to an exemplary embodiment. Fig. 9As shown, the multi-channel speech collected by the microphone array obtains the corresponding target Mel-spectrogram information or latent space feature information after multi-channel filtering. The target Mel-spectrogram information or latent space feature information is input into a generative model adapted thereto to perform speech reconstruction, and a near-field speech signal of a time domain waveform can be obtained.

[0272] It can be seen from the technical solutions provided by the above embodiments of this specification that the embodiments of the present disclosure input the multi-channel far-field speech signal collected from the target sound source into a preset multi-channel filter model, perform speech enhancement processing, and obtain target spectral information; wherein the preset multi-channel filter model is a filter model obtained by adaptively updating the original multi-channel filter model according to the multi-channel far-field speech signal, or a filter model obtained by adaptively constructing the prediction result of the multi-channel far-field speech signal according to the first neural network model, which can more adaptively perform speech enhancement processing such as noise reduction and reverberation on the multi-channel remote speech signal to improve the speech quality of the target spectral information; the embodiments of the present disclosure input the target spectral information into a preset near-field speech generation model, perform speech reconstruction processing, adjust the amplitude response and / or phase response of different frequencies in the target spectral information, realize the change in speech hearing, and can improve the speech damage that may be caused by the preset multi-channel filter model after speech enhancement processing, thereby obtaining a near-field speech signal corresponding to the multi-channel far-field speech signal, improving the speech quality and speech intelligibility, and meeting the needs of more voice communications and applications.

[0273] Fig.10 is a block diagram of a speech signal processing device according to an exemplary embodiment. Fig.10 , the device comprises:

[0274] An acquisition module 1010 is configured to acquire a multi-channel far-field speech signal collected from a target sound source;

[0275] The speech enhancement module 1020 is configured to input the multi-channel far-field speech signal into a preset multi-channel filter model, perform speech enhancement processing, and obtain target spectral information; the preset multi-channel filter model is a filter model obtained by adaptively updating the original multi-channel filter model according to the multi-channel far-field speech signal, or a filter model obtained by adaptively constructing the prediction result of the multi-channel far-field speech signal according to the first neural network model;

[0276] The speech reconstruction module 1030 is configured to input the target speech spectrum information into a preset near-field speech generation model, perform speech reconstruction processing, and obtain a near-field speech signal corresponding to the multi-channel far-field speech signal.

[0277] In an optional embodiment, the preset near-field speech generation model is a vocoder model obtained by training a generative adversarial network, the preset near-field speech generation model includes a generative network, and the speech reconstruction module 1030 includes:

[0278] A mel spectrum determination unit is configured to determine target mel spectrum information corresponding to the target spectrum information;

[0279] An upsampling unit is configured to input the target Mel spectrum information into the input convolution layer of the generating network, perform upsampling processing, and obtain at least one original sampling point information; the at least one original sampling point information represents the amplitude and phase corresponding to at least one time-frequency point;

[0280] A multi-receptive field fusion unit is configured to input the at least one original sampling point information into the multi-receptive field fusion layer of the generation network, perform fusion processing based on multiple receptive fields, and obtain at least one target sampling point information;

[0281] The time-frequency conversion unit is configured to input the at least one target sampling point information into the output convolution layer of the generation network, perform time-frequency conversion processing, and obtain the near-field speech signal.

[0282] In an optional embodiment, the preset near-field speech generation model is an audio codec model based on a deep learning network, the preset near-field speech generation model includes an encoder, a quantizer and a decoder, and the speech reconstruction module 1030 includes:

[0283] A feature encoding unit is configured to input the target spectral information into the encoder, perform feature encoding processing, and obtain latent space feature information corresponding to the target spectral information;

[0284] A feature quantization unit is configured to input the latent space feature information into the quantizer, perform feature quantization processing, and obtain quantized feature information;

[0285] The feature decoding unit is configured to input the quantized feature information into the decoder, perform feature decoding processing, and obtain the near-field speech signal.

[0286] In an optional embodiment, the multi-channel far-field speech signal includes a plurality of far-field speech signals corresponding to a plurality of channels one by one, and the speech enhancement module 1020 includes:

[0287] A first signal spectrum determination unit is configured to determine spectrum information corresponding to each far-field speech signal in the plurality of far-field speech signals;

[0288] A signal weight determination unit, configured to determine a signal weight corresponding to each far-field speech signal based on the preset multi-channel filtering model;

[0289] The weighting unit is configured to perform weighted summation processing according to the spectrum information corresponding to each far-field speech signal and the signal weight corresponding to each far-field speech signal to obtain the target spectrum information.

[0290] In an optional embodiment, the multi-channel far-field voice signal includes a plurality of far-field voice signals corresponding one-to-one to the plurality of channels, and the apparatus further includes:

[0291] A second signal spectrum determination unit is configured to determine spectrum information corresponding to each far-field speech signal in the plurality of far-field speech signals;

[0292] A model update parameter calculation unit is configured to calculate a model update parameter based on the spectrum information corresponding to each far-field speech signal, wherein the model update parameter indicates the importance of each far-field speech signal;

[0293] The adaptive updating unit is configured to perform adaptive updating on the original multi-channel filtering model based on the model updating parameters to obtain the preset multi-channel filtering model.

[0294] In an optional embodiment, either the first signal spectrum determining unit or the second signal spectrum determining unit includes:

[0295] The short-time Fourier transform subunit is configured to perform short-time Fourier transform processing on each of the multiple far-field speech signals to obtain spectrum information corresponding to each far-field speech signal.

[0296] In an optional embodiment, the model update parameter calculation unit includes:

[0297] A first feature extraction subunit is configured to perform feature extraction processing on the spectral information corresponding to each far-field speech signal to obtain time-frequency feature information corresponding to each far-field speech signal;

[0298] The first covariance calculation subunit is configured to calculate a first spatial covariance matrix and a second spatial covariance matrix based on the time-frequency feature information corresponding to each far-field speech signal; the first spatial covariance matrix indicates the degree of correlation between each far-field speech signal; the second spatial covariance matrix indicates the degree of correlation between the noise of each far-field speech signal;

[0299] A second covariance calculation subunit is configured to obtain a third spatial covariance matrix based on the first spatial covariance matrix and the second spatial covariance matrix;

[0300] an eigendecomposition subunit, configured to perform eigendecomposition processing on the third spatial covariance matrix to obtain a steering vector, wherein the steering vector indicates a direction of a target far-field speech signal in the multi-channel far-field speech signal, wherein the target far-field speech signal is a far-field speech signal least affected by noise;

[0301] The model update parameter first calculation subunit is configured to calculate the model update parameter based on the steering vector and the second spatial covariance matrix.

[0302] In an optional embodiment, the model update parameter calculation unit includes:

[0303] A first feature extraction subunit is configured to perform feature extraction processing on the spectral information corresponding to each far-field speech signal to obtain time-frequency feature information corresponding to each far-field speech signal;

[0304] The third covariance calculation subunit is configured to calculate a first spatial covariance matrix and a second spatial covariance matrix based on the time-frequency feature information corresponding to each far-field speech signal; the first spatial covariance matrix indicates the degree of correlation between each far-field speech signal; the second spatial covariance matrix indicates the degree of correlation between the noise components of each far-field speech signal;

[0305] The second subunit of the model update parameter is configured to input the first spatial covariance matrix and the second spatial covariance matrix into a second neural network model, perform parameter prediction processing, and obtain the model update parameter.

[0306] In an optional embodiment, the device further comprises:

[0307] A feature representation unit is configured to input the multi-channel far-field speech signal into the feature representation network, perform feature representation processing, and obtain first speech feature data;

[0308] A feature extraction unit is configured to input the first speech feature data into the feature extraction network, perform feature extraction processing, and obtain second speech feature data;

[0309] A feature mapping unit is configured to input the second speech feature data into the feature mapping network, perform feature mapping processing, and obtain a prediction result, wherein the prediction result is a filter model construction parameter;

[0310] The model building unit is configured to build the preset multi-channel filtering model according to the prediction result.

[0311] In an optional embodiment, the device further comprises:

[0312] A first training data acquisition unit is configured to acquire a clean speech sample signal and a near-field speech reference signal corresponding to the clean speech sample signal;

[0313] A sample Mel spectrum determination unit is configured to determine the sample Mel spectrum information corresponding to the clean speech sample signal;

[0314] A sample speech reconstruction unit is configured to input the sample Mel spectrum information into the current generation network, perform speech reconstruction processing, and obtain a corresponding current near-field speech sample signal;

[0315] A first reconstruction loss data determining unit is configured to determine a current first reconstruction loss data according to the current near-field speech sample signal and the near-field speech reference signal; the current first reconstruction loss data indicates a Manhattan distance between the current near-field speech sample signal and the near-field speech reference signal in a current training round;

[0316] A first identification result determination unit is configured to input the current near-field speech sample signal into a current identification network, perform speech identification processing, and obtain a current first identification result, wherein the current first identification result indicates whether the current near-field speech sample signal is identified as a corresponding near-field speech reference signal in a current training round;

[0317] A second reconstruction loss data determining unit is configured to determine current second reconstruction loss data based on the current first identification result;

[0318] A third reconstruction loss data determining unit is configured to determine current third reconstruction loss data according to the current first reconstruction loss data and the current second reconstruction loss data;

[0319] The first training unit is configured to train the current generation network according to the current third reconstruction loss data to obtain a next generation network.

[0320] A second identification result determination unit is configured to input the near-field speech reference signal and the current near-field speech sample signal into the current identification network, perform speech identification processing, and obtain a current second identification result, wherein the current second identification result indicates the accuracy of distinguishing the near-field speech reference signal and the current near-field speech sample signal in the current training round;

[0321] A first identification loss data determining unit, configured to determine current first identification loss data based on the current second identification result;

[0322] A second training unit is configured to perform training on the current identification network based on the current first identification loss data to obtain a next identification network;

[0323] A loop unit, configured to execute the next generation network as the current generation network of the next training round, and the next identification network as the current identification network of the next training round, and cyclically execute the above training process;

[0324] The judgment unit is configured to terminate the training when the current generation network and the current identification network reach a preset convergence condition, thereby obtaining the generation network and the identification network.

[0325] In an optional embodiment, the device further comprises:

[0326] A second training data acquisition unit is configured to acquire a multi-channel original speech sample signal and a clean speech sample signal corresponding to the multi-channel original speech sample signal;

[0327] A sample speech enhancement unit is configured to input each of the multi-channel original speech sample signals into an initial multi-channel filtering model, perform speech enhancement processing, and obtain first sample spectral information;

[0328] A sample signal spectrum determination unit is configured to determine second sample spectrum information corresponding to the clean speech sample signal;

[0329] A first enhancement loss data determining unit is configured to determine first enhancement loss data based on the first sample spectral information and the second sample spectral information; the first enhancement loss data indicates a degree of spectral difference between the first sample spectral information and the second sample spectral information in a norm space;

[0330] The third training unit is configured to perform training on the initial multi-channel filtering model based on the first enhanced loss data to obtain the original multi-channel filtering model.

[0331] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0332] In an exemplary embodiment, an electronic device is also provided, including: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the speech signal processing method as in the embodiment of the present disclosure.

[0333] Fig.11 is a block diagram of an electronic device for implementing a method for processing a speech signal according to an exemplary embodiment. The electronic device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig.11 As shown. The electronic device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for processing a voice signal is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0334] Fig.12 is a block diagram of an electronic device for implementing a method for processing a speech signal according to an exemplary embodiment. The electronic device may be a server, and its internal structure diagram may be as shown in FIG. Fig.12 As shown. The electronic device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a voice signal processing method is implemented.

[0335] Those skilled in the art will understand that Fig.11 and Fig.12 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present disclosure, and does not constitute a limitation on the electronic device to which the scheme of the present disclosure is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0336] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the speech signal processing method in the embodiment of the present disclosure.

[0337] In an exemplary embodiment, a computer program product is also provided, including computer instructions, and when the computer instructions are executed by a processor, the speech signal processing method in the embodiment of the present disclosure is implemented.

[0338] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0339] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0340] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A speech signal processing method, characterized in that: The method comprises: Acquire multi-channel far-field speech signals collected from target sound sources; The multi-channel far-field speech signal is input into a preset multi-channel filter model to perform speech enhancement processing to obtain target spectral information; the preset multi-channel filter model is a filter model obtained by adaptively updating the original multi-channel filter model according to the multi-channel far-field speech signal, or a filter model adaptively constructed according to the prediction result of the multi-channel far-field speech signal by the first neural network model; The target spectrum information is input into a preset near-field speech generation model, and speech reconstruction processing is performed to obtain a near-field speech signal corresponding to the multi-channel far-field speech signal.

2. The method according to claim 1, characterized in that The preset near-field speech generation model is a vocoder model obtained by training a generative adversarial network, the preset near-field speech generation model includes a generation network, the target spectrum information is input into the preset near-field speech generation model, speech reconstruction processing is performed, and a near-field speech signal corresponding to the multi-channel far-field speech signal is obtained, including: Determine target Mel spectrum information corresponding to the target spectrum information; Input the target Mel spectrum information into the input convolution layer of the generation network, perform upsampling processing, and obtain at least one original sampling point information; the at least one original sampling point information represents the amplitude and phase corresponding to at least one time-frequency point; Inputting the at least one original sampling point information into the multi-receptive field fusion layer of the generation network, performing fusion processing based on the multi-receptive field, and obtaining at least one target sampling point information; The at least one target sampling point information is input into the output convolution layer of the generation network, and time-frequency conversion processing is performed to obtain the near-field speech signal.

3. The method according to claim 1, characterized in that The preset near-field speech generation model is an audio codec model based on a deep learning network, and the preset near-field speech generation model includes an encoder, a quantizer, and a decoder. The target spectrum information is input into the preset near-field speech generation model, and speech reconstruction processing is performed to obtain a near-field speech signal corresponding to the multi-channel far-field speech signal, including: Inputting the target spectral information into the encoder, performing feature encoding processing, and obtaining latent space feature information corresponding to the target spectral information; Inputting the latent space feature information into the quantizer to perform feature quantization processing to obtain quantized feature information; The quantized feature information is input into the decoder, and feature decoding processing is performed to obtain the near-field speech signal.

4. The method according to claim 1, characterized in that The multi-channel far-field speech signal includes a plurality of far-field speech signals corresponding to a plurality of channels one by one, and the multi-channel far-field speech signal is input into a preset multi-channel filter model to perform speech enhancement processing to obtain target spectral information, including: Determining spectral information corresponding to each far-field speech signal among the multiple far-field speech signals; Based on the preset multi-channel filtering model, determining a signal weight corresponding to each far-field speech signal; A weighted summation process is performed according to the spectrum information corresponding to each far-field speech signal and the signal weight corresponding to each far-field speech signal to obtain the target spectrum information.

5. The method according to claim 1, characterized in that The multi-channel far-field voice signal includes a plurality of far-field voice signals corresponding one-to-one to a plurality of channels, and the method further includes: Determining spectral information corresponding to each far-field speech signal among the multiple far-field speech signals; Based on the spectral information corresponding to each far-field speech signal, a model update parameter is calculated, where the model update parameter indicates the importance of each far-field speech signal; Based on the model update parameters, the original multi-channel filtering model is adaptively updated to obtain the preset multi-channel filtering model.

6. The method according to any one of claims 4 or 5, characterized in that: The determining of the spectral information corresponding to each far-field speech signal among the multiple far-field speech signals includes: Perform short-time Fourier transform processing on each of the multiple far-field speech signals to obtain spectral information corresponding to each far-field speech signal.

7. The method according to claim 5, characterized in that The calculating the model update parameter based on the spectral information corresponding to each far-field speech signal includes: Performing feature extraction processing on the spectral information corresponding to each far-field speech signal to obtain time-frequency feature information corresponding to each far-field speech signal; Based on the time-frequency feature information corresponding to each far-field speech signal, a first spatial covariance matrix and a second spatial covariance matrix are calculated; the first spatial covariance matrix indicates the degree of correlation between each far-field speech signal; the second spatial covariance matrix indicates the degree of correlation between the noise of each far-field speech signal; Based on the first spatial covariance matrix and the second spatial covariance matrix, obtaining a third spatial covariance matrix; Performing eigendecomposition processing on the third spatial covariance matrix to obtain a steering vector, wherein the steering vector indicates a direction of a target far-field speech signal in the multi-channel far-field speech signal, wherein the target far-field speech signal is a far-field speech signal least affected by noise; The model update parameter is calculated based on the steering vector and the second spatial covariance matrix.

8. The method according to claim 5, characterized in that The calculating the model update parameter based on the spectral information corresponding to each far-field speech signal includes: Performing feature extraction processing on the spectral information corresponding to each far-field speech signal to obtain time-frequency feature information corresponding to each far-field speech signal; Based on the time-frequency feature information corresponding to each far-field speech signal, a first spatial covariance matrix and a second spatial covariance matrix are calculated; the first spatial covariance matrix indicates the degree of correlation between each far-field speech signal; the second spatial covariance matrix indicates the degree of correlation between the noise components of each far-field speech signal; The first spatial covariance matrix and the second spatial covariance matrix are input into a second neural network model, and parameter prediction processing is performed to obtain the model update parameters.

9. The method according to claim 1, characterized in that: The first neural network model includes a feature representation network, a feature extraction network and a feature mapping network, and the method further includes: Inputting the multi-channel far-field speech signal into the feature representation network, performing feature representation processing, and obtaining first speech feature data; Inputting the first speech feature data into the feature extraction network to perform feature extraction processing to obtain second speech feature data; Inputting the second speech feature data into the feature mapping network, performing feature mapping processing, and obtaining a prediction result, wherein the prediction result is a parameter for constructing a filtering model; The preset multi-channel filtering model is constructed according to the prediction results.

10. The method according to claim 2, characterized in that The method further comprises: Acquire a clean speech sample signal and a near-field speech reference signal corresponding to the clean speech sample signal; Determine sample Mel spectrum information corresponding to the clean speech sample signal; Inputting the sample Mel spectrum information into the current generation network, performing speech reconstruction processing, and obtaining the corresponding current near-field speech sample signal; Determining a current first reconstruction loss data according to the current near-field speech sample signal and the near-field speech reference signal; the current first reconstruction loss data indicates a Manhattan distance between the current near-field speech sample signal and the near-field speech reference signal in a current training round; Inputting the current near-field speech sample signal into the current identification network to perform speech identification processing to obtain a current first identification result, wherein the current first identification result indicates whether the current near-field speech sample signal is identified as a corresponding near-field speech reference signal in the current training round; Based on the current first identification result, determining current second reconstruction loss data; Determining current third reconstruction loss data according to the current first reconstruction loss data and the current second reconstruction loss data; Training the current generation network according to the current third reconstruction loss data to obtain a next generation network; Inputting the near-field speech reference signal and the current near-field speech sample signal into the current identification network, performing speech identification processing, and obtaining a current second identification result, wherein the current second identification result indicates the degree of accuracy of distinguishing the near-field speech reference signal and the current near-field speech sample signal in the current training round; Based on the current second identification result, determining current first identification loss data; Based on the current first identification loss data, training the current identification network to obtain a next identification network; Using the next generation network as the current generation network of the next training round, and using the next identification network as the current identification network of the next training round, and looping through the above training process; When the current generation network and the current identification network reach a preset convergence condition, the training is terminated to obtain the generation network and the identification network.

11. The method according to claim 1, characterized in that: The method further comprises: Acquire a multi-channel original voice sample signal and a clean voice sample signal corresponding to the multi-channel original voice sample signal; Input each of the multi-channel original speech sample signals into an initial multi-channel filtering model to perform speech enhancement processing to obtain first sample spectral information; Determining second sample spectrum information corresponding to the clean speech sample signal; Based on the first sample spectral information and the second sample spectral information, determining first enhancement loss data; the first enhancement loss data indicates the degree of spectral difference between the first sample spectral information and the second sample spectral information in the norm space; The initial multi-channel filtering model is trained based on the first enhancement loss data to obtain the original multi-channel filtering model.

12. A speech signal processing device, characterized in that: The device comprises: An acquisition module is configured to acquire a multi-channel far-field speech signal collected from a target sound source; The speech enhancement module is configured to input the multi-channel far-field speech signal into a preset multi-channel filter model, perform speech enhancement processing, and obtain target spectral information; the preset multi-channel filter model is a filter model obtained by adaptively updating the original multi-channel filter model according to the multi-channel far-field speech signal, or a filter model adaptively constructed according to the prediction result of the first neural network model for the multi-channel far-field speech signal; The speech reconstruction module is configured to input the target speech spectrum information into a preset near-field speech generation model, perform speech reconstruction processing, and obtain a near-field speech signal corresponding to the multi-channel far-field speech signal.

13. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the speech signal processing method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the speech signal processing method according to any one of claims 1 to 11.

15. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the speech signal processing method according to any one of claims 1 to 11 is implemented.

Images