A method and system for estimating direction of arrival of speech

Abstract

The present invention discloses a method for estimating the direction of arrival of speech, comprising: obtaining mixed speech audio in a far-field environment; performing frame division, windowing and Fourier transform processing on the mixed speech audio to obtain a spectrum signal of each frame of audio; inputting the spectrum signal into a sound source position estimation neural network model, and the sound source position estimation neural network model outputs a frame-level Cartesian coordinate estimate of a target speaker; performing an average pooling operation on the frame-level Cartesian coordinate estimate of the target speaker in the time dimension to obtain a sentence-level Cartesian coordinate estimate of the target speaker; performing an inverse tangent operation on the sentence-level Cartesian coordinate estimate of the target speaker to obtain a direction of arrival estimate of the target speaker. The present invention provides a method and system for estimating the direction of arrival of speech, which can overcome the influence of non-speech frames and interfering speaker frames on the effect of the direction of arrival estimation model in a multi-speaker far-field environment, and realize low-latency direction of arrival estimation.

Description

Technical Field

[0001] The present invention relates to the field of speech array signal processing, and in particular to a speech direction of arrival estimation method and system. Background Art

[0002] Speech direction of arrival estimation is an important topic in the field of speech array signal processing. Its goal is to obtain the direction of speech signal generation. Speech direction of arrival estimation has a wide range of applications in wireless communication, radar, underwater communication, smart home and other environments.

[0003] Most traditional algorithms for speech direction of arrival estimation, such as the "multiple signal classification algorithm" and the "joint controllable response power and phase transformation algorithm", perform estimation at the signal processing level. Their performance is very limited for different noise and reverberation environments.

[0004] In recent years, the DOA estimation model based on deep neural networks has developed rapidly, and its performance has far exceeded that of traditional methods. However, in a far-field environment with multiple speakers, the effect of the DOA estimation model will be affected by non-speech frames and interfering speaker frames. Currently, there are several solutions:

[0005] (1) The voice activity estimation algorithm is introduced to remove non-speech frames and the direction of arrival estimation is implemented at the frame level. This method alleviates the impact of non-speech frames to a certain extent, but the preprocessing method increases the amount of model calculation. When the voice activity estimation is inaccurate, it will also bring cumulative errors to the system.

[0006] (2) Post-processing is used during the inference process, that is, K maximum values ​​are selected from the frame-level estimation results as the estimated direction of arrival, where K is the number of speakers in the speech. Due to the introduction of the manual selection process, this method is not suitable for online systems;

[0007] (3) A pre-separation or pre-extraction method is introduced to separate the mixed speech to obtain the pure speech of the target speaker, and then the direction of arrival of the obtained pure speech is estimated. However, this method is not applicable when the direction of arrival estimation technology is required to provide prior knowledge for the speech separation technology. Summary of the invention

[0008] To address the defects in the prior art, an embodiment of the present invention provides a speech direction of arrival estimation method and system, which can overcome the influence of non-speech frames and interfering speaker frames on the effect of the direction of arrival estimation model in a multi-speaker far-field environment, and realize low-latency direction of arrival estimation.

[0009] In a first aspect, an embodiment of the present invention records a method for estimating a speech direction of arrival, the method comprising the following steps:

[0010] Acquire mixed speech audio in a far-field environment; the mixed speech audio includes speech information of at least one target speaker;

[0011] The mixed speech audio is framed, windowed and Fourier transformed to obtain a frequency spectrum signal of each frame of audio;

[0012] Inputting the spectrum signal of each frame of audio into a sound source position estimation neural network model, the sound source position estimation neural network model outputs a frame-level Cartesian coordinate estimation of the target speaker; the sound source position estimation neural network model includes a self-attention module, the self-attention module extracts the audio frame of the target speaker by assigning different weights to the input audio frames; the sound source position estimation neural network model is obtained by training;

[0013] performing an average pooling operation on the frame-level Cartesian coordinate estimate of the target speaker in the time dimension to obtain a sentence-level Cartesian coordinate estimate of the target speaker;

[0014] An arctangent operation is performed on the sentence-level Cartesian coordinate estimate of the speaker to obtain a direction of arrival estimate of the target speaker.

[0015] Preferably, the sound source position estimation neural network model includes a convolution input module, four layers of densely connected-time-frequency dimension self-attention modules and a fully connected layer; each layer of the four layers of densely connected-time-frequency dimension self-attention modules includes a densely connected convolution module, a self-attention module in the time dimension and a self-attention module in the frequency dimension.

[0016] In a second aspect, an embodiment of the present invention records a method for training a neural network model for estimating a sound source position, comprising:

[0017] Acquire a mixed speech audio training set and a label set in a far-field environment; the label set is used to record the sound source position vector of the target speaker in the mixed speech audio of the training set; the mixed speech audio includes speech information of at least one target speaker;

[0018] The mixed speech audio is framed, windowed, and Fourier transformed to obtain a spectrum signal of the mixed speech audio in a training set, wherein the spectrum signal includes a real part and an imaginary part;

[0019] Inputting the concatenated data of the real part and the imaginary part of the spectrum signal into the sound source position estimation neural network, and the sound source position estimation neural network model outputs the frame-level Cartesian coordinate estimation of the target speaker;

[0020] Performing an average pooling operation on the frame-level Cartesian coordinate estimate of the target speaker in the time dimension to obtain the sentence-level Cartesian coordinate estimate of the target speaker; the sentence-level Cartesian coordinate estimate of the target speaker is used to indicate the sound source position information;

[0021] The mean square error of the sound source position is calculated according to the sound source position information and the label set; and the sound source position estimation neural network is adjusted according to the mean square error.

[0022] Preferably, the sound source position estimation neural network model includes a convolution input module, four layers of densely connected-time-frequency dimension self-attention modules and a fully connected layer; each layer of the four layers of densely connected-time-frequency dimension self-attention modules includes a densely connected convolution module, a self-attention module in the time dimension and a self-attention module in the frequency dimension.

[0023] In a third aspect, an embodiment of the present invention records a speech direction of arrival estimation system, the system comprising:

[0024] The signal receiving device may be a multi-channel microphone array, used to obtain mixed voice audio in a far-field environment; the mixed voice audio includes voice information of at least one speaker;

[0025] A signal processing module, used for performing frame division, windowing and Fourier transform processing on the mixed speech audio to obtain a spectrum signal of each frame of audio;

[0026] A neural network module, used for inputting the spectrum signal of each frame of audio into a sound source position estimation network model, and the sound source position estimation network model outputs a frame-level Cartesian coordinate estimation of the speaker; the sound source position estimation network model is obtained by training;

[0027] a sentence-level sound source position calculation module, configured to perform an average pooling operation on the frame-level Cartesian coordinate estimate of the speaker in the time dimension to obtain the sentence-level Cartesian coordinate estimate of the speaker; the sentence-level Cartesian coordinate estimate of the speaker is used to indicate the sound source position information;

[0028] The direction of arrival calculation module is used to perform an arctangent operation on the Cartesian coordinate estimate of the speaker at the sentence level to obtain the direction of arrival estimate of the speaker.

[0029] In a fourth aspect, a server is provided, comprising a memory and a processor, wherein:

[0030] The memory stores executable instructions;

[0031] The processor is used to read and execute executable instructions stored in the memory to implement any of the above methods.

[0032] In a fifth aspect, a storage medium is provided, characterized in that when instructions in the storage medium are executed by a processor, any of the methods described above can be executed.

[0033] It can be understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0035] Figure 1 A flow chart of a method for estimating a speech direction of arrival provided by an embodiment of the present invention;

[0036] Figure 2 A block diagram of a speech direction of arrival estimation system provided by an embodiment of the present invention;

[0037] Figure 3 A flowchart of a neural network module training in a speech direction of arrival estimation system provided by an embodiment of the present invention;

[0038] Figure 4 The schematic diagram is a brief structural diagram of a server in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] Figure 1 A flow chart of a method for estimating a speech direction of arrival provided by an embodiment of the present invention comprises the following steps:

[0041] Step S11, obtaining mixed speech audio in a far-field environment; the mixed speech audio includes speech information of at least one target speaker.

[0042] The mixed speech audio in the far-field environment also includes speech information that interferes with the speaker and non-speech noise information.

[0043] Step S12, performing frame division, windowing and Fourier transformation on the mixed speech audio to obtain a spectrum signal of each frame of audio.

[0044] In a feasible implementation manner, the following steps S121-S123 are included:

[0045] S121, divide the mixed speech audio into frames, each frame is 25 milliseconds, and the frame shift is 6.25 milliseconds;

[0046] S122, adding a window to each frame of audio, wherein the window function is a Hamming window;

[0047] S123, performing a 512-point Fourier transform on each frame of audio to obtain a frequency spectrum of each frame of audio.

[0048] Step S13, inputting the frequency spectrum signal of each frame of audio into the sound source position estimation neural network model, and the sound source position estimation neural network model outputs the frame-level Cartesian coordinate estimation of the target speaker.

[0049] The estimated frame-level Cartesian coordinates are obtained based on the spectrum and sound source position estimation neural network of each frame of audio in, is the Cartesian coordinate of the t-th frame of audio, 1≤t≤T, and T represents the total number of frames of input speech.

[0050] The neural network model for sound source position estimation includes a convolution input module, four layers of densely connected-time-frequency self-attention modules and one fully connected layer; each layer of the four layers of densely connected-time-frequency self-attention modules includes a densely connected convolution module, a self-attention module in the time dimension and a self-attention module in the frequency dimension.

[0051] The neural network model for sound source location estimation is obtained by training. The self-attention module it contains consists of a self-attention module in the time dimension and a self-attention module in the frequency dimension. For a specific target speaker, the non-speech frame does not contain any speaker information, while the interfering speaker frame will interfere with the target speaker's DOA estimation, making model training more difficult and affecting the model effect. Through training, the self-attention module in the time dimension can assign different weights to the high-dimensional feature vectors of each frame according to the importance of different frames of the input audio, so as to extract high-dimensional feature vectors related to the target speaker. The self-attention network module in the frequency dimension can evaluate the relevance and importance of different frequency points on the same frame, and assign weights to each frequency point according to the evaluation results.

[0052] The sound source location estimation neural network model can also output an ideal low latency. In the proposed sound source location estimation neural network model, the original latency mainly comes from three points: one is the value of the frame shift in the Fourier transform processing in step S12; the second is the convolution input module in the model; and the third is the self-attention module in the time dimension of the model. For the second point, the traditional convolution network is replaced by a causal convolution network, so that the convolution network no longer generates latency; for the third point, each self-attention module in the time dimension includes three parallel two-dimensional convolution layers, which are used to calculate the Q, K, and V values ​​of the self-attention network. According to the Q, K, and V values, the output of the self-attention module in the time dimension can be obtained. The calculation method of the non-causal self-attention mechanism is:

[0053]

[0054] A=SoftMax(V),

[0055] Among them, d n represents the dimension of vector K, and A is the output.

[0056] In the present invention, the above formula is improved as follows:

[0057]

[0058] The Mask function is defined as:

[0059]

[0060] This function makes the estimation of the Cartesian coordinates at the current frame t' depend only on the current frame and the historical frames, so no delay is generated.

[0061] Therefore, the delay in the system only comes from the first point mentioned above, that is, the value of the frame shift in the Fourier transform process. Similar to what is mentioned in step S121, the length of the frame shift is 6.25 milliseconds, so the delay of the entire system is also 6.25 milliseconds, achieving low-delay processing.

[0062] In further operations in the future, the direction of arrival of the designated target speaker can also be estimated in combination with the speaker extraction technology. The speaker extraction technology requires the preparation of the registered voice of the designated target speaker in advance, and the "data after the splicing of the real part and the imaginary part of the spectrum" of the registered voice and the "spectral signal of each frame of audio" described in the above step S13 of the present invention are used as the input of the sound source position estimation neural network model, and the estimated Cartesian coordinates of the designated target speaker can be output, and the direction of arrival can be obtained.

[0063] Step S14, performing an average pooling operation on the frame-level Cartesian coordinate estimate of the target speaker in the time dimension to obtain the sentence-level Cartesian coordinate estimate of the target speaker.

[0064] Assume that represents the Cartesian coordinates estimated on the t-th frame, where T represents the total number of speech frames, then the average pooling process can be expressed as:

[0065]

[0066] in is the final estimated sentence-level Cartesian coordinate.

[0067] Step S15, performing an arctangent operation on the Cartesian coordinate estimate of the speaker at the sentence level to obtain a direction of arrival estimate of the target speaker.

[0068] The calculation process of the direction of arrival is:

[0069]

[0070] In the formula, Represents the estimated direction of arrival of the target speaker.

[0071] Figure 2 A block diagram of a speech direction of arrival estimation system provided by an embodiment of the present invention, the system includes: a signal receiving device 21, a signal processing module 22, a neural network module 23, a sentence level sound source position calculation module 24, and a direction of arrival calculation module 25. Specifically:

[0072] The signal receiving device 21 may be a multi-channel microphone array, which is used to obtain mixed voice audio in a far-field environment; the mixed voice audio includes voice information of at least one speaker;

[0073] The signal processing module 22 is used to perform framing, windowing and Fourier transform processing on the mixed speech audio to obtain a spectrum signal of each frame of audio;

[0074] A neural network module 23 is used to input the spectrum signal of each frame of audio into a sound source position estimation neural network model, and the sound source position estimation neural network model outputs a frame-level Cartesian coordinate estimation of the speaker; the sound source position estimation neural network model is obtained by training;

[0075] A sentence-level sound source location calculation module 24 is used to perform an average pooling operation on the speaker's frame-level Cartesian coordinate estimate in the time dimension to obtain the speaker's sentence-level Cartesian coordinate estimate; the speaker's sentence-level Cartesian coordinate estimate is used to indicate the sound source location information;

[0076] The direction of arrival calculation module 25 is used to perform an arctangent operation on the Cartesian coordinate estimate of the speaker at the sentence level to obtain the direction of arrival estimate of the speaker.

[0077] Figure 3 A flowchart of training a neural network module in a speech direction of arrival estimation system provided in an embodiment of the present invention. In a speech direction of arrival estimation system provided in an embodiment of the present invention, the neural network module includes a sound source position estimation neural network model. The process of training the sound source position estimation neural network model includes the following steps S31-S32:

[0078] Step S31, obtaining a mixed speech audio training set and a label set in a far-field environment, and determining the training speech audio and labels.

[0079] In one embodiment, a mixed speech audio training set is established, and the labels in the label set are used to record the two-dimensional vector of the sound source position of the target speaker in the training speech audio; the training speech audio and the label are determined, and the training speech audio is framed, windowed and Fourier transformed to obtain a training speech spectrum; the training speech spectrum includes a real part and an imaginary part.

[0080] Step S32: training a sound source position estimation neural network based on the training speech spectrum and labels.

[0081] In one embodiment, the data obtained by concatenating the real and imaginary parts of the training speech spectrum is used as the input of the sound source position estimation network, the sound source position vector estimation is used as the output, and the mean square error of the sound source position is calculated; training is performed with the mean square error value being within a set threshold as the goal, to obtain a trained sound source position estimation neural network.

[0082] The mean square error of the sound source position can be recorded as the first loss function. The mean square error of the sound source position can be calculated based on the mean square error of the sound source position vector estimated by the network output and the sound source position vector recorded by the label. The set threshold is recorded as the first threshold.

[0083] The input of the sound source location estimation neural network is the concatenation of the real and imaginary parts of the spectrum obtained after Fourier transformation of the training speech audio. The sound source location estimation network includes a convolution input module, 4 layers of densely connected-time-frequency dimension self-attention modules and 1 fully connected layer; each layer of densely connected-time-frequency dimension self-attention module includes a densely connected convolution module, a time dimension self-attention module and a frequency dimension self-attention module. The convolution input module includes a two-dimensional convolution layer, a linear rectification function and a batch normalization processing layer. The number of output nodes of the two-dimensional convolution layer is (2×m), where m is the number of channels of the input speech, and the convolution kernel size and stride are (1, 1) and (1, 1) respectively. The densely connected convolution module includes 5 consecutive convolution modules, each of which contains a two-dimensional convolution layer, a parameterized linear rectification function, a batch normalization processing layer and a residual connection. The number of output nodes of the two-dimensional convolution layer is (2×m), and the convolution kernel size and stride are (2, 3) and (1, 1) respectively. Each time dimension self-attention module includes three parallel two-dimensional convolutional layers, which are used to calculate the Q, K, and V values ​​of the self-attention network. The output of the time dimension self-attention module can be obtained according to the Q, K, and V values. The convolution kernel size of these three two-dimensional convolutional layers is (1, 1), and the input vector is downsampled in the frequency dimension. The input dimensions of the four-layer time dimension self-attention module are B×256×(2×m)×T, B×64×(2×m)×T, B×16×(2×m)×T, and B×4×(2×m)×T, respectively; the output dimensions are B×64×(2×m)×T, B×16×(2×m)×T, B×4×(2×m)×T, and B×1×(2×m)×T, respectively. Among them, B and T represent the size of the batch data and the number of speech frames of the input data, respectively. The structure of the frequency dimension self-attention module is the same as that of the time dimension self-attention module, the only difference is that the input vector is not downsampled. The input dimensions of the 4-layer frequency-dimensional self-attention module are B×(2×m)×T×64, B×(2×m)×T×16, B×(2×m)×T×4, and B×(2×m)×T×1 respectively; the output dimensions are the same as the input dimensions. The number of output nodes of the fully connected layer is 2. The dimension of the final output data of the network is B×T×2. The output data is the estimated sound source position vector of the target speaker at the frame level.

[0084] Based on the same inventive concept, an embodiment of the present invention further provides a server, including a memory 41 and a processor 42, wherein:

[0085] The memory 41 stores executable instructions;

[0086] The processor 42 is used to read and execute executable instructions stored in the memory to implement any of the above methods.

[0087] Based on the same inventive concept, an embodiment of the present invention further provides a storage medium, which enables execution of any of the above-described methods when instructions in the storage medium are executed by a processor.

[0088] The present invention proposes a method for estimating the direction of arrival of speech. The method uses a densely connected time-frequency dimension self-attention network. Among them, the densely connected convolutional network module can retain a long period of historical information and effectively deal with the gradient vanishing problem; the time dimension self-attention network module evaluates the importance of different speech frames, and assigns different weights to the high-dimensional feature vectors of each speech frame according to the evaluation results; the frequency dimension self-attention network module evaluates the relevance and importance of different frequency points on the same speech frame, and assigns different weights to the high-dimensional feature vectors at different frequency points according to the evaluation results. The method realizes the direction of arrival estimation at the sentence level and solves the negative impact of non-speech frames and interfering speaker speech frames on the estimation results in a far-field noisy environment.

[0089] It should be noted that the method provided herein is not inherently related to any specific computer, virtual device or other equipment. Various general devices can also be used together with the teaching based on this. According to the above description, it is obvious to construct the structure required for this type of device. In addition, the present invention is not directed to any specific programming language. It should be understood that various programming languages ​​can be utilized to implement the present invention described herein, and the description of the call of specific languages ​​and system function modules above is only for the purpose of disclosing the best mode of the invention.

[0090] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be implemented without these specific details. In some examples, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0091] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention belong to the scope of the present invention and its equivalent technology, the present invention is also intended to include these changes and variations.

Claims

1. A method for estimating a speech direction of arrival, characterized in that: include: Acquire mixed speech audio in a far-field environment; the mixed speech audio includes speech information of at least one target speaker; The mixed speech audio is framed, windowed and Fourier transformed to obtain a frequency spectrum signal of each frame of audio; Inputting the spectrum signal of each frame of audio into a sound source position estimation neural network model, the sound source position estimation neural network model outputs a frame-level Cartesian coordinate estimation of the target speaker; the sound source position estimation neural network model includes a self-attention module, the self-attention module extracts the audio frame of the target speaker by assigning different weights to the input audio frames, the self-attention module includes a time dimension self-attention module and a frequency dimension self-attention module; the sound source position estimation neural network model is obtained by training; performing an average pooling operation on the frame-level Cartesian coordinate estimate of the target speaker in the time dimension to obtain a sentence-level Cartesian coordinate estimate of the target speaker; An arctangent operation is performed on the sentence-level Cartesian coordinate estimate of the target speaker to obtain a direction of arrival estimate of the target speaker.

2. The method according to claim 1, characterized in that The sound source position estimation neural network model includes a convolution input module, four layers of dense connection-time-frequency dimension self-attention modules and a fully connected layer; each layer of the four layers of dense connection-time-frequency dimension self-attention modules includes a dense connection convolution module, a time dimension self-attention module and a frequency dimension self-attention module.

3. A method for training a neural network model for estimating a sound source position, comprising: Acquire a mixed speech audio training set and a label set in a far-field environment; the label set is used to record the sound source position vector of the target speaker in the mixed speech audio of the training set; the mixed speech audio includes speech information of at least one target speaker; The mixed speech audio is framed, windowed, and Fourier transformed to obtain a spectrum signal of the mixed speech audio in a training set, wherein the spectrum signal includes a real part and an imaginary part; Inputting the data obtained by splicing the real part and the imaginary part of the spectrum signal into the sound source position estimation neural network, and the sound source position estimation neural network model outputs the frame-level Cartesian coordinate estimation of the target speaker; the sound source position estimation neural network model includes a self-attention module, and the self-attention module extracts the audio frame of the target speaker by assigning different weights to the input audio frames, and the self-attention module includes a self-attention module of the time dimension and a self-attention module of the frequency dimension; Performing an average pooling operation on the frame-level Cartesian coordinate estimate of the target speaker in the time dimension to obtain the sentence-level Cartesian coordinate estimate of the target speaker; the sentence-level Cartesian coordinate estimate of the target speaker is used to indicate the sound source position information; The mean square error of the sound source position is calculated according to the sound source position information and the label set; and the sound source position estimation neural network is adjusted according to the mean square error.

4. The method according to claim 3, characterized in that The sound source position estimation neural network model includes a convolution input module, four layers of dense connection-time-frequency dimension self-attention modules and a fully connected layer; each layer of the four layers of dense connection-time-frequency dimension self-attention modules includes a dense connection convolution module, a time dimension self-attention module and a frequency dimension self-attention module.

5. A speech direction of arrival estimation system, characterized in that: include: A signal receiving device, used to obtain mixed voice audio in a far-field environment; the mixed voice audio includes voice information of at least one speaker; A signal processing module, used for performing frame division, windowing and Fourier transform processing on the mixed speech audio to obtain a spectrum signal of each frame of audio; A neural network module, used for inputting the spectrum signal of each frame of audio into a sound source position estimation neural network model, and the sound source position estimation neural network model outputs a frame-level Cartesian coordinate estimation of the speaker; the sound source position estimation neural network model includes a self-attention module, the self-attention module extracts the audio frame of the speaker by assigning different weights to the input audio frames, and the self-attention module includes a time dimension self-attention module and a frequency dimension self-attention module; the sound source position estimation neural network model is obtained by training; a sentence-level sound source position calculation module, configured to perform an average pooling operation on the frame-level Cartesian coordinate estimate of the speaker in the time dimension to obtain the sentence-level Cartesian coordinate estimate of the speaker; the sentence-level Cartesian coordinate estimate of the speaker is used to indicate the sound source position information; The direction of arrival calculation module is used to perform an arctangent operation on the Cartesian coordinate estimate of the speaker at the sentence level to obtain the direction of arrival estimate of the speaker.

6. The system according to claim 5, characterized in that The sound source position estimation neural network model includes a convolution input module, four layers of dense connection-time-frequency dimension self-attention modules and a fully connected layer; each layer of the four layers of dense connection-time-frequency dimension self-attention modules includes a dense connection convolution module, a time dimension self-attention module and a frequency dimension self-attention module.

7. A server, characterized in that: comprising a memory and a processor, wherein: The memory stores executable instructions; The processor is used to read and execute executable instructions stored in the memory to implement the method according to any one of claims 1 to 4.

8. A storage medium, characterized in that: When the instructions in the storage medium are executed by a processor, the method according to any one of claims 1 to 4 can be performed.

Images