Multi-channel voiceprint recognition method, device and equipment based on transformer framework

Abstract

The present invention discloses a multi-channel voiceprint recognition method, device and equipment based on a transformer framework. The method comprises: decomposing the audio information to be recognized by sound source, and then obtaining a three-channel spectrogram I by short-time Fourier transform; transposing, filling or truncating the same group of three-channel spectrogram I at the same time to obtain two groups of three-channel spectrogram II; inputting the two groups of three-channel spectrogram II into an improved LeViT neural network model, and using the improved LeViT neural network model to recognize and process the voiceprint of the audio information to output the voiceprint recognition information of the audio information, and the voiceprint recognition information at least includes the speaker corresponding to the audio information. Through the method provided by the present invention, by processing the audio information, audio signals of multiple channels are obtained, thereby achieving a higher accuracy in recognizing the audio; and the audio signal can be more accurately recognized by the improved LeViT neural network model, which increases the complexity of the model and greatly improves the accuracy.

Description

Technical Field

[0001] The present invention relates to the field of sound detection technology, and in particular to a multi-channel voiceprint recognition method and device, equipment and storage medium based on a transformer framework. Background Art

[0002] At present, traditional methods for voiceprint recognition tasks include Gaussian Mixture Model-UniversalBackground Model (GMM-UBM), support vector machine (SVM)-based GMM-UBMs and the i-vector models. These traditional methods have poor recognition effects on large-scale data, so deep learning is introduced to process large-scale data.

[0003] The networks that have been proposed are mainly convolution neural networks (CNNs), including single-channel CNNs and multi-channel CNNs. However, due to the limited network complexity of CNNs, the final recognition accuracy is also limited.

[0004] In view of this, it is necessary to propose further improvements to the current voiceprint recognition method. Summary of the invention

[0005] To this end, the purpose of the present invention is to solve the deficiencies in the prior art to at least a certain extent, thereby proposing a multi-channel voiceprint recognition method and device, equipment and storage medium based on a transformer framework.

[0006] In a first aspect, the present invention provides a multi-channel voiceprint recognition method based on a transformer framework, the method comprising:

[0007] The audio information to be identified is decomposed into sound sources, and then a three-channel spectrum is obtained through short-time Fourier transform. Figure I ;

[0008] For the same set of three-channel spectra Figure I Perform transposition, padding or truncation simultaneously to obtain two sets of three-channel spectra Figure II ;

[0009] The two sets of three-channel spectra Figure IIThe audio information is input into an improved LeViT neural network model, and the improved LeViT neural network model is used to perform recognition processing on the voiceprint of the audio information to output voiceprint recognition information of the audio information, wherein the voiceprint recognition information at least includes the speaker corresponding to the audio information.

[0010] In a second aspect, the present invention provides a multi-channel voiceprint recognition device based on a transformer framework, comprising:

[0011] Transformation module: used to decompose the audio information to be identified into sound sources, and then obtain a three-channel spectrum through short-time Fourier transform Figure I ;

[0012] Processing module: used to process the three-channel spectrum of the same group Figure I Perform transposition, padding or truncation simultaneously to obtain two sets of three-channel spectra Figure II ;

[0013] Identification module: used to identify the two groups of three-channel spectra Figure II The improved LeViT neural network model is inputted, and the voiceprint of the audio information is recognized and processed by using the improved LeViT neural network model to output the voiceprint recognition information of the audio information, wherein the voiceprint recognition information at least includes the speaker corresponding to the audio information.

[0014] In the third aspect, the present invention also provides a voiceprint recognition terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements each step of the multi-channel voiceprint recognition method based on the transformer framework as described in the first aspect.

[0015] In a fourth aspect, the present invention further provides a storage medium having a computer program stored thereon, and when the computer program is executed, the various steps in the multi-channel voiceprint recognition method based on the transformer framework as described in the first aspect are implemented.

[0016] The present invention provides a multi-channel voiceprint recognition method, device and equipment based on a transformer framework. The method comprises: performing sound source decomposition on the audio information to be recognized, and then obtaining a three-channel spectrum through short-time Fourier transform. Figure I ; For the same group of three-channel spectrum Figure I Perform transposition, padding or truncation simultaneously to obtain two sets of three-channel spectra Figure II ; The two sets of three-channel spectrum Figure IIThe audio information is input into the improved LeViT neural network model, and the improved LeViT neural network model is used to identify and process the voiceprint of the audio information to output the voiceprint identification information of the audio information, wherein the voiceprint identification information at least includes the speaker corresponding to the audio information. Through the method provided by the present invention, by processing the audio information, audio signals of multiple channels are obtained, thereby achieving a higher accuracy in identifying the audio; and the improved LeViT neural network model can more accurately identify the audio signal, increase the complexity of the model, and greatly improve the accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0018] Figure 1 It is a flowchart of the multi-channel voiceprint recognition method based on the transformer framework of the present invention;

[0019] Figure 2 It is a sub-process diagram of the multi-channel voiceprint recognition method based on the transformer framework of the present invention;

[0020] Figure 3 It is a flowchart of a neural network in a multi-channel voiceprint recognition method based on a transformer framework of the present invention;

[0021] Figure 4 It is another sub-process diagram of the multi-channel voiceprint recognition method based on the transformer framework of the present invention;

[0022] Figure 5 It is a flow chart of the self-attention module of the multi-channel voiceprint recognition method based on the transformer framework of the present invention;

[0023] Figure 6 It is another sub-process diagram of the multi-channel voiceprint recognition method based on the transformer framework of the present invention;

[0024] Figure 7 It is another sub-process diagram of the multi-channel voiceprint recognition method based on the transformer framework of the present invention;

[0025] Figure 8It is a flow chart of the shrinking self-attention module of the multi-channel voiceprint recognition method based on the transformer framework of the present invention;

[0026] Fig. 9 It is a schematic diagram of the program module of the multi-channel voiceprint recognition method based on the transformer framework of the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, features and advantages of the invention more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0028] Please refer to Figure 1 , Figure 1 : is a flow chart of a multi-channel voiceprint recognition method based on a transformer framework in an embodiment of the present application. In this embodiment, the multi-channel voiceprint recognition method based on a transformer framework includes:

[0029] Step 101: Decompose the audio source of the audio information to be identified, and then obtain a three-channel spectrum through short-time Fourier transform. Figure I .

[0030] In this embodiment, the audio information to be identified is first separated into harmonics and percussive waves through HPSS (harmonic-percussive source separation) in libROSA, and then the source signal of the harmonics and percussive waves separated from the original audio information is added to obtain a three-channel audio signal. Then, the three-channel audio signal is subjected to short-time Fourier transform to obtain a three-channel spectrum. Figure I Among them, libROSA is a Python package for music and audio analysis.

[0031] In this embodiment, when performing short-time Fourier transform on the three-channel audio signal, the length of the Hamming window is set to 320 and the jump length is 160, so that a three-channel spectrum of 161×n can be obtained. Figure I , n is the length of the audio. In the audio processing stage, different window functions can be used, such as rectangular window (rectangular), Hanning window (hann), etc.

[0032] Step 102: For the same group of three-channel spectrum Figure IPerform transposition, padding or truncation simultaneously to obtain two sets of three-channel spectra Figure II .

[0033] In this embodiment, a three-channel spectrum of 161×n is obtained. Figure I Then, the three-channel spectrum is analyzed separately. Figure I Transpose to get the n×161 three-channel spectrum Figure I , and then the three-channel spectrum of n×161 Figure I Perform padding or phase to obtain two sets of three-channel spectrum of 300×161 Figure II .

[0034] In this embodiment, for the same set of three-channel spectrum Figure I Two sets of three-channel spectrograms are obtained by performing transposition, padding or truncation twice at the same time.

[0035] Step 103: The two groups of three-channel spectrum Figure II The voiceprint of the audio information is input into an improved LeViT neural network model, and the improved LeViT neural network model is used to perform recognition processing on the voiceprint of the audio information to output voiceprint recognition information of the audio information, wherein the voiceprint recognition information at least includes the speaker corresponding to the audio information.

[0036] In this embodiment, the obtained two sets of three-channel spectrum Figure II As a picture, it is input into the improved LeViT neural network model, where the improved LeViT neural network inputs two sets of three-channel spectrum Figure II The voiceprint in the audio information to be identified is recognized and processed to obtain the voiceprint recognition information of the audio information to be identified, and then the identity information of the speaker of the audio to be identified can be obtained.

[0037] In this embodiment, the improved LeViT neural network model is a visual transformer framework for fast reasoning. The improved LeViT neural network model itself is used to recognize images, and here it is used to recognize audio data. A pre-trained model based on a large amount of image data can be used as the starting point of a speech recognition model, thereby solving the problem of insufficient data for large-scale speech recognition tasks.

[0038] In this embodiment, the performance of the transformer model depends on a large amount of training data, but there is not much high-quality voice data, so we applied an image-based pre-trained model to transfer the knowledge learned in the image field to voiceprint recognition. We first imported a pre-trained model using the ImageNet dataset as the starting point for our training, so we only need to fine-tune the parameters of the model to get a relatively high recognition accuracy. The voiceprint recognition experiment is based on the data of VoxCeleb1, and we used two methods to divide the training set and the test set. The first is to select 21,245 audios first, and then divide the audio into sentences as the training set, and the remaining audio is divided into sentences as the test set data. The second division method is to divide the audio directly into sentences, and then randomly select 85% of the sentences as the training set, and the rest as the test set.

[0039] In this embodiment, the improved LeViT neural network model is first trained in large quantities so that the improved LeViT neural network model can quickly and accurately recognize a large amount of audio information, and then the audio data to be recognized after audio processing is input into the improved LeViT neural network model for recognition processing, so as to obtain the identity information of the speaker corresponding to the audio information to be recognized.

[0040] The embodiment of the present application provides a multi-channel voiceprint recognition method, device and equipment based on a transformer framework. The method comprises: performing sound source decomposition on the audio information to be recognized, and then obtaining a three-channel spectrum by short-time Fourier transform. Figure I ; For the same group of three-channel spectrum Figure I Perform transposition, padding or truncation simultaneously to obtain two sets of three-channel spectra Figure II ; The two sets of three-channel spectrum Figure II The audio information is input into the improved LeViT neural network model, and the improved LeViT neural network model is used to identify and process the voiceprint of the audio information to output the voiceprint identification information of the audio information, wherein the voiceprint identification information at least includes the speaker corresponding to the audio information. Through the method provided by the present invention, by processing the audio information, audio signals of multiple channels are obtained, thereby achieving a higher accuracy in identifying the audio; and the improved LeViT neural network model can more accurately identify the audio signal, increase the complexity of the model, and greatly improve the accuracy.

[0041] Further, refer to Figure 2 , Figure 2is a sub-process diagram of a multi-channel voiceprint recognition method based on a transformer framework in an embodiment of the present application. In this embodiment, the improved LeViT neural network model includes a feature extraction model module, a self-attention-multilayer perceptron module and a shrinking self-attention module. Figure II Inputting the audio information into an improved LeViT neural network model and using the improved LeViT neural network model to perform recognition processing on the voiceprint of the audio information, specifically including:

[0042] Step 201: The two groups of three-channel spectra are extracted by the feature extraction module. Figure II Perform feature extraction;

[0043] Step 202: The two groups of three-channel spectra after feature extraction are sequentially processed through the first stage, the second stage, and the third stage. Figure II Processing; the first stage and the second stage include the self-attention-multilayer perceptron module, the contraction self-attention module and the multilayer perceptron module in sequence, and the third stage includes the self-attention-multilayer perceptron module and the average pooling layer module in sequence.

[0044] In this embodiment, the improved LeViT neural network model includes a feature extraction module, a self-attention-multilayer perceptron module and a shrinkage self-attention module. First, the feature extraction module is used to extract the two sets of three-channel spectra. Figure II For feature extraction, the feature extraction module consists of a 4-layer convolutional network, where the convolution kernel of each layer of the convolutional network is 3×3 and the step size is 2. The convolutional network can better extract the features and connections of the graph context. Figure II The data is input into the improved LeViT neural network model, and a 384×14×14 feature tensor is obtained through the feature extraction module in the improved LeViT neural network model. Figure II After passing through the feature extraction module, it is processed by the self-attention-multilayer perceptron module and the contraction self-attention module in the improved LeViT neural network model respectively.

[0045] In this example, see Figure 3 , Figure 3 To improve the processing flow of the LeViT neural network model, two sets of three-channel spectra are Figure II After feature extraction, two sets of three-channel spectra are Figure IIThe first stage, the second stage, and the third stage are processed in sequence, wherein the first stage includes the self-attention-multilayer perceptron module, the contraction self-attention module, and the multilayer perceptron module in sequence; the second stage includes the self-attention-multilayer perceptron module, the contraction self-attention module, and the multilayer perceptron module in sequence; the third stage includes the self-attention-multilayer perceptron module and the average pooling layer module in sequence. Among them, the self-attention module can learn the weight of each audio feature, and the multilayer perceptron module can map the audio feature to a larger dimension, which can represent more information about the audio feature; the average pooling layer is used to extract the feature map of the last convolutional layer, and output the feature points of each feature map, construct a feature vector based on all the feature points, and then input the feature vector into the SoftMax classifier (i.e., according to the category corresponding to the classification task) according to the category corresponding to the classification task. Figure 3 A supervised classifier is used in the algorithm to obtain the identity information of the speaker corresponding to the audio information to be identified.

[0046] In this embodiment, the first stage processes the three-channel spectrum Figure II The tensor dimension obtained in the final stage is 384×14×14; the tensor dimension obtained in the second stage is 512×14×14; the tensor dimension obtained in the third stage is 768×4×4; where the tensor represents the characteristics of the spectrum.

[0047] Further, for the same group of three-channel spectrum Figure I Perform transposition, padding or truncation simultaneously to obtain two sets of three-channel spectra Figure II , also includes:

[0048] The three-channel spectrum is filtered by Mel filter Figure I Perform filtering.

[0049] In this embodiment, when performing short-time Fourier transform on the three-channel audio signal, the length of the Hamming window is set to 1024 and the jump length is 512, so that a three-channel spectrum of 513×n can be obtained. Figure I , n is the length of the audio. In order to reduce the computational overhead, the three-channel spectrum is processed by the Mel rate converter Figure I , and a 64×n spectrum is obtained. Then the 64×n spectrum is transposed, padded or truncated to obtain another set of 300×64 three-channel spectrums. Figure II In addition to the Mel filter, Discrete Cosine Transform (DCT) and other transformations can also be performed to obtain Mel-frequency Cepstral Coefficients (MFCC).

[0050] In this embodiment, the first method is to perform three-channel spectrum Figure IDirectly transpose, fill or truncate; the second method is to first use the Mel filter to filter the three-channel spectrum Figure I Filter the three-channel spectrum after filtering. Figure I Transpose, fill or truncate. Among them, two methods are used to perform three-channel spectrum Figure I The processing is performed simultaneously, thus obtaining two sets of three-channel spectra Figure II .

[0051] In this embodiment, the spectrum graph obtained by filtering with a Mel filter has a sound frequency that is closer to the frequency audible to humans, and the reduced spectrum dimension can reduce the computational overhead. However, due to the loss of some information, the final recognition accuracy is lower than that of the spectrum graph not filtered with a Mel filter.

[0052] Further, refer to Figure 4 , Figure 4 is another sub-process diagram of the multi-channel voiceprint recognition method based on the transformer framework in the embodiment of the present application. In this embodiment, the self-attention module performs the three-channel spectrum Figure II Processing, including:

[0053] Step 301: extract the two sets of three-channel spectra after feature extraction. Figure II Perform a linear transformation and then calculate the first self-attention;

[0054] Step 302: transpose the first self-attention, change the dimension, and input it into the activation function for calculation, and then obtain the first tensor dimension through the first linear layer.

[0055] In this embodiment, the two sets of three-channel spectra after feature extraction are Figure II Perform linear transformation, which is simpler and more direct than convolutional networks. The first self-attention can be obtained by calculation, and then the first self-attention is transposed, dimensionally changed, and hardswish activated, and finally connected to a first linear layer to obtain a specific tensor dimension. Each stage of the self-attention-multilayer perceptron module includes 4 layers, and the dimension of the tensor is not changed.

[0056] Further, refer to Figure 6 , Figure 6 This is another sub-process diagram of the multi-channel voiceprint recognition method based on the transformer framework in the embodiment of the present application. In this embodiment, the two groups of three-channel spectra after feature extraction are Figure II Perform a linear transformation and then calculate the first self-attention, which specifically includes:

[0057] Step 401: extract the two sets of three-channel spectra after feature extraction. Figure IIPerform a linear transformation to obtain the parameters in the first self-attention, wherein the parameters include at least Q, K, and V, where Q is a query parameter, K is a key correlation parameter, and V is a value query parameter;

[0058] Step 402: Calculate the first self-attention using the parameters Q, K, and V.

[0059] In this example, see Figure 5 and Figure 6 , Figure 5 is the self-attention module, C represents the number of channels, H represents the height, W represents the width, D represents the number of heads (appearing in the self-attention model), and N represents the batchsize; the tensor dimension after feature extraction is linearly transformed to obtain the parameters Q (query), K (key), and V (value) of the first self-attention, and then the parameters of the first self-attention are substituted into the following formula (1), and the first self-attention is obtained through the following formula (1).

[0060]

[0061] Further, refer to Figure 7 , Figure 7 is another sub-process diagram of the multi-channel voiceprint recognition method based on the transformer framework in the embodiment of the present application. In this embodiment, the shrinking self-attention module is used to analyze the three-channel spectrum. Figure II Processing, including:

[0062] Step 501: the two groups of three-channel spectra processed by the self-attention-multilayer perceptron module Figure II After sampling, the parameters Q, K, and V of the second self-attention are obtained;

[0063] Step 502: Reduce the length and width of the parameter Q by half, and then calculate the second self-attention;

[0064] Step 503: transpose the second self-attention, change the dimension, input it into the activation function for calculation, and then obtain the second tensor dimension through the second linear layer.

[0065] In this example, see Figure 7 and Figure 8 , Figure 8For the shrinking self-attention module, C represents the number of channels, H represents the height, W represents the width, D represents the number of heads, and N represents the batch size. The difference between the shrinking self-attention module and the above self-attention module is that the parameter Q of the self-attention is different. The length and width of the parameter Q in the shrinking self-attention module are reduced by half, resulting in the length and width of the final output vector being half of the original. Because the length and width of the parameter Q in the shrinking self-attention module have been reduced by half, the residual connection structure is not used. Among them, the shrinking self-attention module reduces the length and width dimensions of the audio features while learning the weights.

[0066] In this embodiment, two sets of three-channel spectrum Figure II After being processed by the self-attention-multilayer perceptron module, the shrinking self-attention module first processes the two sets of three-channel spectra. Figure II After sampling and linear transformation, the parameters Q, K, and V of the second self-attention are obtained, where the length and width of the parameter Q are reduced by half. The second self-attention is calculated using the above formula (1), and the obtained second self-attention is transposed and transformed, and the activation function and the second linear layer are performed to obtain the second tensor dimension.

[0067] Further, the two groups of three-channel spectra after feature extraction are sequentially processed through the first stage, the second stage, and the third stage. Figure II After processing, it also includes:

[0068] The two groups of three-channel spectra processed by the third stage Figure II Input into the softmax classifier to obtain the speaker corresponding to the audio information.

[0069] In this embodiment, a softmax classifier is used to classify and identify the obtained tensor. The softmax classifier can determine the probability of each category. The category with the highest probability is the category output last, thereby obtaining the identity information of the speaker corresponding to the audio information.

[0070] Furthermore, in this embodiment, the VoxCeleb1 dataset is used for voiceprint recognition experiments. Two partitioning methods are used for the dataset. The first partitioning method is to partition the training set and the test set at the video slice level, and the second partitioning method is to partition the training set and the test set at the sentence level. The first partitioning method will cause the video slices in the test set to not appear at all in the training set, which is more challenging. GMM-UBM (Gaussian Mixture Model-UniversalBackground Model) and I-vectors+SVM (Support Vector Machine) are traditional methods for voiceprint recognition, and their performance on this data is not ideal. Under the second partitioning strategy, and when only STFT is used, the top1 accuracy of the single-channel model based on transformer proposed by us is 92.97%. At the same time, the top1 accuracy of the multi-channel model proposed by us is 94.94% and 98.94%, which is about 2% higher than the top1 accuracy of the single-channel model. In general, the second partitioning strategy performs better than the first strategy, because in the first strategy, the audio to which the sentences in the test set belong does not appear in the training set at all. In addition, the spectrum using only STFT is more accurate than the spectrum using Mel filter, because Mel filter reduces the complexity of the data and saves computational cost.

[0071] Furthermore, the embodiment of the present application also provides a multi-channel voiceprint recognition device 600 based on a transformer framework. Fig. 9 : is a schematic diagram of a program module of a multi-channel voiceprint recognition device based on a transformer framework in an embodiment of the present application. In this embodiment, the multi-channel voiceprint recognition device 600 based on a transformer framework includes:

[0072] Transformation module 601: used to decompose the audio information to be identified into sound sources, and then obtain a three-channel spectrum through short-time Fourier transform Figure I ;

[0073] Processing module 602: used to process the same group of three-channel spectrum Figure I Perform transposition, padding or truncation simultaneously to obtain two sets of three-channel spectra Figure II ;

[0074] Identification module 603: used to identify the two groups of three-channel spectrum Figure IIThe improved LeViT neural network model is inputted, and the voiceprint of the audio information is recognized and processed by using the improved LeViT neural network model to output the voiceprint recognition information of the audio information, wherein the voiceprint recognition information at least includes the speaker corresponding to the audio information.

[0075] The embodiment of the present application provides a multi-channel voiceprint recognition device 600 based on a transformer framework, which can achieve: decomposing the audio information to be recognized into sound sources, and then obtaining a three-channel spectrum through short-time Fourier transform Figure I ; For the same group of three-channel spectrum Figure I Perform transposition, padding or truncation simultaneously to obtain two sets of three-channel spectra Figure II ; The two sets of three-channel spectrum Figure II The audio information is input into the improved LeViT neural network model, and the improved LeViT neural network model is used to identify and process the voiceprint of the audio information to output the voiceprint identification information of the audio information, wherein the voiceprint identification information at least includes the speaker corresponding to the audio information. Through the method provided by the present invention, by processing the audio information, audio signals of multiple channels are obtained, thereby achieving a higher accuracy in identifying the audio; and the improved LeViT neural network model can more accurately identify the audio signal, increase the complexity of the model, and greatly improve the accuracy.

[0076] Furthermore, the present application also provides a voiceprint recognition terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the various steps in the multi-channel voiceprint recognition method based on the transformer framework as described above.

[0077] Furthermore, the present application also provides a storage medium on which a computer program is stored. When the computer is executed by a processor, the various steps in the multi-channel voiceprint recognition method based on the transformer framework as described above are implemented.

[0078] Each functional module in each embodiment of the present invention may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may be stored in a computer-readable storage medium.

[0079] Based on this understanding, the technical solution of the specification of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., various media that can store program codes.

[0080] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present invention is not limited by the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention. In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0081] For those skilled in the art, according to the concept of the embodiments of the present application, there may be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A multi-channel voiceprint recognition method based on transformer framework, characterized in that: The method comprises: The audio information to be identified is decomposed into sound sources, and then a three-channel spectrum graph I is obtained through short-time Fourier transform; The same group of three-channel spectrum graphs I are transposed, padded or truncated simultaneously to obtain two groups of three-channel spectrum graphs II; The two groups of three-channel spectrograms II are input into an improved LeViT neural network model, and the improved LeViT neural network model is used to identify the voiceprint of the audio information to output the voiceprint identification information of the audio information, wherein the voiceprint identification information at least includes the speaker corresponding to the audio information, wherein the improved LeViT neural network model includes a feature extraction module, a self-attention-multilayer perceptron module and a contraction self-attention module, Performing feature extraction on the two groups of three-channel spectrum graphs II by the feature extraction module; The two groups of three-channel spectrograms II after feature extraction are processed in sequence through the first stage, the second stage, and the third stage; the first stage and the second stage include the self-attention-multilayer perceptron module, the contraction self-attention module, and the multilayer perceptron module in sequence, and the third stage includes the self-attention-multilayer perceptron module and the average pooling layer module in sequence.

2. The method according to claim 1, characterized in that The same group of three-channel spectrum graphs I are transposed, padded or truncated simultaneously to obtain two groups of three-channel spectrum graphs II, and further comprising: The three-channel spectrum diagram I is filtered by a Mel filter.

3. The method according to claim 1, characterized in that The self-attention module processes the three-channel spectrogram II, specifically including: Performing a linear transformation on the two groups of three-channel spectrograms II after feature extraction, and then calculating the first self-attention; The first self-attention is transposed, dimensionally changed, and input into the activation function for calculation, and then the first tensor dimension is obtained through the first linear layer.

4. The method according to claim 3, characterized in that The linear transformation of the two groups of three-channel spectrograms II after feature extraction and then calculating the first self-attention specifically includes: Performing a linear transformation on the two groups of three-channel spectrograms II after feature extraction to obtain parameters in the first self-attention, wherein the parameters include at least Q, K, and V, where Q is a query parameter, K is a key correlation parameter, and V is a value query parameter; The first self-attention is calculated using the parameters Q, K, and V.

5. The method according to claim 2, characterized in that: The shrinking self-attention module processes the three-channel spectrogram II, specifically including: After sampling the two groups of three-channel spectrograms II processed by the self-attention-multilayer perceptron module, the parameters Q, K, and V of the second self-attention are obtained; Reduce the length and width of the parameter Q by half, and then calculate the second self-attention; The second self-attention is transposed, dimensionally changed, and input into the activation function for calculation, and then the second tensor dimension is obtained through the second linear layer.

6. The method according to claim 1, characterized in that After the two groups of three-channel spectrograms II after feature extraction are processed in the first stage, the second stage, and the third stage in sequence, the following further comprises: The two groups of three-channel spectrograms II processed in the third stage are input into a softmax classifier to obtain the speaker corresponding to the audio information.

7. A multi-channel voiceprint recognition device based on a transformer framework, characterized in that: include: Transformation module: used to decompose the audio information to be identified into sound sources, and then obtain a three-channel spectrum diagram I through short-time Fourier transform; Processing module: used for simultaneously transposing, padding or truncating the same group of three-channel spectrum graphs I to obtain two groups of three-channel spectrum graphs II; Recognition module: used for inputting the two groups of three-channel spectrograms II into the improved LeViT neural network model, and using the improved LeViT neural network model to recognize the voiceprint of the audio information and output the voiceprint recognition information of the audio information, wherein the voiceprint recognition information at least includes the speaker corresponding to the audio information, wherein the improved LeViT neural network model includes a feature extraction module, a self-attention-multilayer perceptron module and a contraction self-attention module, Performing feature extraction on the two groups of three-channel spectrum graphs II by the feature extraction module; The two groups of three-channel spectrograms II after feature extraction are processed in sequence through the first stage, the second stage, and the third stage; the first stage and the second stage include the self-attention-multilayer perceptron module, the contraction self-attention module, and the multilayer perceptron module in sequence, and the third stage includes the self-attention-multilayer perceptron module and the average pooling layer module in sequence.

8. A voiceprint recognition terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, each step of the multi-channel voiceprint recognition method based on the transformer framework is implemented.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, each step of the multi-channel voiceprint recognition method based on the transformer framework as described in any one of claims 1 to 6 is implemented.

Images