Self-supervised learning model-based speaker identification method in real sound field environment

Abstract

The invention discloses a speaker recognition method in a real sound field environment based on a self-supervised learning model, and the method comprises the steps: obtaining large-scale label-free pure audio data, and adding noise and reverberation to the large-scale label-free pure audio data; clustering and labeling the pure audio data; taking the clustering pseudo labels as learning targets, and inputting a large-scale no-label noise reverberation-containing audio database into a self-supervised learning model for training; a speaker recognition network model is constructed, the speaker recognition network model comprises a front end and a rear end, the trained self-supervised learning model is migrated to the front end, and the rear end sequentially comprises a plurality of one-dimensional convolutional networks, a time delay neural network, an aggregation and pooling network and a full-connection output network; training the speaker recognition network model; and brand new voice signals are tested. According to the method, the problem of low speaker recognition precision in strong noise and reverberation environments is solved, and the method is suitable for various application scenes and has important promotion significance in voiceprint recognition under various severe conditions.

Description

Technical Field

[0001] The present invention relates to the technical field of speaker recognition and deep learning, and specifically to a speaker recognition method in a real sound field environment based on a self-supervised learning model. Background Art

[0002] Speaker recognition is a task to identify the speaker, including speaker discrimination, speaker confirmation and multi-speaker classification. Each person's unique vocal tract structure, pronunciation habits and accent make it possible to determine the speaker's identity through voiceprint features in speech. Similar to fingerprint, face, iris and other recognition methods, voiceprint recognition has the advantages of non-contact and high recognition accuracy, and has a wide range of uses in access control, user authentication, forensic evidence, etc.

[0003] In an ideal sound field environment (the speaker's voice features are not disturbed by environmental noise, sensor noise, other speakers' voices and echoes), various speaker recognition methods based on deep neural networks have achieved excellent results in terms of error rate and other key indicators. However, the problem of speaker recognition in strong noise and reverberation when the signal-to-noise ratio is close to 0 has been less studied. In actual sound field conditions, there are often sudden noise signals with large energy and obvious speech reverberation. For example, in a closed conference room, there may be sounds of opening and closing doors, coughing, footsteps, etc. In addition, in far-field sound fields, the low signal-to-noise ratio of voice signals and acoustic sensor equipment noise cannot be ignored, which leads to a sharp decline in speaker recognition results. Therefore, in a real sound field environment, it is difficult to achieve high-precision speaker identity confirmation. Summary of the invention

[0004] Purpose of the invention: In order to solve the problem in the prior art that speaker recognition equipment is difficult to achieve high-precision speaker identity judgment in a strong noise and reverberation environment, the present invention provides a speaker recognition method for a real sound field environment based on a self-supervised learning model.

[0005] Technical solution: A method for speaker recognition in a real sound field environment based on a self-supervised learning model, comprising the following steps:

[0006] Step 1: Use microphone equipment to obtain large-scale unlabeled clean audio data, use the room impulse response function to add random reverberation interference items to the clean audio data, and linearly add noise interference items to obtain a large-scale unlabeled audio database containing noise reverberation; cluster and label the large-scale unlabeled clean audio data;

[0007] A large-scale unlabeled audio database containing noise and reverberation is input into the self-supervised learning model, and the clustering results of the corresponding large-scale unlabeled pure audio data are used as pseudo labels. The pseudo labels are used as learning targets to complete the training task of the self-supervised learning model.

[0008] Step 2: Use the self-supervised learning model trained in step 1 to build a speaker recognition network model, and train and simultaneously verify the speaker recognition network model;

[0009] Step 3: Extract test audio data from a large-scale unlabeled audio database containing noise and reverberation, and make corresponding speaker classification labels; use the speaker recognition network model obtained in step 2 to test the new speech signal.

[0010] Furthermore, in step one, the method of clustering labels is: obtaining the Mel frequency cepstrum coefficients of large-scale unlabeled clean audio data, and using the K-means++ algorithm to cluster labels for the large-scale unlabeled clean audio data.

[0011] Furthermore, in step 2, the speaker recognition network model includes a front end and a back end. The self-supervised learning model trained in step 1 is migrated to the front end as a feature extractor for speech data. The back end includes several one-dimensional convolutional networks, time-delay neural networks, aggregation and pooling networks, and fully connected output networks in sequence.

[0012] Furthermore, in step 2, training audio data is extracted from the large-scale unlabeled audio database containing noise and reverberation, corresponding speaker classification labels are made, and the labeled training audio data is input into the constructed speaker recognition network model, using the speaker's classification number as the label to complete the training and synchronous verification of the speaker recognition network model.

[0013] Furthermore, in step 2, the extracted training audio data accounts for 5%-10% of a large-scale unlabeled audio database containing noise and reverberation.

[0014] Furthermore, in step three, audio data other than training audio data is extracted from a large-scale unlabeled audio database containing noise and reverberation as test audio data.

[0015] Furthermore, in step 2, the one-dimensional convolutional network and the time-delay neural network both include a linear rectification function and a batch regularization function.

[0016] Furthermore, in step 2, the training and synchronization verification process specifically includes:

[0017] The audio features output by the front-end Transformer encoder are converted into speech features [B, D, T] through one-dimensional convolution;

[0018] Input speech features [B, D, T] into two consecutive one-dimensional convolutional neural networks to obtain speech features [B, F, T];

[0019] Input speech features [B, F, T] into three consecutive time-delay neural networks, each of which includes a residual network unit and a squeeze-excitation unit;

[0020] The output results of three consecutive time-delay neural networks are then passed through the fully connected layer, the linear rectification layer, the fully connected layer, and the S-shaped rectification layer in sequence, and the operations are readjusted to output the final results.

[0021] Furthermore, the test in step three also includes calculating evaluation indicators such as error rate for the calculated classification vector, wherein the associated evaluation indicators are false acceptance rate, false rejection rate and threshold, different threshold settings correspond to respective false acceptance rates and false rejection rates, and when the false acceptance rate is equal to the false rejection rate, it is an equal error rate.

[0022] Furthermore, the equal error rate is calculated as follows:

[0023] 1) Input the speech features output by the self-supervised learning model into the speaker recognition system to obtain the classification vector;

[0024] 2) Calculate the cosine similarity between the classification vectors corresponding to each audio, which is defined as the target score;

[0025] 3) Add the target label and target score to the scoring list;

[0026] 4) The threshold is gradually increased from 0 to 0.99 at a growth rate of 0.01, and the corresponding false acceptance rate and false rejection rate are obtained, and then the equal error rate of the entire test set speech data is calculated. The lower the equal error rate, the better the recognition effect.

[0027] Compared with the prior art, the present invention provides a speaker recognition method for a real sound field environment based on a self-supervised learning model, which has the following beneficial effects:

[0028] A large amount of pure speech data from different speakers is processed by random reverberation to generate a speech signal with reverberation that simulates a real conference room. It is then linearly superimposed with a specially made noise signal at an extremely low signal-to-noise ratio. The generated speech signal with reverberation and noise is as close to the real conference room speech signal as possible. Mixed speech containing noise and reverberation is used as input, and pseudo labels made from a large number of unlabeled pure signals are used as iteration targets. The self-supervised learning model enhances the unlabeled data in three aspects: reducing noise, alleviating reverberation, and reducing random interference in the later model. The pseudo labels provide a dynamic adaptive filter extraction unit for the subsequent speaker recognition method to a certain extent, improving the robustness of the feature to strong noise and reverberation.

[0029] By introducing the cross-channel fusion convolution mechanism into the residual network, the contextual attention and fusion of adjacent channel areas in the speech features are strengthened; the non-average attention mechanism is introduced into the squeeze excitation network, so that each channel performs an average operation on the weight information at the time domain frame level, strengthening the difference in the contribution of different frames to the speaker information; the multi-head attention statistical pooling mechanism is introduced to replace the attention score statistical pooling operation, strengthening the difference in the contribution of different frequency bands to the time domain frame, and dividing the full channel space into several independent sub-channel spaces to minimize the mutual interference between the sub-channel spaces. The speaker recognition network is optimized from multiple angles. In view of the characteristics of noise and reverberation presented in the feature extractor, improvements are made in the residual network, squeeze excitation network and pooling layer network to better process severely damaged speech. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A flowchart of a speaker recognition method in a real sound field environment based on a self-supervised learning model;

[0031] Figure 2 It is the training flowchart of the self-supervised learning model;

[0032] Figure 3 It is the training flow chart of the speaker recognition network model. DETAILED DESCRIPTION

[0033] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments.

[0034] like Figure 1 As shown, a speaker recognition method in a real sound field environment based on a self-supervised learning model includes the following steps:

[0035] Step 1: Use microphone equipment to obtain large-scale unlabeled audio data in a low-noise and low-reverberation sound field. Each audio data contains almost no typical noise and reverberation interference items, so it can be considered as pure audio data; use the room impulse response function to add random reverberation interference items to the pure audio data, and linearly add noise interference items to obtain a large-scale unlabeled audio database containing strong noise and reverberation;

[0036] Obtain the Mel frequency cepstrum coefficients of large-scale unlabeled clean audio data, and use the K-means++ algorithm to cluster and label large-scale unlabeled clean audio data;

[0037] A large-scale unlabeled audio database containing noise and reverberation is input into a self-supervised learning model, and the clustering results of MFCC kmeans++ of the corresponding large-scale unlabeled pure audio data are used as pseudo labels. The pseudo labels are used as learning targets to complete the training task of the self-supervised learning model. The self-supervised learning model includes a convolutional neural network encoder, a Transformer encoder, and a projection matrix in sequence.

[0038] In this embodiment, large-scale unlabeled speech data containing strong noise and reverberation interference items is used as input data, and the MFCC clustering result of the corresponding pure speech data is used as label data.

[0039] like Figure 2 As shown in the figure, a speech x of length L is converted into T (T=L / 320) features (the number of channels is 512) after seven convolution operations of the convolutional neural network encoder. The step size of the seven convolution operations is [5, 2, 2, 2, 2, 2], and the seven convolution kernel sizes are [10, 3, 3, 3, 3, 2, 2]. Among them, the convolutional neural network encoder participates in parameter adjustment during training.

[0040]

[0041] n, m, k, and b refer to the convolution kernel size, input channel, output channel, and output channel offset, respectively. ij , W ijk ,y k They refer to the input speech component, convolution parameters and output speech component respectively. The T features are divided into two categories: participating in masking (randomly selecting the position where the mask starts to appear, 10 consecutive features are masked and assigned a value of 0) and not participating in masking. The features participating in masking account for 20% of the total features. The output feature sequences of the features participating in masking and the features not participating in masking after passing through the Transformer encoder are Among them, the Transformer encoder participates in parameter adjustment during training. The output feature sequence is converted into the corresponding probability density function through the projection matrix and the codebook number corresponding to the pseudo label. The probability density functions corresponding to the participating mask and the non-participating mask are respectively:

[0042]

[0043] t is the specific time domain frame number, is the probability density of the feature sequence participating in the mask, is the probability density of the feature sequence that does not participate in the mask, A is the projection matrix that can be back-propagated, 0t is a frame in the feature sequence, e c-tg is the codebook after adding pseudo labels, c represents a specific category selected by the pseudo label, and ec′ are the codebooks after C classification. Each codebook participates in parameter adjustment during training. τ is the adjustment scale parameter. sim calculates the cosine similarity between two tensors.

[0044] The total cross entropy loss function is calculated for equations (2) and (3), as shown in equation (4):

[0045] L=λ·L m +(1-λ)·L um (4)

[0046] λ is the adjustment factor, L m is the loss function finally calculated by the eigenvalues ​​participating in the mask, L um is the loss function obtained by the final calculation of the feature value that does not participate in the mask, defined as Equation (5) and Equation (6),

[0047]

[0048] Among them, the subscripts m and um represent whether the speech features participate in the mask or not participate in the mask, respectively.

[0049] The self-supervised learning model uses the MFCC clustering values ​​of pure speech as pseudo labels for iterative learning, with 20 rounds of learning. During the iteration process, the convolutional neural network encoding layer, the Transformer encoding layer, and the projection matrix all participate in parameter learning and adjustment. When the self-supervised learning model is subsequently used as a feature extractor, the projection matrix is ​​removed, and the convolutional neural network encoding layer and the Transformer encoding layer do not participate in parameter adjustment. At the same time, parameter learning is performed using the loss function of the downstream task. Specifically for the speaker recognition task, the additional angle margin normalized exponential function is used as the corresponding loss function.

[0050] Step 2: construct a speaker recognition network model, which includes a front end and a back end. The self-supervised learning model trained in step 1 is migrated to the front end as a feature extractor for speech data. The back end includes several one-dimensional convolutional networks, time-delay neural networks, aggregation and pooling networks, and fully connected output networks in sequence.

[0051] In the large-scale unlabeled audio database containing noise and reverberation in step 1, a small part of the audio (5%-10%) is extracted as training audio data, and corresponding speaker classification labels are made. The labeled training audio data is input into the constructed speaker recognition network model, and the classification numbers of several speakers are used as labels to complete the training and synchronous verification of the speaker recognition network model.

[0052] The specific methods for speaker recognition model training and verification are as follows:

[0053] A small portion (5%-10%) of the large-scale unlabeled audio data containing noise and reverberation in step 1 is used as input, and the corresponding speaker number is used as the classification label.

[0054] like Figure 3 As shown in the figure, the original dimension of the speech signal is [B, L]. The self-supervised learning model trained in step 1 is used as the speech feature extractor. In order to smoothly cascade the self-supervised learning model with the back-end speaker identification network, the audio features output by the Transformer encoder are converted into the common format of the speaker identification network, namely, speech features [B, D, T], through one-dimensional convolution. Each of the following neural network sub-networks is operated by the linear rectification function and batch regularization parameters.

[0055] The speech features [B, D, T] are input into two consecutive one-dimensional convolutional neural networks to obtain the speech features [B, F, T] for dimension adjustment. The convolution kernel sizes k of the two one-dimensional convolutional neural networks are [1, 5], the input channels are 80, and the output channels are 256.

[0056]

[0057] The speech features [B, F, T] are input into three consecutive time-delay neural networks to achieve the aggregation and fusion extraction of feature information. Each layer of the time-delay neural network module contains a fusion residual network and a squeeze excitation-attention average network. In addition, the one-dimensional convolutional network in the time-delay neural network uses the dilated convolution technology to expand the perception field of view. The convolution expansion coefficients are [2, 3, 4] respectively. Each time-delay neural network contains a residual network unit and a squeeze excitation unit.

[0058] Among them, the residual network unit is shown in formula (8), and the improved fusion residual network unit is as follows Figure 3 As shown, its shape is similar to a "U" network. Split the input x into s parts, each part is expressed as x i , the number of feature channels in each group is 1 / s of the original number of channels; except x1, each group of features will undergo a 3*3 convolution, and the convolution operation is recorded as K i (); In addition to x1 and x2, the feature x of the i-th group i First, with the previous group K i-1 () output, and the result of the addition is K i() operation; in the descending stage of the "U" type network, the i-th feature is fused into the i+1-th feature through 3*3 convolution. In the ascending stage of the "U" type network, the i-th feature is fused into the i-1-th feature through 3*3 convolution. In addition, the n-th feature is convolved and directly sent to the next layer, and then 1*1 convolution operation is performed, and then fused together, finally realizing the contextual attention and fusion of adjacent channel areas. It can extract features of different receptive fields and multiple scales, and effectively extract global features and local features.

[0059]

[0060] The improved extrusion excitation unit is as follows Figure 3 As shown in equations (9) to (13), where v c represents the cth convolution kernel, x i represents the i-th input. After the conversion operation of formula (9), the extracted features are obtained, where the dimension is H*W*C, H and W are height and width respectively, C is the number of channels, and x s is the specific characteristic component of x, is the specific convolution parameter.

[0061]

[0062] Formula (10) and Formula (11) represent non-average squeezing operations, which are used to calculate the numerical distribution of each layer of channels and convert H*W*C into 1*1*C output. Specifically, x(B, F, T) is compressed, restored, and probabilized to obtain the weight coefficient α for all F channels and each T channel, and its dimension is [B, F, 1]. And x uses α to find the average in the T dimension. Among them, the linear operation corresponding to V1 and b3 represents compression, and the linear operation corresponding to V2 and b4 represents restoration to the original state. Then, the weight coefficient α is obtained through the normalized exponential operation, and then the Hadamard product with x is performed to obtain z. c .

[0063] α=Softmax(V2·tanh(V1·x+b3)+b4) (10)

[0064]

[0065] Formula (12) represents the excitation operation. W1 is multiplied by z, which is a fully connected layer operation. The dimension of W1 is C / r*C. r is a scaling parameter. The purpose is to reduce the number of channels and thus reduce the amount of calculation. The dimension of z is 1*1*C. The result of W1z is 1*1*C / r. After that, it passes through a linear rectification layer, and the output dimension remains unchanged. It is then multiplied by W2, which is also a fully connected layer operation. The dimension of W2 is C*C / r, and its output dimension is 1*1*C. Finally, it passes through the S-shaped rectification function to obtain s.

[0066] s=F ex (z, W)=σ(g(z, W))=σ(W2δ(W1z)) (12)

[0067] Formula (13) represents the channel-by-channel adjustment operation. The adjustment coefficient of each channel is Hadamard product with the original feature information to obtain the adjusted output

[0068]

[0069] Specifically, the output [B, F, 1] obtained by equation (11) passes through the fully connected layer, the linear rectifier layer, the fully connected layer, the S-shaped rectifier layer, and the readjustment operation. The linear operation corresponding to W1 and b1 represents squeezing, compressing the F channel, and the linear operation corresponding to W2 and b2 represents excitation, restoring the F channel to its original state; x f Expands to x ft (T dimension), and then rescaled with x to get

[0070] x f =sigmoid(W2·relu(W1·z c +b1)+b2) (14)

[0071]

[0072] The outputs of the three previous time-delay neural networks, each with a dimension of (B, F, T), are concatenated according to the feature dimension, and the dimension of the concatenated feature h is (B, 3*F, T). The mean and standard deviation are calculated in the T dimension, each with a dimension of (B, 3*F), and the mean and standard deviation are stacked T times in the T dimension to restore to (B, 3*F, T); the feature, mean, and standard deviation are concatenated in the F dimension to obtain a feature x with a dimension of (B, 9*F, T).

[0073] Specifically, the aggregated x is separated into k non-overlapping subspaces along the channel dimension F, After that, the features of different time dimensions in the same subchannel space are extracted Use formula (16) to calculate the factor e of each subchannel space i .

[0074] e i =v I tanh(W i ·x i +b I )+c i (16)

[0075] Where: W i∈R F×(d / k) is the first linear unit, v i ∈R F×F is the second linear unit, b i ∈R F×T and c i ∈R F×T is the double offset variable. Here we use the channel-isolated self-attention score. i Apply the exponential normalization function in the time dimension T to obtain the percentage of spatial data frames for different sub-channels α i , which reflects the importance distribution of data frames at different times.

[0076]

[0077] For k α i The concatenation is performed according to the channel dimension to obtain α (weight coefficient). At this time, the sum of each row of the feature map α in the T dimension is equal to 1, which is the effect of the activation of the normalized exponential function; and because it has the same dimension as h, α can be regarded as an attention score. The mean and standard deviation based on attention are calculated using equations (18) and (19), and the feature dimensions obtained are all (B, 3*F). Then they are concatenated according to the feature dimension to obtain the final output of the attention statistical pooling layer, with a dimension of (B, 6*F).

[0078]

[0079] in, a t 、h t , They represent the mean, the attention score corresponding to each frame, the specific features corresponding to each frame, and the standard deviation respectively. ⊙ represents the Hadamard product operation. The feature with dimension (B, 6*F) is sent to the fully connected layer to obtain a vector (B, 192). The vector is used to calculate the additional angular margin loss function to finally obtain the classification vector.

[0080] Step 3: In a large-scale unlabeled audio database containing noise and reverberation, in addition to the training audio data extracted in step 2, test audio data is extracted to make corresponding speaker classification labels; the speaker recognition network model obtained in step 2 is used to test a brand new speech signal, wherein the brand new speech signal is a speech signal that has never appeared in the audio database training set.

[0081] The evaluation index equal error rate is calculated for the calculated classification vector, where the associated evaluation index is the false acceptance rate, the false rejection rate and the threshold. Different threshold settings correspond to their own false acceptance rate and false rejection rate. When the false acceptance rate is equal to the false rejection rate, it is an equal error rate.

[0082] The error rate is calculated as follows:

[0083] 1) Input the speech features output by the self-supervised learning model into the speaker recognition system to obtain the classification vector;

[0084] 2) Calculate the cosine similarity between the classification vectors corresponding to each audio, which is defined as the target score;

[0085] 3) Add the target label and target score to the scoring list;

[0086] 4) The threshold is gradually increased from 0 to 0.99 at a growth rate of 0.01, and the corresponding false acceptance rate and false rejection rate are obtained, and then the equal error rate of the entire test set speech data is calculated. The lower the equal error rate, the better the recognition effect.

[0087] The effect of this embodiment is verified by the following comparative experiment.

[0088] The clean speaker voices used in the experiment come from the open source Chinese speech corpus, zhvoice (abbreviated as ZH). The ZH corpus is made up of 8 open source data sets such as Zhaidatatang and Zhaishell, which have been processed by noise reduction and silence removal. It contains about 3,200 speakers, about 900 hours of audio, and a sampling frequency of 16kHz. ZH is expanded to a version with both noise and reverberation for conference room scenarios, referred to as NR-ZH.

[0089] For NR-ZH, we need to simulate and generate reverberant speech first, and the room impulse response is generated by the pyroomacoustics library. It is worth noting that considering that conference rooms are usually large, NR-ZH uses an extended version of T 60 ,In addition, pyroomacoustics uses dual sound sources and dual microphones by default, ,while the simplest speaker recognition system only requires monophonic speech, ,and two sets of exactly the same sound source and microphone parameters are ,input into the whamroom module, and only the left channel of the first microphone is ,used as the reverberation output signal.

[0090] After that, a noise signal is randomly selected for each reverberated speech signal and linearly superimposed with a random signal-to-noise ratio ([0,0.5] dB). If the noise length is less than the speech length, the front part of the noise is filled with the end of the noise to make up for the speech length. The noise comes from the sound signals downloaded from the Soundsnap website (including air conditioning, fans, mobile phone vibrations, coughing, sneezing, walking, flipping through documents, keyboards, etc., a total of 1950) and the sound signals actually recorded in the working conference room using the U-PHORIAUMC202HD sound sensor produced by Behringer (sampled at 16 kHz, the length is cropped to 3 seconds, a total of 480).

[0091] The baseline models include TDNN (512 channels), Resnet-SE (each residual module is set to [32, 64, 128, 256] channels), Res2net (each residual module is set to [64, 128, 256, 512] channels), Eres2net (each residual module is set to [32, 64, 128, 256] channels), ECAPA-TDNN (512 channels) and CAM++ (32 channels).

[0092] By comparing multiple baseline models, the indicator superiority of the speaker recognition method proposed in the present invention can be verified. In a real sound field environment, due to the presence of other speakers and noise, the spectra of audio, reverberation and noise are difficult to distinguish. After being processed by the self-supervised learning model + speaker recognition network, the best recognition effect is achieved.

[0093] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A speaker recognition method in a real sound field environment based on a self-supervised learning model, characterized in that: The following steps are involved: Step 1: Use microphone equipment to obtain large-scale unlabeled clean audio data, use the room impulse response function to add random reverberation interference items to the clean audio data, and linearly add noise interference items to obtain a large-scale unlabeled audio database containing noise reverberation; cluster and label the large-scale unlabeled clean audio data; A large-scale unlabeled audio database containing noise and reverberation is input into the self-supervised learning model, and the clustering results of the corresponding large-scale unlabeled pure audio data are used as pseudo labels. The pseudo labels are used as learning targets to complete the training task of the self-supervised learning model. Step 2: Use the self-supervised learning model trained in step 1 to build a speaker recognition network model, and train and simultaneously verify the speaker recognition network model; Step 3: Extract test audio data from a large-scale unlabeled audio database containing noise and reverberation, and make corresponding speaker classification labels; use the speaker recognition network model obtained in step 2 to test the new speech signal.

2. The method for speaker recognition in a real sound field environment based on a self-supervised learning model according to claim 1, characterized in that: In step 1, the method of clustering labels is: obtaining the Mel frequency cepstrum coefficients of large-scale unlabeled clean audio data, and using the K-means++ algorithm to cluster the large-scale unlabeled clean audio data.

3. The speaker recognition method in a real sound field environment based on a self-supervised learning model according to claim 1 or 2, characterized in that: In step 2, the speaker recognition network model includes a front end and a back end. The self-supervised learning model trained in step 1 is migrated to the front end as a feature extractor for speech data. The back end includes several one-dimensional convolutional networks, time-delay neural networks, aggregation and pooling networks, and fully connected output networks in sequence.

4. The method for speaker recognition in a real sound field environment based on a self-supervised learning model according to claim 1 or 2, characterized in that: In step 2, training audio data is extracted from the large-scale unlabeled audio database containing noise and reverberation, and corresponding speaker classification labels are made. The labeled training audio data is input into the constructed speaker recognition network model, and the speaker classification number is used as the label to complete the training and synchronous verification of the speaker recognition network model.

5. The method for speaker recognition in a real sound field environment based on a self-supervised learning model according to claim 4, characterized in that: In step 2, the extracted training audio data accounts for 5%-10% of a large-scale unlabeled audio database containing noise and reverberation.

6. The method for speaker recognition in a real sound field environment based on a self-supervised learning model according to claim 4, characterized in that: In step three, audio data other than training audio data is extracted from a large-scale unlabeled audio database containing noise and reverberation as test audio data.

7. The method for speaker recognition in a real sound field environment based on a self-supervised learning model according to claim 1 or 2, characterized in that: In step 2, the one-dimensional convolutional network and the time-delay neural network both include a linear rectification function and a batch regularization function.

8. The method for speaker recognition in a real sound field environment based on a self-supervised learning model according to claim 1 or 2, characterized in that: In step 2, the training and synchronous verification process specifically includes: The audio features output by the front-end Transformer encoder are converted into speech features [B, D, T] through one-dimensional convolution; Input speech features [B, D, T] into two consecutive one-dimensional convolutional neural networks to obtain speech features [B, F, T]; Input speech features [B, F, T] into three consecutive time-delay neural networks, each of which includes a residual network unit and a squeeze-excitation unit; The output results of three consecutive time-delay neural networks are then passed through the fully connected layer, the linear rectification layer, the fully connected layer, and the S-shaped rectification layer in sequence, and the operations are readjusted to output the final results.

9. The method for speaker recognition in a real sound field environment based on a self-supervised learning model according to claim 1 or 2, characterized in that: The test in step three also includes calculating the error rate of the evaluation indicators for the calculated classification vector, where the associated evaluation indicators are the false acceptance rate, the false rejection rate and the threshold. Different threshold settings correspond to their own false acceptance rate and false rejection rate. When the false acceptance rate is equal to the false rejection rate, it is an equal error rate.

10. The method for speaker recognition in a real sound field environment based on a self-supervised learning model according to claim 9, characterized in that: The error rate is calculated as follows: 1) Input the speech features output by the self-supervised learning model into the speaker recognition system to obtain the classification vector; 2) Calculate the cosine similarity between the classification vectors corresponding to each audio, which is defined as the target score; 3) Add the target label and target score to the scoring list; 4) The threshold is gradually increased from 0 to 0.99 at a growth rate of 0.01, and the corresponding false acceptance rate and false rejection rate are obtained, and then the equal error rate of the entire test set speech data is calculated. The lower the equal error rate, the better the recognition effect.

Images