A talk scenario speaker recognition method

Abstract

The application discloses a kind of talk scene speaker recognition method, it is related to speech recognition field, the present method uses the FBank spectrum of audio signal as the superficial feature representation of speaker, various local patterns in superficial speaker feature representation are enhanced by voiceprint encoder, and the depth representation of speaker feature is generated;Then use these depth representations to train PLDA classifier;Finally, using the structure re-parameter mechanism merges the redundant network layer in voiceprint encoder, and using voiceprint encoder and PLDA classifier predicts the probability that different speech signals belong to the same speaker, not only can carry out effective speaker recognition, but also can speed up the identification speed, satisfy real-time requirement.

Description

Technical Field

[0001] This invention relates to the field of speech recognition, and more specifically to a method for speaker recognition in a call scenario. Background Technology

[0002] Speaker recognition is a crucial method in biometric identification, with significant applications in criminal investigation, finance, and security. Among these applications, speaker recognition in spoken audio is a particularly challenging real-world scenario. This type of audio is characterized by three main features: low sampling rate, complex background noise, and high real-time requirements. Existing methods, often based on deep learning, perform well on high-quality audio data but struggle in low-sampling-rate and noisy environments. Furthermore, current methods tend to use complex network structures to optimize accuracy, which is insufficient for meeting real-time performance demands. Summary of the Invention

[0003] To address the aforementioned shortcomings in existing technologies, this invention provides a speaker recognition method for call scenarios, which solves the problems of poor performance and difficulty in meeting real-time requirements in existing methods in low sampling rate and noisy call environments.

[0004] To achieve the above-mentioned objectives, the technical solution adopted by this invention is as follows:

[0005] A method for speaker recognition in a phone call scenario is provided, which includes the following steps:

[0006] S1. Process the audio signal and calculate the FBank spectrum of the processed audio signal to obtain the speech dataset;

[0007] S2. Construct a first neural network model containing a voiceprint encoder and a voiceprint decoder. Based on the speech dataset, train the first neural network model in a self-supervised manner to obtain a pre-trained voiceprint encoder.

[0008] S3. Construct a speaker classifier based on a multilayer perceptron, namely the first speaker classifier; and use the network formed by concatenating the pre-trained voiceprint encoder with the first speaker classifier as the second neural network model.

[0009] S4. Train the pre-trained voiceprint encoder in the second neural network model using the speech dataset to obtain the fine-tuned voiceprint encoder.

[0010] S5. Establish a speaker classifier based on PLDA, i.e., the second speaker classifier; and use the network formed by connecting the fine-tuned voiceprint encoder and the second speaker classifier as the third neural network model.

[0011] S6. Train the second speaker classifier in the third neural network model using the speech dataset to obtain the trained third neural network model.

[0012] S7. Based on the structural reparameter mechanism, the third neural network model after compression training is used to obtain the fourth neural network model; the target audio signal is identified through the fourth neural network model to complete the speaker identification in the call scene.

[0013] Furthermore, the specific method for step S1 is as follows:

[0014] The FBank spectrum is calculated after the audio signal is segmented, windowed, and Mel filtered. Combined with the speaker labels of the audio signal, a speech dataset is obtained.

[0015] Furthermore, the voiceprint encoder includes at least 28 cascaded multi-scale convolutional blocks, an attention statistical pooling module, and a fully connected layer; wherein:

[0016] The first multi-scale convolutional block takes the FBank spectrum as input and outputs the corresponding features;

[0017] The l-th multi-scale convolutional block is used to take the features output by the (l-1)-th multi-scale convolutional block as input and output the corresponding features; where l is greater than or equal to 2; the output of the last multi-scale convolutional block is a frame-level feature.

[0018] The attention statistical pooling module and fully connected layers are used to take the frame-level features output by the last multi-scale convolutional block as input and the corresponding output as the speaker's deep features.

[0019] Furthermore, the specific processing of input data by a single multi-scale convolutional block includes the following sub-steps:

[0020] A1. Input the input data into a 1×1 convolutional layer and a 3×3 convolutional layer respectively to obtain feature maps of two different scales;

[0021] A2. Perform input batch normalization on the input data and the feature maps of two different scales respectively, and input the outputs of the three batch normalizations into three DropBlock regularization layers to obtain three normalized and regularized features.

[0022] A3. The three normalized input feature addition layers obtained in step A2 are added together and then input into the SE module, which is then activated by the Leaky ReLU function to obtain the output.

[0023] Furthermore, the specific method of using frame-level features as input and the corresponding output as the speaker's deep features includes the following sub-steps:

[0024] B1. According to the formula:

[0025] u t =f(Wht +b)

[0026] Each frame in the frame-level features is mapped to a low-dimensional subspace through a fully connected layer; where h t Represents the t-th frame in the frame-level features; W represents the linear mapping; b is the bias; f(.) represents the activation function; u t The mapped low-dimensional frame;

[0027] B2. According to the formula:

[0028]

[0029] Obtain the attention weights for each frame in the frame-level features; where α t is the attention weight of the t-th frame in the frame-level feature; exp represents the exponent with the natural constant e as the base; T is the total number of frames in the frame-level feature, which is also the total number of frames in the FBank spectrum; Indicate u t The transpose of ; k is the bias;

[0030] B3. According to the formula:

[0031]

[0032]

[0033] Obtain the weighted average of frame-level features and weighted standard deviation Where ⊙ represents element-wise multiplication;

[0034] B4. According to the formula:

[0035]

[0036] The speaker's deep features R are obtained; where conc[.] is the concatenation function.

[0037] Furthermore, the voiceprint decoder comprises a fully connected layer, a 1D convolutional layer, a warp module, and a multi-scale transposed convolutional network connected in sequence; wherein:

[0038] Fully connected layers are used to map the output of the voiceprint encoder to a high-dimensional subspace to obtain high-dimensional feature vectors;

[0039] 1D convolutional layers are used to increase the number of channels in high-dimensional feature vectors while maintaining the feature dimension of high-dimensional feature vectors, resulting in multi-channel high-dimensional feature vectors;

[0040] The deformation module is used to deform the same-dimensional components of each channel vector in the multi-channel high-dimensional feature vector into a two-dimensional feature map to obtain a multi-channel feature map.

[0041] A multi-scale transposed convolutional network is used to restore the multi-channel feature map to the FBank spectrum, which is the output of the first neural network model.

[0042] Furthermore, the multi-scale transposed convolutional network comprises 15 multi-scale transposed convolutional modules, 1 2D transposed convolutional layer, 5 multi-scale transposed convolutional modules, 1 2D transposed convolutional layer, 3 multi-scale transposed convolutional modules, 2 2D transposed convolutional layers, 1 multi-scale transposed convolutional module, and 1 2D transposed convolutional layer, all connected in series. The specific processing steps of a single multi-scale transposed convolutional block for the input include the following sub-steps:

[0043] C1. Input the input data into a 1×1 transposed convolutional layer and a 3×3 transposed convolutional layer respectively to obtain two different scales of multi-channel feature maps;

[0044] C2. Perform batch normalization and DropBlock regularization on the original input data and the two different scales of multi-channel feature maps obtained in step C1 to obtain three normalized and regularized features.

[0045] C3. The three normalized features obtained in step C2 are added together and input into the SE module, and then activated by the LeakyReLU function to obtain the output of the multi-scale transposed convolutional block.

[0046] Furthermore, the loss function expression for training the voiceprint encoder in the first neural network model using a self-supervised method in step S2 is as follows:

[0047]

[0048] Where loss1 represents the loss value; N represents the training sample batch size; D represents the total number of components contained in the FBank spectrum; and T represents the total number of frames in the FBank spectrum. Let j represent the j components of the FBank spectrum of the i-th audio signal; Let j represent the j-th components of the FBank spectrum of the i-th audio signal output by the first neural network model; This is an L2 regularization term.

[0049] Furthermore, the loss function expression for training the pre-trained voiceprint encoder in the second neural network model in step S4 is as follows:

[0050]

[0051]

[0052] Where loss2 represents the loss value; N represents the batch size of the training samples; E represents the number of speakers; y ijIt is a function that takes the value 1 or 0, when y ij A value of 1 indicates that the i-th sample belongs to the j-th speaker; when y... ij A value of 0 indicates that the i-th sample does not belong to the j-th speaker; p ij Let represent the probability that the i-th sample belongs to the j-th speaker; log(.) represents the logarithm to the base 10; e is the natural constant; z ij z represents the confidence level that the i-th sample belongs to the j-th speaker; ik This represents the confidence level that the i-th sample belongs to the k-th speaker.

[0053] Furthermore, the specific method for compressing the trained third neural network model based on the structural reparameter refactoring mechanism in step S7 includes the following sub-steps:

[0054] S7-1. For each multi-scale convolutional block in the voiceprint encoder of the trained third neural network model, the three-way structure before feature addition is converted into an equivalent 3×3 convolutional layer, resulting in three 3×3 convolutional kernels.

[0055] S7-2. Add the three 3×3 convolution kernels obtained in step S7-1 to obtain a single 3×3 convolution kernel;

[0056] S7-3. Replace the structure before the SE module in the multi-scale convolutional block with the convolutional layer corresponding to the single 3×3 convolutional kernel obtained in step S7-2 to complete the compression of the trained third neural network model based on the structure reparameter mechanism.

[0057] The beneficial effects of this invention are as follows: This method uses the FBank spectrum of the audio signal as a shallow feature representation of the speaker, enhances various local patterns in the shallow speaker feature representation through a voiceprint encoder, and generates a deep representation of the speaker features; then, it uses these deep representations to train a PLDA classifier; finally, it uses a structural reparameter mechanism to merge redundant network layers in the voiceprint encoder, and uses the voiceprint encoder and PLDA classifier to predict the probability that different speech signals belong to the same speaker. This not only enables effective speaker recognition but also speeds up the recognition process, solving the problems of poor performance and difficulty in meeting real-time requirements in existing technologies in low sampling rate and noisy call environments. Attached Figure Description

[0058] Figure 1 This is a flowchart illustrating the method.

[0059] Figure 2 This is a schematic diagram of the structure of a multi-scale convolutional block;

[0060] Figure 3 This is a schematic diagram of the structure of a multi-scale transposed convolutional block. Detailed Implementation

[0061] The specific embodiments of the present invention are described below to enable those skilled in the art to understand the present invention. However, it should be understood that the present invention is not limited to the scope of the specific embodiments. For those skilled in the art, various changes are obvious as long as they are within the spirit and scope of the present invention as defined and determined by the appended claims. All inventions utilizing the concept of the present invention are protected.

[0062] like Figure 1 As shown, the speaker recognition method for this call scenario includes the following steps:

[0063] S1. Process the audio signal and calculate the FBank spectrum of the processed audio signal to obtain the speech dataset;

[0064] S2. Construct a first neural network model containing a voiceprint encoder and a voiceprint decoder. Based on the speech dataset, train the first neural network model in a self-supervised manner to obtain a pre-trained voiceprint encoder. The speech dataset in this step can be an unlabeled speech dataset.

[0065] S3. Construct a speaker classifier based on a multilayer perceptron, namely the first speaker classifier; and use the network formed by concatenating the pre-trained voiceprint encoder with the first speaker classifier as the second neural network model.

[0066] S4. Train the pre-trained voiceprint encoder in the second neural network model using the speech dataset to obtain the fine-tuned voiceprint encoder; the dataset in this step is a labeled speech dataset.

[0067] S5. Establish a speaker classifier based on PLDA, i.e., the second speaker classifier; and use the network formed by connecting the fine-tuned voiceprint encoder and the second speaker classifier as the third neural network model.

[0068] S6. Train the second speaker classifier in the third neural network model using the speech dataset to obtain the trained third neural network model; the dataset in this step is a labeled speech dataset.

[0069] S7. Based on the structural reparameter mechanism, the third neural network model after compression training is used to obtain the fourth neural network model; the target audio signal is identified through the fourth neural network model to complete the speaker identification in the call scene.

[0070] The specific method for step S1 is as follows:

[0071] The FBank spectrum is calculated after the audio signal is segmented, windowed, and Mel filtered. Combined with the speaker labels of the audio signal, a speech dataset is obtained.

[0072] The voiceprint encoder comprises at least 28 cascaded multi-scale convolutional blocks, an attention statistical pooling module, and a fully connected layer; wherein:

[0073] The first multi-scale convolutional block takes the FBank spectrum as input and outputs the corresponding features;

[0074] The l-th multi-scale convolutional block is used to take the features output by the (l-1)-th multi-scale convolutional block as input and output the corresponding features; where l is greater than or equal to 2; the output of the last multi-scale convolutional block is a frame-level feature.

[0075] The attention statistical pooling module and fully connected layers are used to take the frame-level features output by the last multi-scale convolutional block as input and the corresponding output as the speaker's deep features.

[0076] like Figure 2 As shown, the specific processing steps of a single multi-scale convolutional block on the input data include the following sub-steps:

[0077] A1. Input the input data into a 1×1 convolutional layer and a 3×3 convolutional layer respectively to obtain feature maps of two different scales;

[0078] A2. Perform input batch normalization on the input data and the feature maps of two different scales respectively, and input the outputs of the three batch normalizations into three DropBlock regularization layers to obtain three normalized and regularized features.

[0079] A3. The three normalized input feature addition layers obtained in step A2 are added together and then input into the SE module, which is then activated by the Leaky ReLU function to obtain the output.

[0080] The specific method of using frame-level features as input and the corresponding output as the speaker's deep features includes the following sub-steps:

[0081] B1. According to the formula:

[0082] u t =f(Wh t +b)

[0083] Each frame in the frame-level features is mapped to a low-dimensional subspace through a fully connected layer; where h t Represents the t-th frame in the frame-level features; W represents the linear mapping; b is the bias; f(.) represents the activation function; u t The mapped low-dimensional frame;

[0084] B2. According to the formula:

[0085]

[0086] Obtain the attention weights for each frame in the frame-level features; where α t is the attention weight of the t-th frame in the frame-level feature; exp represents the exponent with the natural constant e as the base; T is the total number of frames in the frame-level feature, which is also the total number of frames in the FBank spectrum; Indicate u t The transpose of ; k is the bias;

[0087] B3. According to the formula:

[0088]

[0089]

[0090] Obtain the weighted average of frame-level features and weighted standard deviation Where ⊙ represents element-wise multiplication;

[0091] B4. According to the formula:

[0092]

[0093] The speaker's deep features R are obtained; where conc[.] is the concatenation function.

[0094] The voiceprint decoder consists of a fully connected layer, a 1D convolutional layer, a warp module, and a multi-scale transposed convolutional network connected in sequence; wherein:

[0095] Fully connected layers are used to map the output of the voiceprint encoder to a high-dimensional subspace to obtain high-dimensional feature vectors;

[0096] 1D convolutional layers are used to increase the number of channels in high-dimensional feature vectors while maintaining the feature dimension of high-dimensional feature vectors, resulting in multi-channel high-dimensional feature vectors;

[0097] The deformation module is used to deform the same-dimensional components of each channel vector in the multi-channel high-dimensional feature vector into a two-dimensional feature map to obtain a multi-channel feature map.

[0098] A multi-scale transposed convolutional network is used to restore the multi-channel feature map to the FBank spectrum, which is the output of the first neural network model.

[0099] The multi-scale transposed convolutional network consists of 15 multi-scale transposed convolutional modules, 1 2D transposed convolutional layer, 5 multi-scale transposed convolutional modules, 1 2D transposed convolutional layer, 3 multi-scale transposed convolutional modules, 2 2D transposed convolutional layers, 1 multi-scale transposed convolutional module, and 1 2D transposed convolutional layer, all connected in series. Figure 3 As shown, the specific processing steps of a single multi-scale transposed convolutional block for the input include the following sub-steps:

[0100] C1. Input the input data into a 1×1 transposed convolutional layer and a 3×3 transposed convolutional layer respectively to obtain two different scales of multi-channel feature maps;

[0101] C2. Perform batch normalization and DropBlock regularization on the original input data and the two different scales of multi-channel feature maps obtained in step C1 to obtain three normalized and regularized features.

[0102] C3. The three normalized features obtained in step C2 are added together and input into the SE module, and then activated by the LeakyReLU function to obtain the output of the multi-scale transposed convolutional block.

[0103] The loss function expression for training the voiceprint encoder in the first neural network model using a self-supervised method in step S2 is as follows:

[0104]

[0105] Where loss1 represents the loss value; N represents the training sample batch size; D represents the total number of components contained in the FBank spectrum; and T represents the total number of frames in the FBank spectrum. Let j represent the j components of the FBank spectrum of the i-th audio signal; Let j represent the j-th components of the FBank spectrum of the i-th audio signal output by the first neural network model; This is an L2 regularization term.

[0106] The loss function expression for training the pre-trained voiceprint encoder in the second neural network model in step S4 is as follows:

[0107]

[0108]

[0109] Where loss2 represents the loss value; N represents the batch size of the training samples; E represents the number of speakers; y ij It is a function that takes the value 1 or 0, when y ij A value of 1 indicates that the i-th sample belongs to the j-th speaker; when y... ij A value of 0 indicates that the i-th sample does not belong to the j-th speaker; p ij Let represent the probability that the i-th sample belongs to the j-th speaker; log(.) represents the logarithm to the base 10; e is the natural constant; z ij z represents the confidence level that the i-th sample belongs to the j-th speaker; ik This represents the confidence level that the i-th sample belongs to the k-th speaker.

[0110] The specific method for compressing the trained third neural network model based on the structural reparameter recombination mechanism in step S7 includes the following sub-steps:

[0111] S7-1. For each multi-scale convolutional block in the voiceprint encoder of the trained third neural network model, the three-way structure before feature addition is converted into an equivalent 3×3 convolutional layer, resulting in three 3×3 convolutional kernels.

[0112] S7-2. Add the three 3×3 convolution kernels obtained in step S7-1 to obtain a single 3×3 convolution kernel;

[0113] S7-3. Replace the structure before the SE module in the multi-scale convolutional block with the convolutional layer corresponding to the single 3×3 convolutional kernel obtained in step S7-2 to complete the compression of the trained third neural network model based on the structure reparameter mechanism.

[0114] In one embodiment of the present invention, the training data used in the experiment includes the CN-Celeb2 dataset and the Datatang 200zh dataset.

[0115] The CN-Celeb2 dataset was used for the self-supervised denoising training phase, specifically the training phase of the first neural network model in step S2 using a self-supervised approach. This dataset contains 529,485 audio clips totaling 1090 hours recorded by 2000 speakers from multiple media platforms including NetEase Cloud Music, Changba, Himalaya, and Douyin, covering various speaking scenarios such as vlogs and live streams. Furthermore, the audio clips in this dataset exhibit a wide range of lengths and high speaker diversity, encompassing different ages, genders, regions, occupations, and language habits. The rich sample information in the dataset allows the model to better learn different features and patterns, thereby improving its adaptability to data fluctuations and changes, and enhancing its generalization performance.

[0116] The DataTang 200zh dataset was used for the fine-tuning phase of speaker recognition, specifically the training phase of the pre-trained voiceprint encoder in the second neural network model in step S4 and the training phase of the second speaker classifier in the third neural network model in step S6. This dataset, produced by Beijing DataTang Technology Co., Ltd., contains 200 hours of speech data from 600 speakers from different accent regions in China. The moderate data size and diverse speaker accents help the model quickly transfer general voiceprint knowledge to the field of speaker recognition.

[0117] In this embodiment, the experimental environment for this method is shown in Table 1 below:

[0118] Table 1

[0119]

[0120] The hyperparameters used in this paper during the U-RepVGG self-supervised denoising training phase are shown in Table 2 below:

[0121] Table 2

[0122]

[0123]

[0124] In the supervised training phase of the speaker recognition task using the fourth neural network model (U-RepVGG), the existing model X-Vector, and the existing model ECAPA-TDNN, the hyperparameters used in this paper are similar to those used in the self-supervised denoising phase. The differences are shown in Table 3 below:

[0125] Table 3

[0126] Random Seed 1991 Train Set:Valid Set:Test Set 6:1:3 Total Epochs 20 Batch Size 96

[0127] The performance evaluation metrics in this embodiment include APS, precision, recall, the harmonic mean of precision and recall (F1Score), the equal error rate (EER) when the false negative rate and false positive rate are equal, and the minimum detection cost function (minDCF) which considers the different costs caused by misclassification in different application scenarios based on EER. To verify the impact of various optimization strategies on model performance, we conducted module ablation experiments on the U-RepVGG model, and the results are shown in Table 4 below:

[0128] Table 4

[0129]

[0130]

[0131] As shown in Table 4, although the APS decreased slightly after using the attention statistical pooling module, there were significant improvements in minDCF and EER. This is because the attention statistical pooling module can better capture important information in the audio, thereby improving the model's accuracy.

[0132] The model without DropBlock regularization showed a significant performance decrease compared to the model using this regularization method. Specifically, the minDCF increased from 0.442 to 1.00, and the EER rose from 2.252% to 48.579%. This is because the model complexity was too high without DropBlock regularization, leading to severe overfitting during training. Therefore, DropBlock regularization is effective in improving the generalization performance of this method.

[0133] The model without the SE module showed a slight performance decrease compared to the model using it. Specifically, minDCF improved by 8.14%, and EER increased by 2.18%. This is because the SE module helps the model better capture key features in the audio. However, for speaker recognition, the model without the SE module was already well-versed in learning the key features, so the module had little impact on model performance.

[0134] The model without self-supervised training showed a significant performance decrease compared to the model using this method. Specifically, the minDCF improved to 0.993, and the EER increased to 34.613%. This is because self-supervised training helps the model better learn generalized voiceprint feature representations, enhances robustness to call noise, and thus improves the model's generalization performance.

[0135] Without audio resampling and data augmentation, the model performed poorly on the minDCF and EER metrics. However, after using audio resampling and data augmentation, both the minDCF and EER metrics showed significant improvements, indicating that audio resampling and data augmentation are effective in improving model performance. This is because audio resampling makes the training data closer to real-world call scenarios, while data augmentation helps the model better adapt to different types of audio, thereby improving the model's generalization performance and robustness.

[0136] Various optimization strategies have a positive impact on the performance of the U-RepVGG model, improving its performance. However, the contribution of different optimization strategies varies depending on the different metrics and tasks, and the appropriate strategy should be selected based on the specific circumstances.

[0137] To further illustrate the advantages of our proposed method in speaker recognition tasks within call scenarios, we compared U-RepVGG with two commonly used speaker recognition models, Xvector and ECAPA-TDNN. The results are shown in Table 5 below:

[0138] Table 5

[0139]

[0140] As shown in Table 5, the reparameterized U-RepVGG model significantly reduces the number of parameters and model size while maintaining accuracy, thus substantially improving inference speed. Comparison with Xvector and ECAPA-TDNN reveals that U-RepVGG achieves near-state-of-the-art (SOTA) accuracy while maintaining inference speed close to Xvector. This demonstrates that the proposed method balances accuracy and speed, making it well-suited for the needs of voiceprint recognition in call scenarios.

Claims

1. A method for speaker recognition in a phone call scenario, characterized in that, Includes the following steps: S1. Process the audio signal and calculate the FBank spectrum of the processed audio signal to obtain the speech dataset; S2. Construct a first neural network model containing a voiceprint encoder and a voiceprint decoder. Based on the speech dataset, train the first neural network model in a self-supervised manner to obtain a pre-trained voiceprint encoder. S3. Construct a speaker classifier based on a multilayer perceptron, namely the first speaker classifier; and use the network formed by concatenating the pre-trained voiceprint encoder with the first speaker classifier as the second neural network model. S4. Train the pre-trained voiceprint encoder in the second neural network model using the speech dataset to obtain the fine-tuned voiceprint encoder. S5. Establish a speaker classifier based on PLDA, i.e., the second speaker classifier; and use the network formed by connecting the fine-tuned voiceprint encoder and the second speaker classifier as the third neural network model. S6. Train the second speaker classifier in the third neural network model using the speech dataset to obtain the trained third neural network model. S7. Based on the structural reparameter mechanism, the third neural network model after compression training is used to obtain the fourth neural network model; the target audio signal is identified through the fourth neural network model to complete the speaker identification in the call scene.

2. The speaker recognition method in a call scene according to claim 1, characterized in that, The specific method for step S1 is as follows: The FBank spectrum is calculated after the audio signal is segmented, windowed, and Mel filtered. Combined with the speaker labels of the audio signal, a speech dataset is obtained.

3. The speaker recognition method in a call scene according to claim 1, characterized in that, The voiceprint encoder comprises at least 28 cascaded multi-scale convolutional blocks, an attention statistical pooling module, and a fully connected layer; wherein: The first multi-scale convolutional block takes the FBank spectrum as input and outputs the corresponding features; The l-th multi-scale convolutional block is used to take the features output by the (l-1)-th multi-scale convolutional block as input and output the corresponding features; where l is greater than or equal to 2; the output of the last multi-scale convolutional block is the frame-level feature. The attention-based statistical pooling module and fully connected layers are used to take the frame-level features output from the last multi-scale convolutional block as input and the corresponding output as the speaker's deep features.

4. The speaker recognition method in a call scene according to claim 3, characterized in that, The specific processing steps of a single multi-scale convolutional block on the input data include the following sub-steps: A1. Input the input data into a 1×1 convolutional layer and a 3×3 convolutional layer respectively to obtain feature maps of two different scales; A2. Perform batch normalization on the input data and the feature maps of two different scales respectively, and input the outputs of the three batch normalizations into three DropBlock regularization layers to obtain three normalized features. A3. The three normalized input feature addition layers obtained in step A2 are added together and then input into the SE module, which is then activated by the Leaky ReLU function to obtain the output.

5. The speaker recognition method in a call scene according to claim 3, characterized in that, The specific method of using frame-level features as input and the corresponding output as the speaker's deep features includes the following sub-steps: B1. According to the formula: u t = f(W t +h) Each frame in the frame-level feature is mapped to a low-dimensional subspace by a fully connected layer; wherein h t represents the t-th frame in the frame-level feature; W represents a linear mapping; b is a bias; f(.) represents an activation function; u t is the mapped low-dimensional frame; B2. According to the formula: Obtain the attention weights for each frame in the frame-level features; where α t is the attention weight of the t-th frame in the frame-level feature; exp represents the exponent with the natural constant e as the base; T is the total number of frames in the frame-level feature, which is also the total number of frames in the FBank spectrum; Indicate u t The transpose of ; k is the bias; B3. According to the formula: Obtain the weighted average of frame-level features and weighted standard deviation Where ⊙ represents element-wise multiplication; B4. According to the formula: Obtain the speaker's deep features R; Where conc[.] is the concatenation function.

6. The speaker recognition method in a call scene according to claim 3, characterized in that, The voiceprint decoder consists of a fully connected layer, a 1D convolutional layer, a warp module, and a multi-scale transposed convolutional network connected in sequence; wherein: Fully connected layers are used to map the output of the voiceprint encoder to a high-dimensional subspace to obtain high-dimensional feature vectors; 1D convolutional layers are used to increase the number of channels in high-dimensional feature vectors while maintaining the feature dimension of high-dimensional feature vectors, resulting in multi-channel high-dimensional feature vectors; The deformation module is used to deform the same-dimensional components of each channel vector in the multi-channel high-dimensional feature vector into a two-dimensional feature map to obtain a multi-channel feature map. A multi-scale transposed convolutional network is used to restore the multi-channel feature map to the FBank spectrum, which is the output of the first neural network model.

7. The speaker recognition method in a call scene according to claim 6, characterized in that, The multi-scale transposed convolutional network comprises 15 multi-scale transposed convolutional modules, 1 2D transposed convolutional layer, 5 multi-scale transposed convolutional modules, 1 2D transposed convolutional layer, 3 multi-scale transposed convolutional modules, 2 2D transposed convolutional layers, 1 multi-scale transposed convolutional module, and 1 2D transposed convolutional layer, all connected in series. The specific processing steps of a single multi-scale transposed convolutional block for the input include the following sub-steps: C1. Input the input data into a 1×1 transposed convolutional layer and a 3×3 transposed convolutional layer respectively to obtain two different scales of multi-channel feature maps; C2. Perform batch normalization and DropBlock regularization on the original input data and the two different scales of multi-channel feature maps obtained in step C1 to obtain three normalized and regularized features. C3. The three normalized features obtained in step C2 are added together and input into the SE module, and then activated by the Leaky ReLU function to obtain the output of the multi-scale transposed convolutional block.

8. The speaker recognition method in a call scene according to claim 1, characterized in that, The loss function expression for training the voiceprint encoder in the first neural network model using a self-supervised method in step S2 is as follows: Where loss1 represents the loss value; N represents the training sample batch size; D represents the total number of components contained in the FBank spectrum; and T represents the total number of frames in the FBank spectrum. Let j represent the j components of the FBank spectrum of the i-th audio signal; Let j represent the j components of the FBank spectrum of the i-th audio signal output by the first neural network model; This is an L2 regularization term.

9. The speaker recognition method in a call scene according to claim 1, characterized in that, The loss function expression for training the pre-trained voiceprint encoder in the second neural network model in step S4 is as follows: Where loss2 represents the loss value; N represents the batch size of the training samples; E represents the number of speakers; y ij It is a function that takes the value 1 or 0, when y ij A value of 1 indicates that the i-th sample belongs to the j-th speaker; when y... ij A value of 0 indicates that the i-th sample does not belong to the j-th speaker; p ij Let represent the probability that the i-th sample belongs to the j-th speaker; log(.) represents the logarithm to the base 10; e is the natural constant; z ij z represents the confidence level that the i-th sample belongs to the j-th speaker; ik This represents the confidence level that the i-th sample belongs to the k-th speaker.

10. The speaker recognition method in a call scene according to claim 3, characterized in that, The specific method for compressing the trained third neural network model based on the structural reparameter recombination mechanism in step S7 includes the following sub-steps: S7-1. For each multi-scale convolutional block in the voiceprint encoder of the trained third neural network model, the three-way structure before feature addition is converted into an equivalent 3×3 convolutional layer, resulting in three 3×3 convolutional kernels. S7-2. Add the three 3×3 convolution kernels obtained in step S7-1 to obtain a single 3×3 convolution kernel; S7-3. Replace the structure before the SE module in the multi-scale convolutional block with the convolutional layer corresponding to the single 3×3 convolutional kernel obtained in step S7-2 to complete the compression of the trained third neural network model based on the structure reparameter mechanism.

Images