A method for training a voiceprint recognition feature extraction model and a voiceprint recognition system

Abstract

The present application relates to a kind of training method of voiceprint recognition feature extraction model and voiceprint recognition system, belong to speech recognition technical field;Among them, training method includes: constructing voiceprint recognition training data set;Using feature encoder, the feature extraction, random masking and feature coding of speech data in data set are carried out, and lossy speech feature vector is obtained;Vector padding and position information embedding are carried out to lossy speech feature vector, and the feature vector after mask padding is obtained;Based on the feature vector after mask padding, the speech feature vector corresponding to the part being randomly shielded is predicted, and complete speech feature vector is obtained and is classified by softmax, and label class distribution is obtained;Iterative update is carried out by loss function, and the trained feature encoder is obtained;The random masking function of feature encoder is closed, and feature extraction model is obtained.The present application solves the problem that the voiceprint recognition method in prior art is poor in robustness and cannot accurately identify lossy speech in different scenarios.

Description

Technical Field

[0001] This invention relates to the field of speech recognition technology, and in particular to a training method for a voiceprint recognition feature extraction model and a voiceprint recognition system. Background Technology

[0002] The theoretical basis of voiceprint recognition is that every voice has unique characteristics, which can be used to effectively distinguish the voices of different people. Even though different people say the same thing, the frequency distribution of their voices is different; some sound deep, while others are loud. Just like fingerprints, each person's vocal tract is different, and thus, each person's voice has unique characteristics.

[0003] Furthermore, sound transmission through communication lines is inevitably affected by line noise, and different transmission lines produce different types of noise. Therefore, the challenge of voiceprint recognition in noisy environments is effectively removing various additive and multiplicative noise interferences to improve system robustness. In practical applications, various uncertainties, such as different transmission channels, complex and variable background noise, and the speaker's own physiological fluctuations, often lead to discontinuous or even lossy speech. These uncertainties pose a significant challenge to voiceprint recognition systems, making it difficult to accurately identify discontinuous or lossy speech. Summary of the Invention

[0004] Based on the above analysis, the present invention aims to provide a training method for a voiceprint recognition feature extraction model and a voiceprint recognition system; it solves the problems of poor accuracy of existing voiceprint recognition methods for lossy speech acquired in different environments and limited applicability scenarios.

[0005] The objective of this invention is mainly achieved through the following technical solutions:

[0006] On one hand, this invention discloses a training method for a voiceprint recognition feature extraction model, comprising the following steps:

[0007] Construct a voiceprint recognition training dataset, which includes voice data and labels representing the person to whom the voice data belongs;

[0008] The speech data is subjected to feature extraction, random masking, and feature encoding using a feature encoder to obtain a lossy speech feature vector.

[0009] The lossy speech feature vector is padded with vectors and embedded with positional information to obtain the mask-padded feature vector.

[0010] Based on the feature vector after mask filling, the speech feature vector corresponding to the randomly masked part is predicted to obtain the complete speech feature vector;

[0011] The complete speech feature vector is subjected to average pooling and softmax classification to obtain the label category distribution corresponding to the complete speech feature vector;

[0012] The labels obtained from the voiceprint recognition training dataset and the classifier are iteratively updated using a loss function to obtain a trained feature encoder; based on the trained feature encoder, the feature extraction model is obtained by disabling the random masking function.

[0013] Furthermore, the feature encoder includes: a feature extraction unit, a random masking unit, and a transformer unit;

[0014] The feature extraction unit is used to extract features from the input speech data to obtain a fixed-dimensional speech feature vector;

[0015] The random masking unit randomly masks the speech feature vectors output by the feature extraction unit according to a uniform distribution; and records the positions of the masked vectors and the remaining vectors, embeds position information into the remaining vectors, and obtains the lossy speech data after random masking.

[0016] The transformer unit is used to perform inter-frame attention calculation on the randomly masked lossy speech data to obtain a lossy speech feature vector that has temporal and contextual relationships.

[0017] Furthermore, the shielding rate of the random shielding is greater than 50%.

[0018] Furthermore, the step of performing vector padding and positional information embedding on the lossy speech feature vector includes: assigning a shared learning vector or a fixed non-zero vector to the masked vector; and embedding corresponding positional information into the masked vector and the remaining vector to obtain the masked feature vector.

[0019] Furthermore, the mask-padded feature vector is input into the decoder, and the complete speech feature vector is obtained using the following formula:

[0020]

[0021] in, This represents the speech feature vector after adding the padding vector. Represents location encoding information, f bn f represents the normalization operation. transformer This represents the transformer decoder.

[0022] Furthermore, the predicted vector is obtained by performing average pooling and softmax classification on the complete speech feature vector.

[0023]

[0024] The loss function formula is:

[0025]

[0026] Where x is the input speech sequence, y For speaker category labels.

[0027] Furthermore, the feature extraction model is trained using the Wav2Vec2 model as the initial model, and the encoder and decoder of the Wav2Vec2 model are designed as asymmetric structures during training.

[0028] Furthermore, the voiceprint recognition training dataset is constructed, including:

[0029] Obtain the dataset; the dataset includes speech data from multiple speakers;

[0030] Label the speech data according to the speaker ID;

[0031] The speech data is preprocessed by channel splitting, speech segmentation and unified formatting to obtain mono, fixed-length speech data, thus obtaining a voiceprint recognition training dataset.

[0032] On the other hand, a voiceprint recognition system is also disclosed, the system comprising: a user voice registration library, a feature extraction model, and a voiceprint verification model;

[0033] The user voice registration library is used to store the voice feature vectors corresponding to the user's voice and the corresponding user tags;

[0034] The feature extraction model is trained using the aforementioned training method and is used to extract features from the speech to be recognized and encode the corresponding speech feature vector.

[0035] The voiceprint verification model is used to calculate the similarity value between the voice feature vector of the speech to be identified and the voice feature vector in the voice registration database, and selects the user to which the registered voice feature vector with the highest similarity value and greater than the threshold belongs as the user to which the speech to be identified belongs.

[0036] Furthermore, the user voice registration library is constructed using user voice data, including: using a pre-trained feature extraction model of the voice input of the user to be registered to obtain the voice feature vector of the user to be registered; labeling the voice feature vector of the user to be registered based on the user ID; saving the feature vector of the voice to be registered and the corresponding label to obtain the user voice registration library.

[0037] The present invention can achieve at least the following beneficial effects:

[0038] 1. This invention uses high-density masking operation to train the voiceprint recognition feature extraction model. Based on the lossy speech feature vector output by the feature encoder, the overall speech voiceprint features are fitted and reconstructed. After iterative updates, a powerful voiceprint recognition feature extraction model is obtained, which improves the accuracy of voiceprint recognition of lossy speech in different environments.

[0039] 2. This invention can effectively reduce the size of the encoder and decoder by using random masking operations with a high masking rate, resulting in faster operation and improved training efficiency.

[0040] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0041] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0042] Figure 1 This is a flowchart illustrating the training method of the voiceprint recognition feature extraction model according to an embodiment of the present invention. Detailed Implementation

[0043] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0044] To address the issue of poor accuracy in recognizing lossy speech acquired under different environments, the present invention proposes that the feature extraction model in the voiceprint recognition system can extract all speech features using only a portion of the speech data, enabling the voiceprint recognition system to perform voiceprint recognition on discontinuous or even lossy speech.

[0045] This embodiment discloses a training method for a voiceprint recognition feature extraction model, such as... Figure 1 As shown, it includes the following steps:

[0046] Step S1: Construct a voiceprint recognition training dataset, which includes speech data and labels representing the speaker to whom the speech data belongs. The speech data is obtained by preprocessing the speech of unconstrained speakers in the Chinese speech dataset. Labels are obtained by classifying and annotating the speech of different speakers; for example, each speaker's speech is labeled with a number representing the corresponding label category.

[0047] Specifically, constructing the voiceprint recognition training dataset includes:

[0048] Obtain the dataset; the dataset includes speech data from multiple speakers;

[0049] Construct a voice data tag, wherein the voice data tag is the speaker ID corresponding to the voice data;

[0050] The speech data is preprocessed by channel splitting, speech segmentation and unified formatting to obtain monophonic, fixed-length speech data, which constitutes the voiceprint recognition training dataset.

[0051] Preferably, the dataset used in this embodiment is the Chinese speech datasets cn-celeb1 and cn-celeb2 for unconstrained speaker recognition. cn-celeb1 contains recordings or interviews of 997 people, and cn-celeb2 contains recordings or interviews of 1996 people. Speech segments of speakers with less than fifty segments are deleted, and the remaining data in the cn-celeb1 and cn-celeb2 datasets are labeled according to the speaker ID.

[0052] Further, the data is preprocessed, and the steps are as follows:

[0053] Channel splitting: All speech with more than 1 channel is split into mono speech, and silence removal is performed on the data of each channel separately: The WebRTC speech endpoint detection method is used to detect speech segments. First, the input speech is divided into several segments in 20ms increments. Then, the segment is checked to see if it is silent. If it is, the segment is deleted; otherwise, it is retained.

[0054] Speech segmentation: Speech is segmented into segments of fixed length based on min_token and max_token; segments shorter than min_token are discarded, and segments longer than max_token are truncated. In this embodiment, min_token is 56000, representing a segment of speech no less than 5 seconds, and max_token is 480000, representing a segment of speech no more than 30 seconds.

[0055] Standardized format: The segmented audio is uniformly converted into a format with a sampling rate of 16000 and a sampling precision of 16 bits.

[0056] The preprocessed speech data and corresponding labels constitute a voiceprint recognition dataset, and the training data in it is used as a voiceprint recognition training dataset.

[0057] Step S2: Use a feature encoder to extract features from the speech data, perform random masking and feature encoding to obtain a lossy speech feature vector;

[0058] Specifically, the feature encoder includes: a feature extraction unit, a random masking unit, and a transformer unit;

[0059] The feature extraction unit is used to extract features from the speech data in the input speaker recognition training dataset to obtain a fixed-dimensional speech feature vector;

[0060] The random masking unit randomly masks the speech feature vectors output by the feature extraction unit according to a uniform distribution; and records the positions of the masked vectors and the remaining vectors, embeds position information into the remaining vectors, and obtains the lossy speech data after random masking; in this embodiment, the masking rate of random masking is greater than 50%.

[0061] The transformer unit is used to perform inter-frame attention calculation on randomly masked lossy speech data to obtain lossy speech feature vectors that have temporal and contextual relationships.

[0062] Preferably, the feature encoder in this embodiment uses the wav2vec2 model as the initial model structure without modifying the original model structure. Its input is preprocessed speech sampling data from the training dataset. The original speech data, after sampling, is used as the model input. It passes through the CNN feature extraction layer in wav2vec2 to obtain a speech feature vector representation. Then, a random masking operation is performed, masking a portion of the obtained speech feature vectors while remembering the positions of the masked vectors and the remaining vectors. This embodiment uses a high masking ratio, with a masking probability greater than 50%. The remaining speech vectors are then sequentially arranged, and the embedded position vector information is input into the transformer structure of wav2vec2 to obtain the lossy speech vector representation output by the feature encoder.

[0063] More specifically, based on the wav2vec2 model structure, the feature encoder obtains X = (x0, x1, ..., x2) after sampling the original speech data. lAs input, the signal is processed through a 7-layer convolutional network, with the output of each layer serving as the input to the next. The stride of each layer is (5, 2, 2, 2, 2, 2, 2), and the kernel width is (10, 3, 3, 3, 3, 2, 2). After feature encoding, a speech feature vector with a fixed dimension of 512 is generated, resulting in the hidden layer feature C = (c0, c1, ..., c2). L The vector has dimensions (1, L, 512), where L is equal to l / 320. The 512-dimensional vector is then mapped to a 768-dimensional space to obtain the speech feature vector representation S. Next, the feature vector S is randomly masked, and the remaining vectors after masking are embedded with positional information. This information is then fed into a 12-layer block for inter-frame attention calculation. Each block is a transformer structure with 768 hidden units and a self-attention network, which fully utilizes the transformer's powerful ability to handle temporal relationships. This allows for sufficient contextual encoding of the features of each speech frame, resulting in speech feature vectors that have temporal relationships and contain contextual relationships. The output features of the speech encoded by the 12-layer transformer are: H = (h0, h1, ..., h...). l ), with dimensions [1, 1, 768]; where vectors S and H can be represented as:

[0064] S = f bn (Wf cnn (X)+b);

[0065]

[0066] Among them, f cnn Represents a CNN feature extractor, W∈R p*q This represents a mapping from 512 dimensions to 768 dimensions, where p and q represent the dimensions, which are 512 and 768 respectively. This represents the remaining part of S after being masked, P emb represent Location information, f bn f represents the normalization operation. transformer This represents the transformer encoder in wav2vec2.

[0067] It should be noted that for the feature encoder, random masking is only performed during training; in practical applications, this function needs to be disabled. Since the masked portions of the same speech are random, a speech can be divided into several segments with different textual information, serving as data augmentation. Furthermore, due to the high masking ratio, the left and right segments of each masked portion vector are highly likely to be masked. The model can learn the speaker's voiceprint features through discontinuous speech features, making the trained feature encoder highly robust. Finally, the decoder reconstructs the masked portion vectors, and through iterative training updates, this model can recognize all speech features using only a subset of them. This places higher demands on the encoder's extraction capabilities, allowing for a deeper understanding of speech features and resulting in a powerful feature extraction model.

[0068] For random masking, random sampling masking is performed according to a uniform distribution. Using a high masking rate (i.e., greater than 50%) avoids the task of easily predicting by inferring from unmasked neighboring vectors. A uniform distribution prevents potential center shifts (i.e., higher masking probability near the center of a speech segment). Furthermore, the highly sparse input creates conditions for training efficient feature encoders; very powerful encoders can be trained with minimal computation and memory, improving training efficiency by 2-3 times.

[0069] Step S3: Perform vector padding and positional information embedding on the lossy speech feature vector to obtain the masked feature vector; including:

[0070] The masked vector is assigned a shared learning vector or a fixed non-zero vector; the masked vector and the remaining vector are embedded with the corresponding positional information to obtain the masked feature vector.

[0071] Specifically, for the vector H output by the feature encoder, a unified, learnable vector I is added to the masked part based on the recorded position information, and position vector encoding is added to both the masked and unmasked parts to obtain the feature representation Z.

[0072] Step S4: Based on the feature vector after mask filling, predict the speech feature vector corresponding to the randomly masked part to obtain the complete speech feature vector;

[0073] Specifically, the vector Z obtained after masking is input into the transformer decoder for decoding to obtain the final vector representation Z. out :

[0074]

[0075]

[0076] in, This represents the complete speech feature vector after adding vector I to the vector H output by the encoder. Represents location encoding information, f bn f represents the normalization operation. transformer This represents the transformer decoder.

[0077] Preferably, in this embodiment, when using the wav2vec2 model as the initial model for training, the encoder and decoder of the wav2vec2 model can be designed with an asymmetric structure. Specifically, in this embodiment, the main function of the decoder is to restore the filled feature vector, making it as close as possible to the information before it was masked. Its structure is the same as the transformer encoder. The input is a vector composed of the unmasked feature vector, the padding vector, and the positional encoding; where the positional encoding can represent the positional information of the feature vector in the complete speech. Since the transformer decoder is only used during training to perform the speech reconstruction task, and the final recognition task is completed by the wav2vec encoder, the decoder architecture can be designed independently of the encoder design. Experiments can be conducted with a very small decoder, which is narrower and shallower than the encoder. For example, the encoder uses a 12-layer 768-dimensional transformer structure, and the decoder can use 2 or 3 layers, with a size of 384 dimensions. Compared with the encoder, the computational cost per token in the decoder is less than 10%. Using this asymmetric design, all tokens are processed through a lightweight decoder, which greatly reduces the training time.

[0078] Step S5: Perform average pooling and softmax classification on the complete speech feature vector to obtain the label category distribution corresponding to the complete speech feature vector;

[0079] The output layer of this invention uses an average pooling operation followed by a softmax layer to predict the speaker label category distribution. The output vector dimension is the number of label categories in the training dataset, and the predicted vector is obtained after passing through the softmax layer.

[0080]

[0081] It should be noted that in the original Wav2vec2 model pre-training process, the model loss consists of both adversarial loss and diversity loss. However, in this embodiment, due to the high masking rate operation, predicting the masked information is too difficult for the model, preventing convergence. Furthermore, the ultimate goal of this invention is to obtain the speaker's speech feature vector representation, a text-independent task. This embodiment, based on the decoded complete speech feature vector, uses average pooling and a fully connected layer network to convert the complete speech feature vector into the label dimension corresponding to the task. Probability normalization is then performed using an activation function, and finally, the category information of the speech data is obtained by selecting the category corresponding to the highest confidence value.

[0082] Step S6: Based on the labels in the voiceprint recognition training dataset and the labels obtained by the classifier, the trained feature encoder is iteratively updated through the loss function to obtain the trained feature encoder; based on the trained feature encoder, the random masking function is turned off to obtain the feature extraction model.

[0083] Specifically, this embodiment uses the following loss function for iterative loss update to obtain the trained feature encoder:

[0084]

[0085] Where x is the input speech sequence and y is the speaker category label.

[0086] In practical applications, the random masking function of the trained feature encoder is turned off, thus obtaining the feature extraction model. The speech data to be recognized is input into the feature extraction model, and after CNN feature extraction and transformer encoding, a feature vector with the speaker's voiceprint features is obtained. Based on the obtained feature vector, the speaker's identity is identified and confirmed.

[0087] A second embodiment of the present invention also provides a voiceprint recognition system, the system comprising: a user voice registration database, a feature extraction model, and a voiceprint verification model;

[0088] The user voice registration library stores the voice feature vectors corresponding to a user's voice and the corresponding user tags; specifically, the user voice registration library is constructed from user voice data, including:

[0089] The user's voice input is used to obtain a pre-trained feature extraction model, which generates the user's voice feature vector. The user's voice feature vector is labeled based on the user ID. The feature vector and corresponding label of the voice to be registered are saved to obtain the user voice registration library.

[0090] The feature extraction model is trained using the training method described in the previous embodiment, and is used to extract features from the speech to be recognized and encode the corresponding speech feature vector.

[0091] The voiceprint verification model is used to calculate the similarity value between the voice feature vector of the speech to be identified and the voice feature vector in the voice registration database. The user to which the registered voice feature vector with the highest similarity value and greater than the threshold belongs is selected as the user to which the speech to be identified belongs.

[0092] In this embodiment, for unregistered speaker voice, after feature extraction by the feature extraction model, if the similarity score between the unregistered voice and the voice feature vector in the voice registration database is less than the threshold, then a message is displayed indicating that there is no matching user.

[0093] In summary, this invention employs random masking of input speech features and reconstructs the randomly masked portions, aiming to provide a training method and system for a content-independent voiceprint recognition feature extraction model. The method first uses a wav2vec2 encoder structure to extract features from the input speech, encoding them into a fixed-dimensional vector. The extracted speech feature vector is then subjected to high-proportion random masking and encoding to obtain a lossy speech feature vector. Masking is then performed, and the masked speech features are decoded using a transformer model decoder to restore the masked features. Finally, a classification layer outputs the probability distribution of the input speech's category. Through iterative updates, a powerful voiceprint recognition feature extraction model is obtained. This invention improves the robustness of the feature extraction model through high-masking-rate random masking during training, making it applicable to various voiceprint recognition scenarios and achieving accurate recognition of lossy speech.

[0094] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0095] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A training method for a voiceprint recognition feature extraction model, characterized in that, Includes the following steps: Construct a voiceprint recognition training dataset, which includes voice data and labels representing the person to whom the voice data belongs; The speech data is subjected to feature extraction, random masking, and feature encoding using a feature encoder to obtain a lossy speech feature vector. The lossy speech feature vector is padded with vectors and embedded with positional information to obtain the mask-padded feature vector. Based on the feature vector after mask filling, the speech feature vector corresponding to the randomly masked part is predicted to obtain the complete speech feature vector; The complete speech feature vector is subjected to average pooling and softmax classification to obtain the label category distribution corresponding to the complete speech feature vector; The labels obtained from the voiceprint recognition training dataset and the classifier are iteratively updated using a loss function to obtain a trained feature encoder; based on the trained feature encoder, the feature extraction model is obtained by disabling the random masking function.

2. The training method for the voiceprint recognition feature extraction model according to claim 1, characterized in that, The feature encoder includes: a feature extraction unit, a random masking unit, and a transformer unit; The feature extraction unit is used to extract features from the input speech data to obtain a fixed-dimensional speech feature vector; The random masking unit randomly masks the speech feature vectors output by the feature extraction unit according to a uniform distribution; and records the positions of the masked vectors and the remaining vectors, embeds position information into the remaining vectors, and obtains the lossy speech data after random masking. The transformer unit is used to perform inter-frame attention calculation on the randomly masked lossy speech data to obtain a lossy speech feature vector that has temporal and contextual relationships.

3. The training method for the voiceprint recognition feature extraction model according to claim 2, characterized in that, The random shielding has a shielding rate of more than 50%.

4. The training method for the voiceprint recognition feature extraction model according to claim 2, characterized in that, The step of performing vector padding and positional information embedding on the lossy speech feature vector includes: assigning a shared learning vector or a fixed non-zero vector to the masked vector; and embedding corresponding positional information into the masked vector and the remaining vectors to obtain the masked feature vector.

5. The training method for the voiceprint recognition feature extraction model according to claim 1, characterized in that, The mask-padded feature vector is input into the decoder, and the complete speech feature vector is obtained using the following formula: ； in, This represents the speech feature vector after adding the padding vector. Represents location coding information, Represents normalization operation, This represents the transformer decoder.

6. The training method for the voiceprint recognition feature extraction model according to claim 1, characterized in that, The prediction vector obtained by performing average pooling and softmax classification on the complete speech feature vector , ； The loss function formula is: ； Where x is the input speech sequence and y is the speaker category label.

7. The training method for the voiceprint recognition feature extraction model according to claim 1, characterized in that, The feature extraction model is trained using the Wav2Vec2 model as the initial model. The encoder and decoder of the Wav2Vec2 model are designed as asymmetric structures during training.

8. The training method for the voiceprint recognition feature extraction model according to claim 1, characterized in that, Constructing the voiceprint recognition training dataset includes: Obtain the dataset; the dataset includes speech data from multiple speakers; Label the speech data according to the speaker ID; The speech data is preprocessed by channel splitting, speech segmentation and unified formatting to obtain mono, fixed-length speech data, thus obtaining a voiceprint recognition training dataset.

9. A voiceprint recognition system, characterized in that, include: User voice registration database, feature extraction model, and voiceprint verification model; The user voice registration library is used to store the voice feature vectors corresponding to the user's voice and the corresponding user tags; The feature extraction model is trained by the training method described in any one of claims 1-7, and is used to extract features from the speech to be recognized and encode the corresponding speech feature vector. The voiceprint verification model is used to calculate the similarity value between the voice feature vector of the speech to be identified and the voice feature vector in the voice registration database, and selects the user to which the registered voice feature vector with the highest similarity value and greater than the threshold belongs as the user to which the speech to be identified belongs.

10. The voiceprint recognition system according to claim 9, characterized in that, The user voice registration library is constructed from user voice data, including: using a pre-trained feature extraction model of the voice input of the user to be registered to obtain the voice feature vector of the user to be registered; labeling the voice feature vector of the user to be registered based on the user ID; and saving the voice feature vector and the corresponding label of the user to be registered to obtain the user voice registration library.

Images