Voiceprint recognition method and device, computer device, storage medium and program product

Abstract

The application relates to a voiceprint recognition method and device, computer equipment, a storage medium and a program product. The method comprises the following steps: obtaining to-be-recognized voice data, inputting the to-be-recognized voice data into a preset voiceprint recognition model, and obtaining a voiceprint recognition result of the to-be-recognized voice data. The method can apply a pre-trained voiceprint recognition model to recognize to-be-recognized voice data and obtain a voiceprint recognition result. Since the to-be-trained voiceprint recognition model references the convolutional layer parameters of a generative adversarial network model trained by data augmentation on a small sample voice data training set during training, the to-be-trained voiceprint recognition model references the knowledge obtained by training a large sample voice data set during training, which can further accelerate the convergence rate of the voiceprint recognition model training and improve the accuracy of the voiceprint recognition model recognition.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a voiceprint recognition method, apparatus, computer equipment, storage medium, and program product. Background Technology

[0002] Voiceprint recognition can identify a speaker by extracting the voice features when a person speaks. Due to its advantages such as convenient data collection, high user acceptance, low cost, and non-contact operation, it is suitable for remote identification scenarios and is widely used in banking, securities and other fields.

[0003] Deep learning algorithms, due to their powerful non-linear representation and automatic learning capabilities, have become the mainstream method for extracting deep speech features. Based on this, related technologies employ deep learning-based speaker recognition models to extract deep speech features from speech data to achieve speaker recognition. However, in real-world scenarios, speech data is relatively scarce, making it difficult to obtain enough speech data to train the model. Furthermore, speaker recognition models trained using small-sample speech datasets suffer from low speaker recognition accuracy. Summary of the Invention

[0004] Therefore, it is necessary to provide a voiceprint recognition method, device, computer equipment, storage medium, and program product to address the aforementioned technical problems.

[0005] In a first aspect, a voiceprint recognition method, the method comprising:

[0006] Acquire the speech data to be recognized;

[0007] Input the speech data to be recognized into the preset voiceprint recognition model to obtain the voiceprint recognition result of the speech data to be recognized;

[0008] The initial parameters of the convolutional layers during the training of the voiceprint recognition model are determined based on the parameters of the convolutional layers of the pre-trained generative adversarial network model, which is trained by augmenting a small sample speech data training set.

[0009] In one embodiment, the process of constructing the voiceprint recognition model includes:

[0010] The initial generative adversarial network model is trained using a small sample speech data training set to obtain the generative adversarial network model.

[0011] The network parameters of the generative adversarial network model are transferred to the initial voiceprint recognition model. The initial voiceprint recognition model is trained using a small sample speech data training set to obtain the voiceprint recognition model. The convolutional layer structures of the initial voiceprint recognition model and the generative adversarial network model are the same.

[0012] In one embodiment, the initial generative adversarial network model includes an initial generator network and an initial discriminator network;

[0013] The initial generative adversarial network (GAN) model is then trained using a small sample speech data training set to obtain the GAN model, which includes:

[0014] Preprocess the small sample speech data training set to obtain the preprocessed results;

[0015] Random noise data is input into the initial generator network to obtain generated data. The preprocessing results and generated data are then input into the initial discriminator network. The initial generator network and the initial discriminator network are jointly trained to obtain the generator network and the discriminator network.

[0016] In one embodiment, the preprocessing result and generated data are input into the initial discriminator network, and the initial generator network and the initial discriminator network are jointly trained to obtain the generator network and the discriminator network, including:

[0017] The preprocessing results and generated data are input into the initial discriminator network to obtain the initial discrimination prediction results;

[0018] The prediction error between the initial discrimination prediction result and the standard discrimination result is calculated using the loss function.

[0019] Update the network parameters in the initial generator network and the initial discriminator network based on the prediction error value;

[0020] If the prediction error value meets the preset convergence condition, it is determined that the initial generator network and the initial discriminator network have been trained successfully, and the generator network and discriminator network are obtained.

[0021] In one embodiment, the small sample speech data training set is preprocessed to obtain the preprocessing result, including:

[0022] The few-sample speech data in the training set is divided into frames to obtain multiple speech frame data.

[0023] Windowing is applied to each speech frame data to obtain the corresponding windowed data.

[0024] Perform Fourier transform on each windowed data to determine the two-dimensional spectrogram;

[0025] The frequency data in the two-dimensional spectrogram is mapped onto the Mel scale to obtain Mel spectrogram data, which is then used as the preprocessing result.

[0026] In one embodiment, the discriminator network in the generative adversarial network model includes a first convolutional layer and a first fully connected layer; the initial voiceprint recognition model includes a second convolutional layer and a second fully connected layer.

[0027] The network parameters of the generative adversarial network model are then transferred to the initial speaker recognition model, which is trained using a small sample speech data training set, including:

[0028] The network parameters of the first convolutional layer in the discriminator network are determined as the network parameters of the second convolutional layer in the initial voiceprint recognition model, and the network parameters of the second fully connected layer are initialized.

[0029] The initial voiceprint recognition model was trained using a small sample speech data training set.

[0030] Secondly, a voiceprint recognition device, the device comprising:

[0031] The voice data acquisition module is used to acquire the voice data to be recognized;

[0032] The voiceprint recognition module is used to input the voice data to be recognized into a preset voiceprint recognition model to obtain the voiceprint recognition result of the voice data to be recognized.

[0033] The initial parameters of the convolutional layers during the training of the voiceprint recognition model are determined based on the parameters of the convolutional layers of the pre-trained generative adversarial network model, which is trained by augmenting a small sample speech data training set.

[0034] Thirdly, a computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of any of the methods described in the first aspect embodiments above.

[0035] Fourthly, a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described in the first aspect embodiments.

[0036] Fifthly, a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described in the first aspect embodiments.

[0037] The aforementioned voiceprint recognition method, apparatus, computer equipment, storage medium, and program products allow the computer equipment to acquire speech data to be recognized and input it into a preset voiceprint recognition model to obtain the voiceprint recognition result of the speech data to be recognized. This method can apply a pre-trained voiceprint recognition model to recognize the speech data to be recognized and obtain the voiceprint recognition result. Furthermore, because the voiceprint recognition model to be trained uses the convolutional layer parameters of a generative adversarial network model trained by augmenting a small sample speech data training set, the training of the voiceprint recognition model incorporates knowledge obtained from training on a large sample speech dataset, further accelerating the convergence rate of the voiceprint recognition model training and improving the accuracy of the voiceprint recognition model. Attached Figure Description

[0038] Figure 1 This is a diagram illustrating the application environment of a voiceprint recognition method in one embodiment.

[0039] Figure 2 This is a flowchart illustrating a voiceprint recognition method in one embodiment;

[0040] Figure 3 This is a schematic diagram of a method for constructing a voiceprint recognition model in one embodiment;

[0041] Figure 4 This is a schematic diagram of the method for training an initial generative adversarial network model using a small sample speech data training set to obtain a generative adversarial network model in another embodiment.

[0042] Figure 5 This is a schematic diagram of a method for jointly training an initial generator network and an initial discriminator network in another embodiment.

[0043] Figure 6 This is a flowchart illustrating a method for preprocessing a small sample speech data training set to obtain preprocessing results in another embodiment.

[0044] Figure 7 This is a schematic diagram of a method in another embodiment for transferring network parameters of a generative adversarial network model to an initial voiceprint recognition model and training the initial voiceprint recognition model.

[0045] Figure 8 This is a structural block diagram of a voiceprint recognition device in one embodiment;

[0046] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0048] The voiceprint recognition method provided in this application can be applied to Figure 1 The illustrated voiceprint recognition system includes a voice acquisition device and a computer device. Optionally, the voice acquisition device can be a voice recorder, a voice acquisition device, a voice detector, etc.; the computer device can be a standalone server or a server cluster consisting of multiple servers, and can also be, but is not limited to, various personal computers, laptops, smartphones, tablets, and portable wearable devices. The voice acquisition device and the computer device can communicate with each other; the communication method can be Wi-Fi, mobile network, or Bluetooth connection, etc. The following embodiments will describe the implementation process of the voiceprint recognition method.

[0049] To extract the speech features of a speaker to determine their identity, embodiments of this application provide a voiceprint recognition method, which is applied to... Figure 1 Taking computer equipment as an example, such as Figure 2 As shown, the voiceprint recognition method includes the following steps:

[0050] S100: Acquire the speech data to be recognized.

[0051] Specifically, the voice acquisition device can periodically collect voice data from at least one speaker. Furthermore, the computer device can receive the voice data sent by the voice acquisition device in real time, i.e., the voice data to be recognized. Alternatively, the voice acquisition device can store the collected voice data locally or in the cloud. In practical applications, the computer device can retrieve the pre-stored voice data, i.e., the voice data to be recognized, from the local storage or the cloud. Additionally, the computer device can download simulated voice data, i.e., the voice data to be recognized, from the internet.

[0052] It should be noted that the speech data to be recognized can be speech data containing only the speaker. However, another way to obtain the speech data to be recognized is for a computer device to acquire the speech dataset to be recognized, and then perform feature extraction on the speech dataset to obtain speech data containing only the speaker, i.e., the speech data to be recognized. Optionally, the speech dataset to be recognized can include speech data of the speaker and speech data of non-speakers. In this embodiment, the speech data of non-speakers can be environmental speech data, such as speech data of electronic devices, speech data of nature (such as wind, thunder, flowing water), etc.

[0053] S200. Input the speech data to be recognized into the preset voiceprint recognition model to obtain the voiceprint recognition result of the speech data to be recognized. The initial parameters of the convolutional layer during the training of the voiceprint recognition model are determined according to the convolutional layer parameters of the pre-trained generative adversarial network model, and the generative adversarial network model is trained by augmenting the small sample speech data training set.

[0054] Specifically, the aforementioned preset voiceprint recognition model can be a pre-trained voiceprint recognition model. Optionally, the voiceprint recognition model can be a deep learning network model, such as at least one of convolutional neural network models, recurrent neural network models, and deep belief network models.

[0055] It should be noted that the voiceprint recognition model may include at least one of convolutional layers, fully connected layers, and pooling layers, and the number of convolutional layers, fully connected layers, and pooling layers included in the voiceprint recognition model can be arbitrary and is not limited thereto. However, in this embodiment, the voiceprint recognition model includes at least one convolutional layer. Optionally, before training the voiceprint recognition model, the initial parameters (network parameters) of the convolutional layers during training can be the convolutional layer parameters of the convolutional layers in a pre-trained generative adversarial network model.

[0056] Optionally, the aforementioned data augmentation can be understood as a process of cropping, translating, changing brightness, adding noise, rotating angle, and / or mirroring the speech data in a small sample speech data training set. Optionally, the aforementioned voiceprint recognition result can be the identity of the speaker; the identity of the speaker can be speaker identification or speaker confirmation. Optionally, speaker identification can be understood as performing a one-to-N comparison of the speaker's speech data with the speech data of N speakers in a preset database, finding the speech data with the same comparison result, and identifying the speaker corresponding to the speech data as the speaker; speaker confirmation can be understood as performing a one-to-one comparison of the speaker's speech data with the speech data of the speaker in a preset database to prove that the speaker is the person in question.

[0057] The aforementioned voiceprint recognition method can acquire the speech data to be recognized and input it into a preset voiceprint recognition model to obtain the voiceprint recognition result of the speech data to be recognized. This method can apply a pre-trained voiceprint recognition model to recognize the speech data to be recognized and obtain the voiceprint recognition result. Furthermore, since the voiceprint recognition model to be trained uses the convolutional layer parameters of the generative adversarial network model obtained by data augmentation on a small sample speech data training set, the voiceprint recognition model to be trained uses the knowledge obtained from training on a large sample speech dataset, which can further accelerate the convergence rate of the voiceprint recognition model training and improve the recognition accuracy of the voiceprint recognition model.

[0058] Because training a voiceprint recognition model using a small sample speech data training set often leads to problems such as overfitting, poor generalization ability, and low accuracy in voiceprint recognition, in one embodiment, as follows: Figure 3 As shown, the construction process of the above voiceprint recognition model can be achieved through the following steps:

[0059] S210. The initial generative adversarial network model is trained using a small sample speech data training set to obtain the generative adversarial network model.

[0060] Specifically, the small-sample speech data training set can be a collection of speech data from multiple speakers. Optionally, the generative adversarial network model can include a generative model and a discriminative model. Optionally, both the generative and discriminative models can include at least one of convolutional layers, fully connected layers, and pooling layers. The overall structure of the generative and discriminative models can be the same or different, but their network parameters are different. In this embodiment, the structure of the same layer in different models can be the same or different.

[0061] S220. Transfer the network parameters of the generative adversarial network model to the initial voiceprint recognition model, and train the initial voiceprint recognition model using a small sample speech data training set to obtain the voiceprint recognition model; wherein, the structure of the convolutional layers of the initial voiceprint recognition model and the generative adversarial network model is the same.

[0062] Specifically, transferring network parameters from a generative adversarial network (GAN) model to an initial speaker recognition model can be understood as using some or all of the network parameters of the generator and / or discriminator models in the trained GAN model as parameters from the initial speaker recognition model. Furthermore, the computer can train the initial speaker recognition model using a small sample speech data training set.

[0063] The training set of few-sample speech data used to train the initial voiceprint recognition model can be different from the training set used to train the generative adversarial network model. However, in this embodiment, the training set of few-sample speech data used to train the initial voiceprint recognition model is the same as the training set used to train the generative adversarial network model.

[0064] In this embodiment, both the initial voiceprint recognition model and the generative adversarial network (GAN) model can include convolutional layers. Furthermore, to achieve parameter transfer within the convolutional layers, the structures of the convolutional layers in both the initial voiceprint recognition model and the GAN model can be identical. Optionally, the network structures in the initial voiceprint recognition model and the GAN model, except for the structures preceding the convolutional layers, can be the same or different.

[0065] The aforementioned voiceprint recognition method can transfer the convolutional layer parameters of the generative adversarial network model trained by augmenting a small sample speech data training set to the initial voiceprint recognition model. Then, the initial voiceprint recognition model is trained again using the small sample speech data training set. This allows the initial voiceprint recognition model to utilize knowledge gained from training on a large sample speech dataset, further accelerating the convergence rate of the voiceprint recognition model training and improving its accuracy. Furthermore, by augmenting the small sample speech dataset using a generative adversarial network model, the method reduces the discrepancy between the augmented dataset and the small sample speech dataset. Then, through transfer learning, the convolutional layer parameters trained on the augmented samples are transferred to the voiceprint recognition model for small sample training, thereby improving the generalization ability of the small sample-trained voiceprint recognition model.

[0066] The following describes how to train the initial generative adversarial network (GAN) model using a small-sample speech data training set, enabling the GAN model to learn knowledge from a large-sample speech dataset. In one embodiment, the initial GAN ​​model includes an initial generator network and an initial discriminator network; as shown... Figure 4 As shown, the step in S210 above, which trains the initial generative adversarial network model using a small sample speech data training set to obtain the generative adversarial network model, can be achieved through the following steps:

[0067] S211. Preprocess the small sample speech data training set to obtain the preprocessing results.

[0068] Specifically, computer equipment can preprocess speech data in a small sample speech data training set, such as noise removal (ambient noise, busy tone, ringtone, etc.), data augmentation (aliasing, rate change, random masking in time and frequency domains), cropping, data transformation and / or feature extraction, to obtain preprocessed results.

[0069] S212. Input random noise data into the initial generator network to obtain generated data, and input the preprocessing results and generated data into the initial discriminator network. Jointly train the initial generator network and the initial discriminator network to obtain the generator network and the discriminator network.

[0070] Specifically, the aforementioned random noise can be Gaussian noise, single-frequency noise, impulse noise, fluctuation noise, and / or white noise, etc. During the initial training of the generative adversarial network (GAN) model, the computer device can first input the generated random noise data into the initial generator network to obtain the corresponding simulated data (i.e., generated data). Then, the preprocessed results and the generated data are input into the initial discriminator network to jointly train the initial generator network and the initial discriminator network. When it is determined that both the initial generator network and the initial discriminator network satisfy the corresponding convergence conditions, the initial generator network is then designated as the generator network, and the initial discriminator network is designated as the discriminator network. Optionally, joint training can be understood as the process of simultaneously training the initial generator network and the initial discriminator network.

[0071] It should be noted that the aforementioned initial generator network may include a fully connected layer and multiple deconvolutional layers. A computer device can input random noise data into the initial generator network, convert it into 3D data through a fully connected layer, and then upsample the 3D data through multiple deconvolutional layers to obtain the generated data. Optionally, the amount of feature data output by each deconvolutional layer can be twice that of the previous adjacent deconvolutional layer.

[0072] Understandably, the initial discriminator network can include multiple two-dimensional convolutional layers and one fully connected layer. The computer device can input preprocessed results and generated data into the initial discriminator network, which then performs downsampling through multiple two-dimensional convolutional layers to learn deep speech features from the input data. Finally, the fully connected layer outputs the discrimination result of the initial discriminator network.

[0073] Among them, such as Figure 5 As shown, the step in S212 above, where the preprocessing results and generated data are input into the initial discriminator network, and the initial generator network and initial discriminator network are jointly trained to obtain the generator network and discriminator network, can be achieved through the following steps:

[0074] S2121. Input the preprocessing results and generated data into the initial discriminator network to obtain the initial discrimination prediction results.

[0075] Specifically, the computer device can input the preprocessing results or generated data into the initial discriminator network to obtain the initial discrimination prediction result. However, in this embodiment, the computer device can simultaneously input the preprocessing results and the generated data output by the initial generator network into the initial discriminator network to obtain the initial discrimination prediction result.

[0076] It should be noted that before the initial generator network and the initial discriminator network have finished training, the initial discriminator network can correctly determine the authenticity of the preprocessed results and the generated data. In this case, the initial discriminant prediction result can be the preprocessed results and the generated data carrying labels. Optionally, the labels can distinguish between the authenticity of the preprocessed results and the generated data. Specifically, the preprocessed results can be identified as real data; the generated data, being simulated data, can be identified as pseudo data.

[0077] Optionally, at the end of training for the initial generator network and the initial discriminator network, the initial discriminator network may incorrectly determine the authenticity of the preprocessed results and the generated data; that is, in this case, it may classify the preprocessed results as fake data and the generated data as real data. Furthermore, in this case, the initial discrimination prediction result can also be a combination of preprocessed results and generated data carrying labels. However, in this case, the labels carried by the preprocessed results and generated data are the opposite of those carried by the preprocessed results and generated data output by the initial discriminator network before training of the initial generator network and the initial discriminator network has ended.

[0078] S2122. Calculate the prediction error between the initial discrimination prediction result and the standard discrimination result using the loss function.

[0079] Specifically, the loss function mentioned above can be a 0-1 loss function, a squared loss function, an absolute value loss function, a logarithmic loss function, etc. Optionally, the loss function includes parameters corresponding to the initial discrimination prediction result and the standard discrimination result.

[0080] It should be noted that the computer device can input the initial discrimination prediction result into the loss function to obtain the prediction error value between the initial discrimination prediction result and the standard discrimination result. Optionally, the standard discrimination result can be the preprocessed result carrying labels and the generated data, where the labels carried in the preprocessed result are the labels of the pseudo data, and the labels carried in the generated data are the labels of the real data. Optionally, the standard discrimination result can be understood as the gold standard for training generative adversarial network models.

[0081] S2123. Update the network parameters in the initial generator network and the initial discriminator network according to the prediction error value.

[0082] Specifically, the computer equipment can adjust the network parameters in the initial generator network and the initial discriminator network based on the magnitude of the prediction error. Optionally, if the prediction error is large, the adjusted values ​​of the network parameters can be slightly larger, and if the prediction error is small, the adjusted values ​​of the network parameters can be slightly smaller.

[0083] S2124. If the prediction error value meets the preset convergence condition, it is determined that the initial generator network and the initial discriminator network have been trained and the generator network and discriminator network are obtained.

[0084] Understandably, during the training of the initial generator network and the initial discriminator network, it is necessary to continuously iterate and execute the steps in S211, S2121-S2123 above. After each iteration, it can be determined whether the prediction error value is less than or equal to the preset error threshold, or whether the number of iterations has reached the preset iteration threshold. If it is determined that the prediction error value is less than or equal to the preset error threshold, or the number of iterations has reached the preset iteration threshold, it is determined that the initial generator network and the initial discriminator network have been trained. The current initial generator network is then determined as the generator network, and the current initial discriminator network is determined as the discriminator network.

[0085] The aforementioned voiceprint recognition method can train a generative adversarial network (GAN) model by augmenting the training set of small-sample speech data. This allows the GAN model to learn a large amount of knowledge from large-sample speech data, so that the network parameters of the GAN model can be transferred to the voiceprint recognition model for small-sample training, thereby improving the generalization ability of the small-sample trained voiceprint recognition model.

[0086] When training a generative adversarial network (GAN) model using speech data, it is necessary to extract the features of the speech data. Therefore, before training, the speech data can be preprocessed to obtain Mel spectrogram data (i.e., speech feature data). In one embodiment, for example... Figure 6 As shown, the step in S211 above, which preprocesses the small sample speech data training set to obtain the preprocessing result, may specifically include:

[0087] S2111. The small sample speech data in the small sample speech data training set is divided into frames to obtain multiple speech frame data.

[0088] Specifically, since speech data is non-stationary, but stationary over a short period of time and does not undergo abrupt changes, for ease of processing, the small sample speech data in the training set can be divided into multiple frames of stationary speech data. Optionally, the speech data can be one-dimensional data.

[0089] Optionally, the computer device can divide the small sample speech data in the small sample speech data training set into frames according to time sequence to obtain multiple speech frame data.

[0090] S2112. Window each speech frame data separately to obtain the corresponding windowed data.

[0091] Specifically, to prevent spectral energy leakage during subsequent Fourier transforms of each speech frame, windowing can be applied to each speech frame. Optionally, windowing can be understood as the process of truncating the speech frame data using a window function. Optionally, the window type can be a rectangular window, triangular window, Hanning window, Gaussian window, etc.

[0092] It should be noted that because the Hamming window has non-zero values ​​at both ends, it can reduce leakage from very close side lobes. Therefore, in this embodiment, the computer device can apply a Hamming window to each speech frame data separately to obtain the windowed data corresponding to each speech frame data.

[0093] S2113. Perform Fourier transform on each windowed data to determine the two-dimensional spectrum.

[0094] Furthermore, the computer device can perform Fourier transforms on each windowed data separately, and combine the Fourier transform results in chronological order to obtain a two-dimensional spectrogram. Optionally, the two-dimensional spectrogram can include X-axis data and Y-axis data; the X-axis data can be time, and the Y-axis data can be frequency.

[0095] S2114. Map the frequency data in the two-dimensional spectrogram to the Mel scale to obtain Mel spectrogram data, and determine the Mel spectrogram data as the preprocessing result.

[0096] Specifically, the computer device can map the frequency data f on the Y-axis of the two-dimensional spectrogram to the Mel scale mel according to the mapping relationship between the frequency data f and the Mel scale mel, thus obtaining Mel spectrogram data. Optionally, the mapping relationship can be a proportional relationship, a functional relationship, a logarithmic relationship, and / or an exponential relationship, etc.

[0097] In this embodiment, the mapping relationship between frequency data f and Mel scale mel can be expressed by the formula:

[0098] mel = 2595 * log 10 (1+f / 700) (1);

[0099] The numerical values ​​in formula (1) can also be other values, and there are no restrictions on them.

[0100] The aforementioned voiceprint recognition method can preprocess small sample speech data in a small sample speech data training set to obtain Mel spectrogram data. Then, a generative adversarial network model can be trained using the Mel spectrogram data, thereby enabling the generative adversarial network model to obtain the optimal network parameters corresponding to large samples. Based on this, a voiceprint recognition model with greater generalization ability can be trained through transfer learning.

[0101] As one embodiment, the discriminator network in the generative adversarial network model includes a first convolutional layer and a first fully connected layer; the initial voiceprint recognition model includes a second convolutional layer and a second fully connected layer; as shown in the example. Figure 7 As shown, the step in S220 above, which involves transferring the network parameters of the generative adversarial network model to the initial speaker recognition model and training the initial speaker recognition model using a small sample speech data training set, can be achieved through the following steps:

[0102] S221. Determine the network parameters of the first convolutional layer in the discriminator network as the network parameters of the second convolutional layer in the initial voiceprint recognition model, and initialize the network parameters of the second fully connected layer.

[0103] In this embodiment, the discriminator network in the generative adversarial network model includes a convolutional layer (i.e., the first convolutional layer) and a fully connected layer (i.e., the first fully connected layer); the initial voiceprint recognition model also includes a convolutional layer (i.e., the second convolutional layer) and a fully connected layer (i.e., the second fully connected layer). The structures of the first and second convolutional layers can be identical; the structures of the first and second fully connected layers can be identical or different.

[0104] It should be noted that the computer device can assign the network parameters of the first convolutional layer in the generative adversarial network model to the network parameters of the second convolutional layer in the initial voiceprint recognition model. Simultaneously, the computer device can initialize the network parameters of the second fully connected layer in the initial voiceprint recognition model. If the category output by the voiceprint recognition model is the same as the category output by the discriminator network, then the network parameters of the first fully connected layer in the discriminator network can be determined as the network parameters of the second fully connected layer in the initial voiceprint recognition model. In this case, the structures of the first and second fully connected layers are identical. If the category output by the voiceprint recognition model is different from the category output by the discriminator network, then the network parameters of the second fully connected layer in the initial voiceprint recognition model need to be initialized.

[0105] S222. Train the initial voiceprint recognition model using a small sample speech data training set.

[0106] Furthermore, the computer device can train the initial speaker recognition model using speech data from a small-sample speech data training set. Specifically, the computer device can input all speech data from the small-sample speech data training set into the initial speaker recognition model to obtain a speaker recognition prediction result. A loss function is used to calculate the prediction error between the predicted speaker recognition result and the standard speaker recognition result. The network parameters in the initial speaker recognition model are then updated based on the prediction error value. This process is repeated continuously, inputting all speech data from the small-sample speech data training set into the initial speaker recognition model with updated network parameters, until the prediction error value meets a preset error threshold or the number of iterations reaches a preset iteration threshold, thus obtaining a pre-trained speaker recognition model. The aforementioned standard speaker recognition result can be considered an idealized speaker recognition result, i.e., the gold standard for speaker recognition model training.

[0107] The aforementioned voiceprint recognition method can determine the network parameters of the first convolutional layer in the discriminator network as the network parameters of the second convolutional layer in the initial voiceprint recognition model, and initialize the network parameters of the second fully connected layer. The initial voiceprint recognition model is trained using a small sample speech data training set to obtain the voiceprint recognition result. This method can transfer the network parameters corresponding to large samples learned by the generative adversarial network model to the voiceprint recognition model to be trained, so that the voiceprint recognition model to be trained uses the knowledge obtained from training on the large sample speech dataset, which can further accelerate the convergence rate of the voiceprint recognition model training and improve the recognition accuracy of the voiceprint recognition model.

[0108] To facilitate understanding by those skilled in the art, the voiceprint recognition method provided in this application is described using a computer device as an example. Specifically, the method includes:

[0109] (1) The small sample speech data in the small sample speech data training set is divided into frames to obtain multiple speech frame data.

[0110] (2) Window each speech frame data separately to obtain the corresponding windowed data.

[0111] (3) Perform Fourier transform on each windowed data to determine the two-dimensional spectrum.

[0112] (4) Map the frequency data in the two-dimensional spectrogram to the Mel scale to obtain Mel spectrogram data, and determine the Mel spectrogram data as the preprocessing result.

[0113] (5) Input random noise data into the initial generator network to obtain generated data, and input the preprocessing results and generated data into the initial discriminator network to obtain the initial discrimination prediction results.

[0114] (6) Calculate the prediction error between the initial discrimination prediction result and the standard discrimination result using the loss function.

[0115] (7) Update the network parameters in the initial generator network and the initial discriminator network according to the prediction error value.

[0116] (8) If the prediction error value meets the preset convergence condition, it is determined that the initial generator network and the initial discriminator network have been trained and the generator network and discriminator network are obtained.

[0117] (9) The network parameters of the first convolutional layer in the discriminator network are determined as the network parameters of the second convolutional layer in the initial voiceprint recognition model, and the network parameters of the second fully connected layer are initialized.

[0118] (10) The initial voiceprint recognition model is trained using a small sample speech data training set to obtain the voiceprint recognition model; wherein the structure of the convolutional layer of the initial voiceprint recognition model and the generative adversarial network model is the same.

[0119] (11) Obtain the speech data to be recognized.

[0120] (12) Input the speech data to be recognized into the preset voiceprint recognition model to obtain the voiceprint recognition result of the speech data to be recognized.

[0121] The specific execution process of (1) to (12) above can be found in the description of the above embodiments. The implementation principle and technical effect are similar, and will not be repeated here.

[0122] It should be understood that, although Figure 2-7 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 2-7 At least some of the steps in the process may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0123] In one embodiment, such as Figure 8 As shown, a voiceprint recognition device is provided, including: a voice data acquisition module 11 and a voiceprint recognition module 12, wherein:

[0124] The voice data acquisition module 11 is used to acquire the voice data to be recognized;

[0125] The voiceprint recognition module 12 is used to input the voice data to be recognized into a preset voiceprint recognition model to obtain the voiceprint recognition result of the voice data to be recognized.

[0126] The initial parameters of the convolutional layers during the training of the voiceprint recognition model are determined based on the parameters of the convolutional layers of the pre-trained generative adversarial network model, which is trained by augmenting a small sample speech data training set.

[0127] The voiceprint recognition device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0128] In one embodiment, the voiceprint recognition device further includes: a first model training module and a second model training module, wherein:

[0129] The first model training module is used to train the initial generative adversarial network model using a small sample speech data training set to obtain the generative adversarial network model.

[0130] The second model training module is used to transfer the network parameters of the generative adversarial network model to the initial voiceprint recognition model. The initial voiceprint recognition model is trained using a small sample speech data training set to obtain the voiceprint recognition model. The initial voiceprint recognition model and the generative adversarial network model have the same structure of convolutional layers.

[0131] The voiceprint recognition device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0132] In one embodiment, the initial generative adversarial network model includes an initial generator network and an initial discriminator network; the first model training module includes a preprocessing unit and a joint training unit, wherein:

[0133] The preprocessing unit is used to preprocess the small sample speech data training set to obtain the preprocessing results;

[0134] The joint training unit is used to input random noise data into the initial generator network to obtain generated data, and input the preprocessing results and generated data into the initial discriminator network to jointly train the initial generator network and the initial discriminator network to obtain the generator network and the discriminator network.

[0135] The voiceprint recognition device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0136] In one embodiment, the joint training unit includes: a discriminator network processing subunit, a prediction error calculation subunit, a network parameter update subunit, and a training completion determination subunit, wherein:

[0137] The discriminator network processing subunit is used to input the preprocessing results and generated data into the initial discriminator network to obtain the initial discrimination prediction results;

[0138] The prediction error calculation subunit is used to calculate the prediction error between the initial discrimination prediction result and the standard discrimination result using a loss function.

[0139] The network parameter update subunit is used to update the network parameters in the initial generator network and the initial discriminator network based on the prediction error value.

[0140] The training ends and a sub-unit is determined. When the prediction error value meets the preset convergence condition, it is determined that the initial generator network and the initial discriminator network have been trained and the generator network and discriminator network are obtained.

[0141] The voiceprint recognition device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0142] In one embodiment, the preprocessing unit includes: a framing subunit, a windowing subunit, a Fourier transform subunit, and a data mapping subunit, wherein:

[0143] The framing subunit is used to segment the small sample speech data in the small sample speech data training set into frames to obtain multiple speech frame data.

[0144] The windowing subunit is used to perform windowing processing on each speech frame data to obtain the corresponding windowed data;

[0145] The Fourier transform subunit is used to perform Fourier transform on each windowed data to determine the two-dimensional spectrum.

[0146] The data mapping subunit is used to map the frequency data in the two-dimensional spectrogram to the Mel scale to obtain Mel spectrogram data, and to determine the Mel spectrogram data as the preprocessing result.

[0147] The voiceprint recognition device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0148] In one embodiment, the discriminator network in the generative adversarial network model includes a first convolutional layer and a first fully connected layer; the initial voiceprint recognition model includes a second convolutional layer and a second fully connected layer; the second model training module includes a network parameter initialization unit and a voiceprint recognition model training unit, wherein:

[0149] The network parameter initialization unit is used to determine the network parameters of the first convolutional layer in the discriminator network as the network parameters of the second convolutional layer in the initial voiceprint recognition model, and to initialize the network parameters of the second fully connected layer.

[0150] The voiceprint recognition model training unit is used to train the initial voiceprint recognition model using a small sample speech data training set.

[0151] The voiceprint recognition device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0152] The voiceprint recognition device provided in this embodiment can execute the above method embodiment, and its implementation principle and technical effect are similar, so it will not be described again here.

[0153] For specific limitations regarding the voiceprint recognition device, please refer to the limitations of the voiceprint recognition method above, which will not be repeated here. Each module in the aforementioned voiceprint recognition device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independently of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0154] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores voice data to be recognized. The network interface communicates with external endpoints via a network connection. When the computer program is executed by the processor, it implements a voiceprint recognition method.

[0155] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0156] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0157] Acquire the speech data to be recognized;

[0158] Input the speech data to be recognized into the preset voiceprint recognition model to obtain the voiceprint recognition result of the speech data to be recognized;

[0159] The initial parameters of the convolutional layers during the training of the voiceprint recognition model are determined based on the parameters of the convolutional layers of the pre-trained generative adversarial network model, which is trained by augmenting a small sample speech data training set.

[0160] In one embodiment, a readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0161] Acquire the speech data to be recognized;

[0162] Input the speech data to be recognized into the preset voiceprint recognition model to obtain the voiceprint recognition result of the speech data to be recognized;

[0163] The initial parameters of the convolutional layers during the training of the voiceprint recognition model are determined based on the parameters of the convolutional layers of the pre-trained generative adversarial network model, which is trained by augmenting a small sample speech data training set.

[0164] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0165] Acquire the speech data to be recognized;

[0166] Input the speech data to be recognized into the preset voiceprint recognition model to obtain the voiceprint recognition result of the speech data to be recognized;

[0167] The initial parameters of the convolutional layers during the training of the voiceprint recognition model are determined based on the parameters of the convolutional layers of the pre-trained generative adversarial network model, which is trained by augmenting a small sample speech data training set.

[0168] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0169] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0170] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A voiceprint recognition method, characterized in that, The method includes: Acquire the speech data to be recognized; The voice data to be recognized is input into a preset voiceprint recognition model to obtain the voiceprint recognition result of the voice data to be recognized. The initial parameters of the convolutional layers during the training of the voiceprint recognition model are determined based on the convolutional layer parameters of a pre-trained generative adversarial network (GAN) model. The GAN model includes a generator network and a discriminator network. The discriminator network includes a first convolutional layer and a first fully connected layer. The voiceprint recognition model includes a second convolutional layer and a second fully connected layer, and the second convolutional layer has the same structure as the first convolutional layer. Furthermore, the GAN model is trained by augmenting a small sample speech data training set.

2. The voiceprint recognition method according to claim 1, characterized in that, The construction process of the voiceprint recognition model includes: The initial generative adversarial network model is trained using the small sample speech data training set to obtain the generative adversarial network model. The network parameters of the generative adversarial network model are transferred to the initial voiceprint recognition model, and the initial voiceprint recognition model is trained using the small sample speech data training set to obtain the voiceprint recognition model; wherein the convolutional layer structure of the initial voiceprint recognition model and the generative adversarial network model is the same.

3. The voiceprint recognition method according to claim 2, characterized in that, The initial generative adversarial network model includes an initial generator network and an initial discriminator network; The step of training the initial generative adversarial network model using a small sample speech data training set to obtain the generative adversarial network model includes: The small sample speech data training set is preprocessed to obtain the preprocessing result; Random noise data is input into the initial generator network to obtain generated data. The preprocessing result and the generated data are then input into the initial discriminator network. The initial generator network and the initial discriminator network are jointly trained to obtain the generator network and the discriminator network.

4. The voiceprint recognition method according to claim 3, characterized in that, The step of inputting the preprocessing result and the generated data into the initial discriminator network, and jointly training the initial generator network and the initial discriminator network to obtain the generator network and the discriminator network includes: The preprocessing results and the generated data are input into the initial discriminator network to obtain the initial discrimination prediction results; The prediction error between the initial discrimination prediction result and the standard discrimination result is calculated using the loss function. Update the network parameters in the initial generator network and the initial discriminator network based on the prediction error value; If the prediction error value meets the preset convergence condition, then it is determined that both the initial generator network and the initial discriminator network have been trained successfully, and the generator network and the discriminator network are obtained.

5. The voiceprint recognition method according to claim 3, characterized in that, The preprocessing of the small sample speech data training set to obtain the preprocessing result includes: The few sample speech data in the training set of the few sample speech data is divided into frames to obtain multiple speech frame data; Each of the aforementioned speech frame data is windowed to obtain the corresponding windowed data; Perform Fourier transform on each windowed data to determine the two-dimensional spectrogram; The frequency data in the two-dimensional spectrogram is mapped onto the Mel scale to obtain Mel spectrogram data, and the Mel spectrogram data is determined as the preprocessing result.

6. The voiceprint recognition method according to any one of claims 2-5, characterized in that, The step of transferring the network parameters of the generative adversarial network model to the initial speaker recognition model, and training the initial speaker recognition model using the small sample speech data training set, includes: The network parameters of the first convolutional layer in the discriminator network are determined as the network parameters of the second convolutional layer in the initial voiceprint recognition model, and the network parameters of the second fully connected layer are initialized. The initial voiceprint recognition model is trained using the small sample speech data training set.

7. A voiceprint recognition device, characterized in that, The device includes: The voice data acquisition module is used to acquire the voice data to be recognized; The voiceprint recognition module is used to input the voice data to be recognized into a preset voiceprint recognition model to obtain the voiceprint recognition result of the voice data to be recognized. The initial parameters of the convolutional layers during the training of the voiceprint recognition model are determined based on the convolutional layer parameters of a pre-trained generative adversarial network (GAN) model. The GAN model includes a generator network and a discriminator network. The discriminator network includes a first convolutional layer and a first fully connected layer. The voiceprint recognition model includes a second convolutional layer and a second fully connected layer, and the second convolutional layer has the same structure as the first convolutional layer. Furthermore, the GAN model is trained by augmenting a small sample speech data training set.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the voiceprint recognition method according to any one of claims 1-6.

9. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the voiceprint recognition method according to any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1-6.

Images