Speaker verification model training methods, devices, media and equipment

Abstract

This invention discloses a method, apparatus, medium, and device for training a speaker verification model. The method includes: extracting waveforms from acquired speech audio data to obtain audio waveform data corresponding to the speech audio data; inputting the audio waveform data into a preset classification model to output a predicted label corresponding to the speech audio data; determining the loss value of the preset classification model based on the anti-counterfeiting label and the predicted label corresponding to the speech audio data; and adjusting the parameters of the preset classification model using the loss value to obtain a speaker verification model. This invention adopts a supervised learning approach, using real anti-counterfeiting labels to assist in determining the predicted label of the model, and trains the entire model by minimizing the error between the output and the real label, thereby improving training efficiency and reducing costs.

Description

Technical Field

[0001] This invention relates to the field of intelligent speech technology, and in particular to a method, apparatus, medium and device for training a speaker verification model. Background Technology

[0002] Intelligent voice technology is developing rapidly and is being applied in many scenarios, such as waking up smart homes and conversing with intelligent robots. However, it is also being used in some illegal and insecure areas. For example, voice attacks can be used to impersonate others' voices and commit fraud.

[0003] Currently, there are three main types of common voice attack fraud methods: speech synthesis, speech conversion, and audio playback. Speech synthesis and speech conversion involve synthesizing / converting speech data that closely resembles human speech based on technologies such as neural network models. Audio playback refers to attackers recording the victim's voice in certain scenarios and then playing back the recorded audio to the system, impersonating the victim's voice to deceive the system and the listener.

[0004] To counter the aforementioned voice attack fraud methods, certain anti-attack, anti-voice fraud, and authentication technologies already exist. Current mainstream anti-attack and authentication methods generally involve extracting speech features, then inputting these features into a pre-trained neural network model for authenticity judgment, and outputting the result. However, this feature-based approach often overlooks other essential characteristics of sound, leading to inaccurate model recognition and a less than ideal equal error rate (EER).

[0005] Therefore, there is an urgent need for a speaker verification model training method to reduce the loss of data information in speech feature extraction. Summary of the Invention

[0006] In view of the above problems, the present invention is proposed to provide a speaker verification model training method, apparatus, medium and device that overcomes or at least partially solves the above problems.

[0007] Other features and advantages of the invention will become apparent from the following detailed description, or may be learned in part by practice of the invention.

[0008] According to a first aspect of the present invention, a speaker verification model training method is provided, the speaker verification model training method comprising:

[0009] Waveform data extraction is performed on the acquired speech audio data to obtain the audio waveform data corresponding to the speech audio data;

[0010] The audio waveform data is input into a preset classification model to output the predicted label corresponding to the speech audio data through the preset classification model. The preset classification model includes a signal processing layer composed of a residual network of multiple feature modules, and the stride of the multiple feature modules in the residual network is different.

[0011] Based on the anti-counterfeiting labels corresponding to the pre-acquired voice audio data and the predicted labels output by the preset classification model, the loss value of the preset classification model is determined.

[0012] The speaker verification model is obtained by adjusting the parameters of the preset classification model using the loss value.

[0013] According to a second aspect of the present invention, a speaker verification model training apparatus is provided, the apparatus comprising:

[0014] The waveform extraction module is used to extract waveform data from the acquired speech audio data to obtain the audio waveform data corresponding to the speech audio data.

[0015] The classification prediction module is used to input the audio waveform data into a preset classification model, so as to output the predicted label corresponding to the speech audio data through the preset classification model. The preset classification model includes a signal processing layer composed of a residual network of multiple feature modules, and the stride of the multiple feature modules in the residual network is different.

[0016] The loss calculation module is used to determine the loss value of the preset classification model based on the anti-counterfeiting label corresponding to the pre-acquired voice audio data and the predicted label output by the preset classification model.

[0017] The output module is used to adjust the parameters of the preset classification model using the loss value to obtain the speaker verification model.

[0018] According to a third aspect of the present invention, a computer-readable storage medium is provided, wherein computer program instructions are stored therein, the computer program instructions being loaded and executed by a processor to perform the operations performed by the method described in any of the preceding claims.

[0019] According to a fourth aspect of the present invention, an electronic device is provided, including a processor and a memory, the memory storing computer program instructions executable by the processor, wherein when the processor executes the computer program instructions, it implements the instructions of any of the methods described above.

[0020] The technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages:

[0021] This invention provides a speaker verification model training method, apparatus, medium, and device. The method uses waveform data of speech audio data as input to a preset classification model, reducing data information loss. It employs supervised learning, using real anti-counterfeiting labels to assist in determining the model's predicted labels, thereby determining the loss value of the preset classification model. The parameters of the preset classification model are adjusted based on this loss value. The entire model is trained by minimizing the error between the output and the real label, rather than by progressively training different components to adjust the parameters of the entire model. This achieves speaker recognition verification model training, improves training efficiency, and reduces costs. The trained speaker recognition verification model effectively overcomes the problem of data information loss during speech feature extraction in existing technologies.

[0022] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A flowchart illustrating a speaker verification model training method provided in an embodiment of the present invention;

[0025] Figure 2 This is a schematic diagram of the process for voice endpoint detection in an embodiment of the present invention;

[0026] Figure 3 This is a flowchart illustrating the process of outputting the predicted labels corresponding to the speech audio data in an embodiment of the present invention.

[0027] Figure 4 This is a schematic diagram of the architecture of a pre-defined classification model.

[0028] Figure 5 This is a schematic diagram of the principle structure of a speaker verification model training device provided in an embodiment of the present invention. Detailed Implementation

[0029] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings.

[0030] The accompanying drawings illustrate various structural schematics according to embodiments of the present disclosure. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0031] In the context of this disclosure, when a layer / component is referred to as being "above" another layer / component, that layer / component may be directly above the other layer / component, or there may be an intermediate layer / component between them. Additionally, if a layer / component is "above" another layer / component in one orientation, then when the orientation is reversed, that layer / component may be "below" the other layer / component. In the context of this disclosure, similar or identical components may be denoted by the same or similar reference numerals.

[0032] To better understand the above technical solutions, the following will describe the above technical solutions in detail with reference to specific implementation methods. It should be understood that the embodiments of this disclosure and the specific features in the embodiments are detailed descriptions of the technical solutions of the present invention, rather than limitations on the technical solutions of the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0033] Figure 1 This is a flowchart illustrating a speaker verification model training method provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the speaker verification model training method includes the following steps:

[0034] S1. Extract waveform data from the acquired speech audio data to obtain the audio waveform data corresponding to the speech audio data;

[0035] In this embodiment of the invention, the voice audio data is obtained by performing voice endpoint detection on the collected raw voice audio data. This invention is used to identify voice attack fraud methods; therefore, the raw voice audio data can be audio data containing human voices from any source, and the human voices contained in the raw voice audio data should include both real and fake human voices. The raw voice audio data containing real human voices can be randomly collected in real-world scenarios; the raw voice audio data containing fake human voices can be collected through various means, such as TTS technology (randomly preparing fictitious text and converting it into speech), speech conversion technology (converting speaker A's speech data into speaker B's speech data, keeping the specific content of the speech data unchanged but changing the speaker to simulate different speakers' voices), audio mixing and noise addition (adding different types of reverberation or noise to real human voices), voiceprint synthesis technology (creating a simulated voice of a specific speaker), or any combination of the above methods.

[0036] Of course, it should be noted that the collected raw audio data contains interference from non-human voice audio data; that is, the raw audio data contains both human voice components and noise components. To improve the efficiency and accuracy of speech recognition, please refer to [reference needed]. Figure 2 As shown, in this embodiment of the invention, speech endpoint detection is performed on the collected raw audio data to obtain audio data that includes only human voice segments, specifically including:

[0037] S11. Extract the acoustic features and spectral features corresponding to the original audio data of the speech;

[0038] The acoustic features may include voiced and unvoiced sounds, vowels and consonants of the noisy speech signal; the spectral features may include the vibration frequency, vibration amplitude, loudness and timbre of the noisy speech signal.

[0039] S12. The acoustic features and spectral features are converted to obtain the acoustic feature vector corresponding to the acoustic features and the spectral feature vector corresponding to the spectral features;

[0040] After obtaining the acoustic features and spectral features, the acoustic features and spectral features are converted respectively, so that the acoustic features are converted into corresponding acoustic feature vectors and the spectral features are converted into corresponding spectral feature vectors.

[0041] S13. Obtain the classifier by inputting the acoustic feature vector and the spectral feature vector into the classifier to obtain the acoustic feature vector with added speech label and the spectral feature vector with added speech label.

[0042] In this embodiment of the invention, the classifier is a pre-trained classifier before speech endpoint detection. The classifier can classify the input acoustic feature vectors and spectral feature vectors into speech-class acoustic feature vectors and spectral feature vectors and into non-speech-class acoustic feature vectors and spectral feature vectors by adding speech labels and non-speech labels to the acoustic feature vectors and spectral feature vectors. By inputting the acoustic feature vectors and spectral feature vectors into the classifier, the classifier classifies the input acoustic feature vectors and spectral feature vectors.

[0043] S14. Analyze the acoustic feature vector and the spectral feature vector with added speech tags to obtain the original speech signal;

[0044] By parsing the acoustic feature vector and spectral feature vector with added speech tags, the spectra corresponding to the acoustic and spectral features with added speech tags are obtained respectively. The spectra corresponding to the acoustic and spectral features are then converted into corresponding speech signals, thereby enabling the parsing of the original speech signal corresponding to the spectrum.

[0045] S15. Determine the start and end points of the original speech signal according to the timing of the original speech signal to obtain the speech audio data.

[0046] Based on the timing of the original speech signal after adding voice tags, this embodiment of the invention can take the first occurrence of the voice frame with added voice tags as the starting point of the original speech signal and the last occurrence of the voice frame corresponding to the voice tag as the ending point of the original speech signal to obtain the speech audio data.

[0047] Before performing waveform extraction on the speech audio data, this embodiment of the invention can also annotate the speech audio data to obtain anti-counterfeiting tags for the speech audio data; the anti-counterfeiting tags are used to reflect the true content of the speech audio data, and the anti-counterfeiting tags can be added, for example, by a classifier (which can be the classifier in step S13, or another classifier trained separately).

[0048] In this embodiment of the invention, waveform extraction is performed on the acquired speech audio data to obtain audio waveform data corresponding to the speech audio data. Specifically, in this embodiment of the invention, waveform extraction is performed on the speech audio data to obtain audio waveform data corresponding to the speech audio data, which includes: constructing a waveform extraction model; and using the waveform extraction model to extract the temporal features of the speech audio data to obtain the audio waveform data.

[0049] In this embodiment of the invention, a waveform extraction model (which can be pre-trained) can be constructed to extract features in the time domain of the speech audio data to obtain the audio waveform data corresponding to the speech audio data. The waveform extraction model can be a model constructed using convolutional neural networks and recurrent neural networks that can capture the temporal dynamics of audio data.

[0050] S2. Input the audio waveform data into a preset classification model to output the predicted label corresponding to the speech audio data through the preset classification model; the preset classification model includes a signal processing layer composed of a residual network of multiple feature modules, and the stride of the multiple feature modules in the residual network is different;

[0051] Specifically, see Figure 3 As shown, regarding step S2, in this embodiment of the invention, the audio waveform data is input into a preset classification model to output the predicted label corresponding to the speech audio data through the preset classification model. Specifically, this includes:

[0052] S21. Construct a preset classification model, which includes a pre-emphasis layer, a signal processing layer, an attention layer, a recurrent neural network layer, and a fully connected layer.

[0053] See Figure 4 As shown, the preset classification model constructed in this embodiment of the invention includes a pre-emphasis layer, a signal processing layer, an attention layer, a recurrent neural network layer, and a fully connected layer connected in sequence. Each layer uses the output of the previous layer as the input of the current layer. The audio waveform data is input to the pre-emphasis layer and processed by the fully connected layer to output the predicted label corresponding to the speech audio data.

[0054] S22. Input the audio waveform data into the preset classification model, and perform pre-emphasis processing on the input audio waveform data through the pre-emphasis layer to obtain pre-processed waveform data;

[0055] The pre-emphasis layer is used to pre-emphasize the input audio waveform data, that is, to emphasize the high-frequency signals in the audio waveform data and stabilize the scale of the audio waveform data.

[0056] Specifically, for example, in this embodiment of the invention, a kernel of length 2 is set in the pre-emphasis layer, where the two weights are initialized to -0.97 and 1 respectively. The pre-emphasis layer is then placed on the first hidden layer of the preset classification model and fine-tuned along with other hidden layers during training. It should be noted that during training, the pre-emphasis layer should be fine-tuned using a relatively lower learning rate than other hidden layers (such as signal processing layers, attention layers, recurrent neural network layers, and fully connected layers) to prevent its weights from changing too rapidly. This improves the stability of the audio waveform data output by the pre-emphasis layer, reduces data information loss in the audio waveform data, and enables more accurate extraction of feature information contained in the audio waveform data during subsequent related processing.

[0057] S23. The preprocessed waveform data is processed by the signal processing layer to perform signal transformation and feature information is extracted to obtain audio feature data;

[0058] Combination Figure 4 As shown, in this embodiment of the invention, the signal processing layer includes four feature modules: a first extended feature map module, a second extended feature map module, a third extended feature map module, and a convolutional layer. The first extended feature map module, the second extended feature map module, the third extended feature map module, and the convolutional layer form a residual network.

[0059] The first, second, and third extended feature map modules have the same structure. The first, second, or third extended feature map module is used to process the preprocessed waveform data into a time-frequency representation. In this embodiment, the preprocessed waveform data is transformed to extract time-domain feature information, for example, using Short-Time Fourier Transform (STFT) and Mel-frequency cepstral coefficients (MFCC). In this embodiment, the preprocessed waveform data is transformed to extract frequency-domain feature information, for example, by converting the audio signal into a time-frequency diagram using STFT, representing changes in both time and frequency domains.

[0060] Specifically, the method described in this embodiment of the invention for performing signal transformation processing on the preprocessed waveform data through the signal processing layer and extracting audio feature data includes:

[0061] The preprocessed waveform data is processed by the first extended feature map module to obtain the first output feature;

[0062] The second output feature is obtained by processing the first output feature through the second extended feature map module;

[0063] The third extended feature map module processes the first output feature and the second output feature to obtain the third output feature;

[0064] The first output feature, the second output feature, and the third output feature are input into the convolutional layer to extract feature information and obtain the audio feature data.

[0065] In this embodiment of the invention, the degree of time-series compression of the preprocessed waveform data is configured by the stride size. The stride size is directly proportional to processing efficiency and inversely proportional to performance; that is, a smaller stride slows down the processing speed but produces better performance, while a larger stride provides faster processing speed but reduces performance. To avoid overfitting, in this embodiment, the strides of the first and second extended feature map modules are different, and the strides of the first and second extended feature map modules are larger than those of the third extended feature map module and the convolution module. The stride of the layers is set; for example, for the first extended feature map module, its kernel is set to 3 and the corresponding stride is set to 2, while for the second extended feature map module, its kernel is set to 3 and the corresponding stride is set to 3, thereby alleviating the overfitting phenomenon that occurs during feature extraction of the preprocessed waveform data; while the third extended feature map module and the convolutional layer require better performance, so the stride of the third extended feature map module and the convolutional layer is set to a smaller size. For example, in the embodiment of the present invention, the kernel of the third extended feature map module and the convolutional layer can be set to 1 and the corresponding stride can be set to 1.

[0066] In this embodiment of the invention, the first, second, and third extended feature map modules all incorporate an FMS (Fiduccia-Mattheyses-Sanchis, multi-path hypergraph partitioning algorithm) module. Max pooling is applied before applying the FMS module. Combining the FMS module and max pooling alleviates overfitting during the preprocessing of waveform data. Each extended feature map module contains four layers: three sequentially connected convolutional layers and one pooling layer. The first convolutional layer is constructed using a one-dimensional convolutional neural network, with a ReLU activation function and a BN network layer to accelerate network convergence. The second convolutional layer is constructed using a one-dimensional convolutional neural network with N-fold expansion coefficients (optionally, N can be 2, 4, etc.). This scalable convolutional neural network allows for adjustment of the convolutional field of view for the input data, enabling feature extraction from different perspectives. A ReLU activation function is also added. The third convolutional layer has the same structure as the first, using a one-dimensional convolutional neural network, with the addition of a ReLU activation function and a Batch Normalization (BN) layer to accelerate network convergence. The pooling layer consists of a max pooling function and an FMS module.

[0067] After obtaining the first, second, and third output features, they are all input into the convolutional layer to extract feature information (e.g., time-domain and frequency-domain feature information) to obtain the audio feature data. It should be noted that the first, second, and third output features are all data containing feature information obtained after feature extraction from the preprocessed waveform data. The difference lies in the different parameters set for different feature modules, such as the kernel and stride mentioned earlier, resulting in different extracted feature information. In this embodiment, the convolutional layer can use different kernel sizes and numbers to capture features at different scales. This embodiment is based on a residual network structure, which uses residual connections between the output and input of the convolutional layer to transmit gradients and information, mitigating the problems of gradient vanishing and information loss.

[0068] It should be noted that, in this embodiment of the invention, a nonlinear activation function (ReLU) is added to the signal processing layer to process the result after residual connection, and the audio feature data obtained from the extracted feature results is output.

[0069] S24. The audio feature data is weighted through the attention layer to obtain weighted feature data;

[0070] In this embodiment of the invention, the attention layer enhances the model's focus on important parts of the audio feature data by introducing an attention mechanism, enabling more focused selection of important frames related to the audio feature data and ignoring irrelevant audio frames, which is beneficial for better classification of the audio feature data in the future. For example, the attention parameters of the audio feature data can be determined by using a sigmoid nonlinear activation function. The attention parameters can be used as weights to perform weighted processing based on the audio feature data and the attention parameters to obtain a weighted feature representation, i.e., the weighted feature data.

[0071] S25. The weighted feature data is modeled based on temporal relationships through the recurrent neural network layer to obtain enhanced features;

[0072] In this embodiment of the invention, the recurrent neural network layer is based on a gated convolutional neural network (CNN) to model the temporal relationship of the weighted feature data to obtain the enhanced features; the gated convolutional neural network (CNN) can capture the temporal correlation in the audio signal, greatly enhancing the temporal modeling capability of the features.

[0073] S26. The enhanced features are mapped to the output space through the fully connected layer, and the predicted labels corresponding to the speech audio data are output.

[0074] Specifically, the step of mapping the enhanced features to the output space through the fully connected layer and outputting the predicted label corresponding to the speech audio data includes: mapping the enhanced features to the output space; in the output space, using an activation function to make a prediction based on the enhanced features to generate a probability distribution result; and determining and outputting the predicted label corresponding to the speech audio data based on the probability distribution result.

[0075] The output layer can be selected according to actual needs in the embodiments of the present invention. For example, in the embodiments of the present invention, a classification-based training framework is adopted, and the classification cross-entropy objective function is used. The loss is calculated by comparing the soft maximum nonlinear classification head with the single hot ground truth label.

[0076] For example, in this embodiment of the invention, AAM-softmax is used as the objective function, as shown in the following formula:

[0077]

[0078] Define x respectively i y iW and W represent the speaker embedding (or embedding code), its corresponding predicted label, and the weight matrix of the classification head, where the classification head is the fully connected layer. Each row of the weight matrix W is the center vector (center point) of the speaker in the augmented features; where cos(θy) i i) is the L2 norm normalized x i The dot product between Wj and Sj, where s and m are a scaling factor and a margin hyperparameter, respectively, and i is the index of a utterance within a mini-batch size N, and 0. <i<N。

[0079] An activation function is then used to predict based on the enhanced features, generating a probability distribution result. The probability distribution result indicates whether the speech audio data is a real human voice or a fake human voice. Based on the probability distribution result, the predicted label corresponding to the speech audio data is determined and output.

[0080] S3. Determine the loss value of the preset classification model based on the anti-counterfeiting label corresponding to the pre-acquired voice audio data and the predicted label output by the preset classification model.

[0081] In this embodiment of the invention, the loss value of the preset classification model can be calculated and determined using various loss functions, such as cross-entropy loss function, hinge loss function, exponential loss function, etc.

[0082] S4. Adjust the parameters of the preset classification model using the loss value to obtain the speaker verification model.

[0083] In this embodiment of the invention, based on the determined loss value of the preset classification model, the parameters of each level within the preset classification model are adjusted accordingly to obtain a speaker verification model. The parameters can be the level itself, or the kernel, stride, learning rate, or network function used for each level, etc., which can be selected and adjusted according to the actual situation. For example, in this embodiment of the invention, after classification training is completed, the classification head (e.g., the fully connected layer) can be removed to obtain the speaker verification model.

[0084] The speaker verification model training method described in this embodiment of the invention has the following advantages compared with the prior art:

[0085] 1. The waveform data of the speech audio data is used as the input of the preset classification model, and the scale of the original waveform data is stabilized by the pre-emphasis technology, which reduces the loss of data information;

[0086] 2. A new model architecture was constructed, including a pre-emphasis layer, a signal processing layer, an attention layer, a recurrent neural network layer, and a fully connected layer. The signal processing layer is a residual network composed of a first extended feature map module, a second extended feature map module, a third extended feature map module, and a convolutional layer. This makes the extracted features more reliable when performing feature extraction on waveform data, thereby improving the accuracy of the entire model training process.

[0087] 3. The model is trained using supervised learning. Real anti-counterfeiting labels are used to assist in determining the predicted labels of the model, thereby determining the loss value of the preset classification model. At the same time, an end-to-end model structure is used to adjust the parameters of the preset classification model through the loss value. The entire model is trained by minimizing the error between the output and the real label, rather than by gradually training different components to adjust the parameters of the entire model. This achieves the training of the speaker recognition verification model, improves training efficiency, and reduces costs. The trained speaker recognition verification model effectively overcomes the problem of data information loss during speech feature extraction in the prior art.

[0088] Based on the above embodiments, as a supplement to the above... Figure 1 The present invention provides an embodiment of a speaker verification model training device for the implementation of the method shown, which is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices, see reference. Figure 5 As shown, the speaker verification model training device includes:

[0089] The waveform extraction module 100 is used to extract waveform data from the acquired speech audio data to obtain the audio waveform data corresponding to the speech audio data.

[0090] The classification prediction module 200 is used to input the audio waveform data into a preset classification model, so as to output the predicted label corresponding to the speech audio data through the preset classification model. The preset classification model includes a signal processing layer composed of a residual network of multiple feature modules, and the stride of the multiple feature modules in the residual network is different.

[0091] The loss calculation module 300 is used to determine the loss value of the preset classification model based on the anti-counterfeiting label corresponding to the pre-acquired voice audio data and the predicted label output by the preset classification model.

[0092] The output module 400 is used to adjust the parameters of the preset classification model using the loss value to obtain the speaker verification model.

[0093] The speaker verification model training device described in this embodiment of the invention can execute the speaker verification model training method provided in the above embodiments. The speaker verification model training device has the corresponding functional steps and beneficial effects of the speaker verification model training method described in the above embodiments. For details, please refer to the embodiments of the speaker verification model training method described above. The embodiments of the present invention will not be repeated here.

[0094] This invention also provides an electronic device, which may include a processor and a memory, wherein the processor and memory can be connected via a bus or other means. The processor may be a Central Processing Unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations thereof. The memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the speaker verification model training method in this invention embodiment. The processor executes various functional applications and data processing by running the non-transitory software programs, instructions, and modules stored in the memory, thereby implementing the speaker verification model training method in the above method embodiment.

[0095] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. The one or more modules are stored in the memory and, when executed by the processor, perform the speaker verification model training method as described in the above method embodiments. Specific details of the above electronic device can be understood by referring to the corresponding descriptions and effects in the above method embodiments, and will not be repeated here. Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it may include the processes of the embodiments of the above methods. The storage medium may be a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD), etc.; the storage medium may also include a combination of the above types of memory.

[0096] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0097] Similarly, it should be understood that, in order to streamline this disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0098] It should be noted that the above embodiments are illustrative of the invention and not restrictive of the invention, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims.

Claims

1. A speaker verification model training method, characterized in that, The method includes: Waveform data extraction is performed on the acquired speech audio data to obtain the audio waveform data corresponding to the speech audio data; the original speech audio data is audio data containing human voices, and the human voices contained in the original speech audio data should include both real human voices and fake human voices; The audio waveform data is input into a preset classification model to output the predicted label corresponding to the speech audio data through the preset classification model. The preset classification model includes a signal processing layer composed of a residual network of multiple feature modules, and the stride of the multiple feature modules in the residual network is different. Based on the anti-counterfeiting labels corresponding to the pre-acquired voice audio data and the predicted labels output by the preset classification model, the loss value of the preset classification model is determined. The speaker verification model is obtained by adjusting the parameters of the preset classification model using the loss value. Before performing waveform data extraction on the acquired speech audio data, the process also includes: Extract the acoustic features and spectral features corresponding to the original audio data of the speech; The acoustic features and spectral features are transformed to obtain the acoustic feature vector corresponding to the acoustic features and the spectral feature vector corresponding to the spectral features; Obtain a classifier by inputting the acoustic feature vector and spectral feature vector into the classifier to obtain an acoustic feature vector with added speech tags and a spectral feature vector with added speech tags. The acoustic feature vector and the spectral feature vector with added speech tags are analyzed to obtain the original speech signal; The start and end points of the original speech signal are determined based on the timing of the original speech signal to obtain the speech audio data.

2. The speaker verification model training method according to claim 1, characterized in that, The step of extracting waveform data from the acquired speech audio data to obtain the corresponding audio waveform data includes: Construct a waveform extraction model; The waveform extraction model is used to extract the temporal features of the speech audio data to obtain the audio waveform data.

3. The speaker verification model training method according to claim 1, characterized in that, The preset classification model includes a pre-emphasis layer, a signal processing layer, an attention layer, a recurrent neural network layer, and a fully connected layer; the step of inputting the audio waveform data into the preset classification model to output the predicted label corresponding to the speech audio data through the preset classification model includes: The audio waveform data is input into the preset classification model, and the input audio waveform data is pre-emphasized through the pre-emphasis layer to obtain pre-processed waveform data. The signal processing layer performs signal transformation processing on the preprocessed waveform data and extracts feature information to obtain audio feature data. The audio feature data is weighted by the attention layer to obtain weighted feature data; Enhanced features are obtained by modeling the weighted feature data based on temporal relationships through the recurrent neural network layer; The enhanced features are mapped to the output space through the fully connected layer, and the predicted labels corresponding to the speech audio data are output.

4. The speaker verification model training method according to claim 3, characterized in that: The signal processing layer includes four feature modules: a first extended feature map module, a second extended feature map module, a third extended feature map module, and a convolutional layer. The first extended feature map module, the second extended feature map module, the third extended feature map module, and the convolutional layer form a residual network. The first extended feature map module, the second extended feature map module, and the third extended feature map module have the same structure. However, the stride of the first extended feature map module and the second extended feature map module is different, and the stride of the first extended feature map module and the second extended feature map module is greater than the stride of the third extended feature map module and the convolutional layer.

5. The speaker verification model training method according to claim 4, characterized in that, The step of performing signal transformation processing on the preprocessed waveform data through the signal processing layer and extracting feature information to obtain audio feature data includes: The preprocessed waveform data is processed by the first extended feature map module to obtain the first output feature; The second output feature is obtained by processing the first output feature through the second extended feature map module; The third output feature is obtained by processing the first output feature and the second output feature through the third extended feature map module. The first output feature, the second output feature, and the third output feature are input into the convolutional layer to extract feature information and obtain the audio feature data.

6. The speaker verification model training method according to claim 3, characterized in that, The step of mapping the enhanced features to the output space through the fully connected layer and outputting the predicted label corresponding to the speech audio data includes: Map the enhanced features to the output space; In the output space, an activation function is used to predict based on the enhanced features to generate a probability distribution result. Based on the probability distribution result, the predicted label corresponding to the speech audio data is determined and output.

7. A speaker verification model training device, applied to the speaker verification model training method according to any one of claims 1-6, characterized in that, The device includes: The waveform extraction module is used to extract waveform data from the acquired speech audio data to obtain the audio waveform data corresponding to the speech audio data. The classification prediction module is used to input the audio waveform data into a preset classification model, so as to output the predicted label corresponding to the speech audio data through the preset classification model. The preset classification model includes a signal processing layer composed of a residual network of multiple feature modules, and the stride of the multiple feature modules in the residual network is different. The loss calculation module is used to determine the loss value of the preset classification model based on the anti-counterfeiting label corresponding to the pre-acquired voice audio data and the predicted label output by the preset classification model. The output module is used to adjust the parameters of the preset classification model using the loss value to obtain the speaker verification model.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which are loaded and executed by a processor to perform the operations performed by the method as described in any one of claims 1-6.

9. An electronic device comprising a processor and a memory, characterized in that, The memory stores computer program instructions that can be executed by the processor, and when the processor executes the computer program instructions, it implements the instructions of the method as described in any one of claims 1-6.

Images