Speaker verification model training method and device, medium and equipment

Abstract

The invention discloses a speaker verification model training method and device, a medium and equipment, and the method comprises the steps: carrying out the waveform extraction of obtained voice audio data, and obtaining the audio waveform data corresponding to the voice audio data; inputting the audio waveform data into a preset classification model so as to output a prediction label corresponding to the voice audio data through the preset classification model; determining a loss value of the preset classification model according to an anti-counterfeit label and a prediction label corresponding to the voice audio data; and performing parameter adjustment on the preset classification model through the loss value to obtain a speaker verification model. According to the method, the real anti-counterfeit label is utilized to perform auxiliary judgment on the prediction label of the model in a supervised learning mode, the error between the minimum output and the real label is utilized to train the whole model, the training efficiency is improved, and the cost is reduced.

Description

Technical Field

[0001] The present invention relates to the field of intelligent speech technology, and in particular to a speaker verification model training method, device, medium and equipment. Background Art

[0002] Intelligent voice technology is developing rapidly and is applied in many scenarios, such as waking up smart homes and talking to smart robots. However, voice technology is also being applied in some illegal and unsafe fields. For example, voice attack fraud is used to imitate other people's voices and conduct fraudulent activities.

[0003] At present, there are three common voice attack fraud methods, namely speech synthesis, speech conversion, and recording playback. Speech synthesis and speech conversion are based on technologies such as neural network models to synthesize / convert speech data that is similar to real people's voices. Recording playback means that the attacker uses a recording device to record the voice of the attacked person in certain scenarios, and then plays the recorded voice back to the system to impersonate the voice of the attacked person to deceive the system and the listener.

[0004] At present, there are certain anti-attack, anti-voice fraud, and authentication technologies for the above-mentioned voice attack and fraud methods. The current mainstream anti-attack and authentication methods are generally: voice feature extraction of the voice, and then inputting the extracted voice features into a pre-trained neural network model for authenticity discrimination, and outputting the discrimination results. However, this method of judging in the feature dimension often ignores other characteristics of the sound nature, resulting in inaccurate model recognition and unsatisfactory equal error rate (EER).

[0005] Therefore, there is an urgent need for a speaker verification model training method to reduce the loss of data information in speech feature extraction. Summary of the invention

[0006] In view of the above problems, the present invention is proposed to provide a speaker verification model training method, apparatus, medium and device that overcome the above problems or at least partially solve the above problems.

[0007] Other features and advantages of the present invention will become apparent from the following detailed description, or may be learned in part by practice of the present invention.

[0008] According to a first aspect of an embodiment of the present invention, a speaker verification model training method is provided, the speaker verification model training method comprising:

[0009] Extracting waveform data from the acquired speech and audio data to obtain audio waveform data corresponding to the speech and audio data;

[0010] Inputting the audio waveform data into a preset classification model to output a prediction label corresponding to the speech audio data through the preset classification model, wherein the preset classification model includes a signal processing layer of a residual network composed of multiple feature modules, and the strides of the multiple feature modules in the residual network are different;

[0011] Determine the loss value of the preset classification model according to the anti-counterfeiting label corresponding to the voice audio data obtained in advance and the predicted label output by the preset classification model;

[0012] The parameters of the preset classification model are adjusted according to the loss value to obtain a speaker verification model.

[0013] According to a second aspect of an embodiment of the present invention, a speaker verification model training device is provided, the device comprising:

[0014] A waveform extraction module is used to extract waveform data from the acquired speech audio data to obtain audio waveform data corresponding to the speech audio data;

[0015] A classification prediction module, used for inputting the audio waveform data into a preset classification model to output a prediction label corresponding to the speech audio data through the preset classification model, wherein the preset classification model includes a signal processing layer of a residual network composed of multiple feature modules, and the strides of the multiple feature modules in the residual network are different;

[0016] A loss calculation module, used to determine the loss value of the preset classification model according to the anti-counterfeiting label corresponding to the voice audio data obtained in advance and the predicted label output by the preset classification model;

[0017] The output module is used to adjust the parameters of the preset classification model according to the loss value to obtain a speaker verification model.

[0018] According to a third aspect of an embodiment of the present invention, a computer-readable storage medium is provided, in which computer program instructions are stored. The computer program instructions are loaded and executed by a processor to implement the operations performed by any of the methods described above.

[0019] According to a fourth aspect of an embodiment of the present invention, there is provided an electronic device, comprising a processor and a memory, wherein the memory stores computer program instructions that can be executed by the processor, and when the processor executes the computer program instructions, it implements instructions of any of the methods described above.

[0020] The technical solution provided in the embodiments of the present invention has at least the following technical effects or advantages:

[0021] The embodiments of the present invention provide a speaker verification model training method, device, medium and equipment. The method uses waveform data of speech audio data as the input of a preset classification model, reduces the loss of data information, adopts a supervised learning method to use real anti-counterfeiting labels to assist in determining the predicted labels of the model to determine the loss value of the preset classification model, adjusts the parameters of the preset classification model according to the loss value, and trains the entire model by minimizing the error between the output and the real label, rather than adjusting the parameters of the entire model by gradually training different components. The training of the speaker recognition verification model is realized, the training efficiency is improved and the cost is reduced. The speaker recognition verification model obtained by training effectively overcomes the problem of data information loss during speech feature extraction in the prior art.

[0022] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0024] Figure 1 A flowchart of a speaker verification model training method provided by an embodiment of the present invention;

[0025] Figure 2 A schematic diagram of a process for performing voice endpoint detection according to an embodiment of the present invention;

[0026] Figure 3 A schematic diagram of a process for outputting a prediction label corresponding to the speech audio data according to an embodiment of the present invention;

[0027] Figure 4 A schematic diagram for reference of the architecture of the preset classification model;

[0028] Figure 5 A schematic diagram of the principle structure of a speaker verification model training device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings.

[0030] Various structural schematic diagrams according to embodiments of the present disclosure are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may further design regions / layers with different shapes, sizes, and relative positions according to actual needs.

[0031] In the context of the present disclosure, when a layer / element is referred to as being "on" another layer / element, the layer / element may be directly on the other layer / element, or there may be an intervening layer / element therebetween. Additionally, if a layer / element is "on" another layer / element in one orientation, the layer / element may be "below" the other layer / element when the orientation is reversed. In the context of the present disclosure, similar or identical components may be denoted by the same or similar reference numerals.

[0032] In order to better understand the above-mentioned technical scheme, the above-mentioned technical scheme will be described in detail below in combination with specific implementation methods. It should be understood that the embodiments of the present disclosure and the specific features in the embodiments are detailed descriptions of the technical scheme of the present invention, rather than limitations on the technical scheme of the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0033] Figure 1 FIG. 1 is a flow chart of a speaker verification model training method provided by an embodiment of the present invention. Figure 1 As shown, the speaker verification model training method includes the following steps:

[0034] S1, extracting waveform data from the acquired voice and audio data to obtain audio waveform data corresponding to the voice and audio data;

[0035] In an embodiment of the present invention, the voice audio data is obtained by performing voice endpoint detection on the collected original voice audio data. The present invention is used to identify voice attack fraud methods, therefore, the original voice audio data can be audio data containing human voice from any source, and the human voice contained in the original voice audio data should include both real human voice and false human voice; wherein, the original voice audio data of the real human voice can be randomly collected in a real scene; the original voice audio data of the false human voice can be collected by a variety of means, for example, TTS technology (randomly preparing fictitious text and converting the text into voice), voice conversion technology (i.e. converting the voice data of speaker A into the voice data of speaker B, the specific content of this voice data remains unchanged, what is changed is the speaker, so as to simulate the voice of different speakers), audio mixing and noise addition (i.e. adding different types of reverberation or noise to the real human voice), voiceprint synthesis technology (i.e. creating a voice that simulates a specific speaker), or any combination of the above multiple means.

[0036] Of course, it should be noted that since the collected original speech audio data contains interference from non-human voice audio data, that is, the original speech audio data contains both human voice and noise parts, in order to improve the efficiency and accuracy of speech recognition, refer to Figure 2 As shown, the embodiment of the present invention performs voice endpoint detection on the collected original voice audio data to obtain the voice audio data including only human voice segments, specifically including:

[0037] S11, extracting acoustic features and spectral features corresponding to the original audio data of the speech;

[0038] Among them, the acoustic features may include characteristic information such as unvoiced sounds, voiced sounds, vowels, consonants, etc. of the noisy speech signal; the spectral features may include characteristic information such as vibration frequency, vibration amplitude, loudness, timbre, etc. of the noisy speech signal.

[0039] S12, converting the acoustic feature and the spectral feature to obtain an acoustic feature vector corresponding to the acoustic feature and a spectral feature vector corresponding to the spectral feature;

[0040] After obtaining the acoustic features and the spectral features, the acoustic features and the spectral features are converted respectively, the acoustic features are converted into corresponding acoustic feature vectors, and the spectral features are converted into corresponding spectral feature vectors.

[0041] S13, obtaining a classifier, inputting the acoustic feature vector and the spectral feature vector into the classifier, and obtaining the acoustic feature vector with the voice tag added and the spectral feature vector with the voice tag added;

[0042] In the embodiment of the present invention, the classifier is a classifier trained before performing speech endpoint detection, and the classifier can add speech labels and non-speech labels to the acoustic feature vector and the spectral feature vector, and divide the input acoustic feature vector and the spectral feature vector into acoustic feature vectors and spectral feature vectors of speech class and acoustic feature vectors and spectral feature vectors of non-speech class. By inputting the acoustic feature vector and the spectral feature vector into the classifier, the classifier is used to classify the input acoustic feature vector and the spectral feature vector.

[0043] S14, parsing the acoustic feature vector with the added voice tag and the spectral feature vector with the added voice tag to obtain an original voice signal;

[0044] By parsing the acoustic feature vector with the added voice tag and the spectral feature vector with the added voice tag, the spectra corresponding to the acoustic features and spectral features with the added voice tag are obtained respectively, and the spectra corresponding to the acoustic features and spectral features are converted into corresponding voice signals, thereby parsing the original voice signal corresponding to the spectra.

[0045] S15. Determine the starting point and the ending point corresponding to the original voice signal according to the time sequence of the original voice signal to obtain the voice audio data.

[0046] According to the timing of the original voice signal after the voice tag is added, the embodiment of the present invention can take the first appearance of the voice frame with the voice tag added as the starting point of the original voice signal, and the last appearance of the voice frame corresponding to the voice tag as the ending point of the original voice signal to obtain the voice audio data.

[0047] Before waveform extraction is performed on the speech audio data, the embodiment of the present invention may also annotate the speech audio data to obtain an anti-counterfeiting label for the speech audio data; the anti-counterfeiting label is used to reflect the true content of the speech audio data, and the anti-counterfeiting label may be added, for example, by a classifier (which may be the classifier in step S13 or another classifier trained separately).

[0048] The embodiment of the present invention performs waveform extraction on the acquired speech audio data to obtain audio waveform data corresponding to the speech audio data; specifically, the embodiment of the present invention performs waveform extraction on the speech audio data to obtain audio waveform data corresponding to the speech audio data, including: constructing a waveform extraction model; using the waveform extraction model to extract time domain features of the speech audio data to obtain the audio waveform data.

[0049] The embodiment of the present invention can perform feature extraction in the time domain of the speech audio data by constructing a waveform extraction model (which can be pre-trained) to obtain audio waveform data corresponding to the speech audio data, wherein the waveform extraction model can be a model constructed using convolutional neural networks and recurrent neural networks that can capture the time dynamics of audio data.

[0050] S2, inputting the audio waveform data into a preset classification model to output a prediction label corresponding to the speech audio data through the preset classification model; the preset classification model includes a signal processing layer of a residual network composed of multiple feature modules, and the strides of the multiple feature modules in the residual network are different;

[0051] Specifically, see Figure 3 As shown, regarding step S2, the embodiment of the present invention inputs the audio waveform data into a preset classification model, so as to output the prediction label corresponding to the speech audio data through the preset classification model, specifically including:

[0052] S21, constructing a preset classification model, wherein the preset classification model includes a pre-emphasis layer, a signal processing layer, an attention layer, a recurrent neural network layer, and a fully connected layer;

[0053] See also Figure 4 As shown, the preset classification model constructed by the embodiment of the present invention includes a pre-emphasis layer, a signal processing layer, an attention layer, a recurrent neural network layer and a fully connected layer connected in sequence. Each layer uses the output of the previous layer as the input of the current layer. The audio waveform data is input into the pre-emphasis layer, and after being processed by the fully connected layer, the predicted label corresponding to the speech audio data is output.

[0054] S22, inputting the audio waveform data into the preset classification model, and performing pre-emphasis processing on the input audio waveform data through the pre-emphasis layer to obtain pre-processed waveform data;

[0055] The pre-emphasis layer is used to perform pre-emphasis processing on the input audio waveform data, that is, to emphasize the high-frequency signal in the audio waveform data and stabilize the scale of the audio waveform data.

[0056] Specifically, for example, an embodiment of the present invention sets a kernel of length 2 in the pre-emphasis layer, wherein two weights are initialized to -0.97 and 1 respectively, and sets the pre-emphasis layer on the first hidden layer in the preset classification model, and fine-tunes it together with other hidden layers during the training process. It should be noted that during the training process, the pre-emphasis layer should be fine-tuned using a relatively lower learning rate than other hidden layers (such as the signal processing layer, the attention layer, the recurrent neural network layer, and the fully connected layer, etc.) to prevent its weight from changing too quickly, so that the stability of the audio waveform data output by the pre-emphasis layer is improved, and the data information loss in the audio waveform data is reduced. At the same time, when the relevant processing is performed based on the audio waveform data in the subsequent process, the feature information contained in the audio waveform data can be more accurately extracted.

[0057] S23, performing signal transformation processing on the preprocessed waveform data through the signal processing layer, and extracting feature information to obtain audio feature data;

[0058] Combination Figure 4 As shown, in an embodiment of the present invention, the signal processing layer includes four feature modules, namely a first extended feature map module, a second extended feature map module, a third extended feature map module and a convolutional layer, and the first extended feature map module, the second extended feature map module, the third extended feature map module and the convolutional layer constitute a residual network.

[0059] The structures of the first extended feature map module, the second extended feature map module and the third extended feature map module are the same, and the first extended feature map module, the second extended feature map module or the third extended feature map module is used to process the preprocessed waveform data into a time and frequency spectrum representation; wherein, the embodiment of the present invention transforms the preprocessed waveform data to extract time domain feature information, for example, short-time Fourier transform (STFT) and Mel-frequency cepstral coefficients (MFCC) can be used; the embodiment of the present invention transforms the preprocessed waveform data to extract frequency domain feature information, for example, the audio signal can be converted into a time-frequency diagram through STFT, that is, to represent changes in time and frequency domains.

[0060] Specifically, the embodiment of the present invention performs signal transformation processing on the preprocessed waveform data through the signal processing layer, and extracts audio feature data, including:

[0061] Processing the preprocessed waveform data by the first extended feature map module to obtain a first output feature;

[0062] Processing the first output feature by the second extended feature map module to obtain a second output feature;

[0063] Processing the first output feature and the second output feature by the third extended feature map module to obtain a third output feature;

[0064] The first output feature, the second output feature and the third output feature are input into the convolution layer, and feature information is extracted to obtain the audio feature data.

[0065] The embodiment of the present invention configures the degree of compression of the processing time series of the preprocessed waveform data by the size of the stride, and the size of the stride is proportional to the processing efficiency and inversely proportional to the performance, that is, a smaller stride will slow down the processing rate but produce better performance, and conversely, a larger stride can provide a faster processing speed but reduce performance; wherein, in order to avoid overfitting, in the embodiment of the present invention, the stride set by the first extended feature map module and the second extended feature map module is different, and the stride set by the first extended feature map module and the second extended feature map module is greater than that of the third extended feature map module and the convolution. The stride of the layer setting; for example, for the first extended feature map module, its kernel is set to 3, and the corresponding stride is set to 2, while for the second extended feature map module, its kernel is set to 3, and the corresponding stride is set to 3, so as to alleviate the overfitting phenomenon in the process of feature extraction for the preprocessed waveform data; and the third extended feature map module and the convolution layer need better performance, so the stride of the third extended feature map module and the convolution layer is smaller, for example, in an embodiment of the present invention, the kernel of the third extended feature map module and the convolution layer can be set to 1, and the corresponding stride can be set to 1.

[0066] In an embodiment of the present invention, the first extended feature map module, the second extended feature map module and the third extended feature map module are all added with FMS (Fiduccia-Mattheyses-Sanchis, multi-way hypergraph partitioning algorithm) modules, and the maximum pooling technology is selected to be applied before the FMS module is applied. The combination of the FMS module and the maximum pooling technology can alleviate the overfitting problem in the preprocessing waveform data processing process. Among them, each extended feature map module includes four layers, including three sequentially connected convolutional layers and a pooling layer. The first convolutional layer is constructed using a one-dimensional convolutional neural network, and a ReLU activation function and a BN network layer are added to speed up the network convergence speed. The second convolutional layer is constructed using a one-dimensional convolutional neural network with an N-fold expansion coefficient (optionally, N can be selected as 2, 4, etc.). Through the scalable convolutional neural network, the convolutional field of view of the convolutional neural network for the input data can be adjusted, and the input data is feature extracted from different fields of view, and at the same time, a ReLU activation function is added. The third convolutional layer has the same structure as the first convolutional layer, that is, it is built using a one-dimensional convolutional neural network, and adds a ReLU activation function and a BN network layer to speed up the network convergence. The pooling layer is composed of a maximum pooling function and an FMS module.

[0067] After obtaining the first output feature, the second output feature and the third output feature, they are all input into the convolution layer to extract feature information (such as time domain feature information and frequency domain feature information, etc.) to obtain the audio feature data; it should be noted that the first output feature, the second output feature and the third output feature are all data containing feature information obtained after feature extraction of the preprocessed waveform data. The difference is that different feature modules have different parameters set, such as the kernel and stride mentioned above, and the corresponding extracted feature information is also different; in an embodiment of the present invention, the convolution layer can use different convolution kernel sizes and numbers to capture features of different scales; the embodiment of the present invention is based on the structure of the residual network, and the output of the convolution layer is residually connected with the input to transfer gradients and information, thereby alleviating the problem of gradient disappearance and information loss.

[0068] It should be noted that the embodiment of the present invention processes the result after residual connection by adding a nonlinear activation function (ReLU) in the signal processing layer, and outputs the audio feature data obtained by extracting various feature results.

[0069] S24, performing weighted processing on the audio feature data through the attention layer to obtain weighted feature data;

[0070] In an embodiment of the present invention, the attention layer enhances the model's attention to important parts of the audio feature data by introducing an attention mechanism, thereby more focused screening out important frames related to the audio feature data and ignoring irrelevant audio frames, which is conducive to subsequent better classification of the audio feature data; for example, the attention parameters of the audio feature data can be determined by using a sigmoid nonlinear activation function, and the attention parameters can be used as weighted weights to perform weighted processing according to the audio feature data and the attention parameters to obtain a weighted feature representation, i.e., the weighted feature data.

[0071] S25, modeling the weighted feature data based on the temporal relationship through the recurrent neural network layer to obtain enhanced features;

[0072] In an embodiment of the present invention, the recurrent neural network layer models the temporal relationship of the weighted feature data based on a gated convolutional network CNN (Convolutional Neural Networks) to obtain the enhanced feature; the gated convolutional network CNN can capture the temporal correlation in the audio signal, greatly enhancing the temporal modeling capability of the feature.

[0073] S26. Mapping the enhanced features to an output space through the fully connected layer, and outputting a predicted label corresponding to the speech audio data.

[0074] Specifically, mapping the enhanced features to the output space through the fully connected layer and outputting the predicted labels corresponding to the speech audio data include: mapping the enhanced features to the output space; in the output space, using an activation function to make predictions based on the enhanced features to generate probability distribution results, and determining and outputting the predicted labels corresponding to the speech audio data based on the probability distribution results.

[0075] The embodiment of the present invention can select the output layer according to actual needs. For example, in the embodiment of the present invention, based on the classification training framework, the classification cross entropy objective function is adopted to calculate the loss by comparing the classification head applying soft maximum nonlinearity with the single-hot ground truth label.

[0076] For example, the embodiment of the present invention uses AAM-softmax as the objective function, as shown in the following formula:

[0077]

[0078] Define x separately i ,y i, W are speaker embedding (or embedding code), its corresponding predicted label, and the weight matrix of the classification head, the classification head is the fully connected layer, and each line of the weight matrix W is the center vector (center point) of the speaker in the enhanced feature; where cos(θy i , i) is the L2 norm normalized x i The dot product between and Wj, s and m are a scale factor and a marginal hyperparameter respectively, i is the index of a speaker within a mini-batch size N, and 0 <i<N。

[0079] An activation function is then used to make predictions based on the enhanced features to generate a probability distribution result, where the probability distribution result indicates whether the speech audio data is a real human voice or a false human voice. Based on the probability distribution result, a prediction label corresponding to the speech audio data is determined and output.

[0080] S3, determining the loss value of the preset classification model according to the anti-counterfeiting label corresponding to the voice audio data obtained in advance and the predicted label output by the preset classification model;

[0081] The embodiment of the present invention can calculate and determine the loss value of the preset classification model through various loss functions, for example, a cross entropy loss function, a hinge loss function, an exponential loss function, etc. can be used.

[0082] S4. Adjust parameters of the preset classification model according to the loss value to obtain a speaker verification model.

[0083] In an embodiment of the present invention, based on the determined loss value of the preset classification model, the parameters of each level in the preset classification model are adjusted accordingly to obtain a speaker verification model. The parameters may be the levels themselves, or the kernel, stride, learning rate or network function corresponding to each level, etc., which may be selected and adjusted according to actual circumstances. For example, in an embodiment of the present invention, after the classification training is completed, the classification head (such as the fully connected layer) may be removed to obtain the speaker verification model.

[0084] Compared with the prior art, the speaker verification model training method described in the embodiment of the present invention has the following advantages:

[0085] 1. The waveform data of speech audio data is used as the input of the preset classification model, and the scale of the original waveform data is stabilized through pre-emphasis technology, reducing the loss of data information;

[0086] 2. A new model architecture is constructed, including a pre-emphasis layer, a signal processing layer, an attention layer, a recurrent neural network layer and a fully connected layer, and the signal processing layer is a residual network composed of a first extended feature map module, a second extended feature map module, a third extended feature map module and a convolutional layer, so that when extracting features from waveform data, the extracted features are more reliable, thereby improving the accuracy of the entire model training process;

[0087] 3. The model is trained by supervised learning, and the real anti-counterfeiting labels are used to assist in judging the predicted labels of the model to determine the loss value of the preset classification model. At the same time, an end-to-end model structure is used to adjust the parameters of the preset classification model according to the loss value. The entire model is trained by minimizing the error between the output and the real label, rather than adjusting the parameters of the entire model by gradually training different components. The training of the speaker recognition verification model is realized, the training efficiency is improved, and the cost is reduced. The speaker recognition verification model obtained by training effectively overcomes the problem of data information loss during speech feature extraction in the prior art.

[0088] Based on the above embodiment, as the above Figure 1 The present invention provides an embodiment of a speaker verification model training device, and the device embodiment is similar to Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices, see Figure 5 As shown, the speaker verification model training device includes:

[0089] The waveform extraction module 100 is used to extract waveform data from the acquired speech audio data to obtain audio waveform data corresponding to the speech audio data;

[0090] A classification prediction module 200, used for inputting the audio waveform data into a preset classification model to output a prediction label corresponding to the speech audio data through the preset classification model, wherein the preset classification model includes a signal processing layer of a residual network composed of a plurality of feature modules, and the strides of the plurality of feature modules in the residual network are different;

[0091] The loss calculation module 300 is used to determine the loss value of the preset classification model according to the anti-counterfeiting label corresponding to the voice audio data obtained in advance and the predicted label output by the preset classification model;

[0092] The output module 400 is used to adjust the parameters of the preset classification model according to the loss value to obtain a speaker verification model.

[0093] The speaker verification model training device described in the embodiment of the present invention can execute the speaker verification model training method provided in the above embodiment. The speaker verification model training device has the corresponding functional steps and beneficial effects of the speaker verification model training method described in the above embodiment. Please refer to the embodiment of the above speaker verification model training method for details. The embodiment of the present invention will not be repeated here.

[0094] An embodiment of the present invention further provides an electronic device, which may include a processor and a memory, wherein the processor and the memory may be connected via a bus or other means. The processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components and other chips, or a combination of the above-mentioned various types of chips. The memory, as a non-transient computer-readable storage medium, may be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the speaker verification model training method in the embodiment of the present invention. The processor executes various functional applications and data processing of the processor by running the non-transient software programs, instructions and modules stored in the memory, that is, the speaker verification model training method in the above method embodiment is implemented.

[0095] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created by the processor, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. The one or more modules are stored in the memory, and when executed by the processor, the speaker verification model training method in the above method embodiment is executed. The specific details of the above electronic device can be understood by corresponding to the corresponding descriptions and effects in the above method embodiment, and will not be repeated here. Those skilled in the art can understand that all or part of the processes in the above embodiment method can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and the program, when executed, may include the processes of the embodiments of the above methods. Among them, the storage medium can be a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memory.

[0096] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0097] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this disclosed method should not be interpreted as reflecting the intention that the claimed invention requires more features than those expressly recited in each claim. Rather, as reflected in the claims, inventive aspects lie in less than all of the features of the individual embodiments previously disclosed. Therefore, the claims that follow the detailed description are hereby expressly incorporated into the detailed description, with each claim itself serving as a separate embodiment of the present invention.

[0098] It should be noted that the above-mentioned embodiments illustrate rather than limit the invention and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims.

Claims

1. A speaker verification model training method, characterized in that: The method comprises: Extracting waveform data from the acquired speech and audio data to obtain audio waveform data corresponding to the speech and audio data; Inputting the audio waveform data into a preset classification model to output a prediction label corresponding to the speech audio data through the preset classification model, wherein the preset classification model includes a signal processing layer of a residual network composed of multiple feature modules, and the strides of the multiple feature modules in the residual network are different; Determine the loss value of the preset classification model according to the anti-counterfeiting label corresponding to the voice audio data obtained in advance and the predicted label output by the preset classification model; The parameters of the preset classification model are adjusted according to the loss value to obtain a speaker verification model.

2. The speaker verification model training method according to claim 1, characterized in that: Before extracting waveform data from the acquired voice audio data, the method further includes: Extracting acoustic features and spectral features corresponding to the original audio data of the speech; Converting the acoustic feature and the spectral feature to obtain an acoustic feature vector corresponding to the acoustic feature and a spectral feature vector corresponding to the spectral feature; Obtain a classifier, input the acoustic feature vector and the spectral feature vector into the classifier, and obtain the acoustic feature vector with the voice tag added and the spectral feature vector with the voice tag added; Parsing the acoustic feature vector with the added voice tag and the spectral feature vector with the added voice tag to obtain an original voice signal; The starting point and the ending point corresponding to the original voice signal are determined according to the time sequence of the original voice signal to obtain the voice audio data.

3. The speaker verification model training method according to claim 1, characterized in that: The step of extracting waveform data from the acquired speech audio data to obtain audio waveform data corresponding to the speech audio data includes: Constructing waveform extraction model; The waveform extraction model is used to extract the time domain features of the speech audio data to obtain the audio waveform data.

4. The speaker verification model training method according to claim 1, characterized in that: The preset classification model includes a pre-emphasis layer, a signal processing layer, an attention layer, a recurrent neural network layer, and a fully connected layer; the step of inputting the audio waveform data into the preset classification model to output a predicted label corresponding to the speech audio data through the preset classification model includes: Inputting the audio waveform data into the preset classification model, performing pre-emphasis processing on the input audio waveform data through the pre-emphasis layer to obtain pre-processed waveform data; Performing signal transformation processing on the preprocessed waveform data through the signal processing layer, and extracting feature information to obtain audio feature data; Performing weighted processing on the audio feature data through the attention layer to obtain weighted feature data; Modeling the weighted feature data based on the temporal relationship through the recurrent neural network layer to obtain enhanced features; The enhanced features are mapped to the output space through the fully connected layer, and the predicted labels corresponding to the speech audio data are output.

5. The speaker verification model training method according to claim 4, characterized in that: The signal processing layer includes four feature modules, namely a first extended feature map module, a second extended feature map module, a third extended feature map module and a convolutional layer. The first extended feature map module, the second extended feature map module, the third extended feature map module and the convolutional layer constitute a residual network. The first extended feature map module, the second extended feature map module and the third extended feature map module have the same structure, wherein the stride set by the first extended feature map module is different from that of the second extended feature map module, and the stride set by the first extended feature map module and the second extended feature map module is greater than the stride set by the third extended feature map module and the convolutional layer.

6. The speaker verification model training method according to claim 5, characterized in that: The performing signal transformation processing on the preprocessed waveform data through the signal processing layer and extracting feature information to obtain audio feature data comprises: Processing the preprocessed waveform data by the first extended feature map module to obtain a first output feature; Processing the first output feature by the second extended feature map module to obtain a second output feature; Processing the first output feature and the second output feature by the third extended feature map module to obtain a third output feature; The first output feature, the second output feature and the third output feature are input into the convolution layer, and feature information is extracted to obtain the audio feature data.

7. The speaker verification model training method according to claim 4, characterized in that: Mapping the enhanced features to the output space through the fully connected layer and outputting the predicted label corresponding to the speech audio data includes: Mapping the enhanced features to an output space; In the output space, an activation function is used to perform prediction based on the enhanced features to generate a probability distribution result, and a prediction label corresponding to the speech audio data is determined and output based on the probability distribution result.

8. A speaker verification model training device, applied to the speaker verification model training method according to any one of claims 1 to 7, characterized in that: The device comprises: A waveform extraction module is used to extract waveform data from the acquired speech audio data to obtain audio waveform data corresponding to the speech audio data; A classification prediction module, used for inputting the audio waveform data into a preset classification model to output a prediction label corresponding to the speech audio data through the preset classification model, wherein the preset classification model includes a signal processing layer of a residual network composed of multiple feature modules, and the strides of the multiple feature modules in the residual network are different; A loss calculation module, used to determine the loss value of the preset classification model according to the anti-counterfeiting label corresponding to the voice audio data obtained in advance and the predicted label output by the preset classification model; The output module is used to adjust the parameters of the preset classification model according to the loss value to obtain a speaker verification model.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which are loaded and executed by a processor to implement the operations performed by the method according to any one of claims 1 to 7.

10. An electronic device comprising a processor and a memory, characterized in that: The memory stores computer program instructions that can be executed by the processor, and when the processor executes the computer program instructions, the processor implements instructions of any one of the methods described in claims 1-7.

Images