A time sequence voiceprint feature combination recognition method and device

Abstract

The present application relates to a kind of time sequence voiceprint feature combination identification method and device.The method includes, the speech feature of audio signal is acquired, the speech feature is input to voiceprint recognition network model after the speech feature is enhanced processing, voiceprint recognition network model is trained, until the voiceprint recognition network model converges.The present application is enhanced by the change condition of speech feature in the time dimension in audio signal, so that model can more accurately identify the transition voiceprint feature state of continuity in a period of time, by the compression of multiple time sequence features, improve the correlation of algorithm model in speech and time sequence signal processing, can accurately identify the audio signal of the same person at different time.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of speech recognition, and in particular to a time sequence voiceprint feature combination recognition method and device. BACKGROUND

[0002] With the rapid development of machine learning, voiceprint recognition technology has great potential and urgent needs in financial anti-fraud and other fields. Deep learning adopts a multi-layer neural layer structure, extracts information from the input layer raw data through layer-by-layer transmission, and automatically mines complex representation features in large data sets, avoiding manual feature engineering. Deep neural networks, stacked denoising autoencoders and long short-term memory networks have been applied, and have good dynamic learning effect compared with traditional machine learning methods. Convolutional neural networks, as a kind of distributed parallel deep network model, have also been widely used in fault diagnosis.

[0003] Compared with other deep learning methods, time delay neural network (TDNN) is equivalent to one-dimensional CNN. TDNN was first proposed by Hinton in 1989, aiming to solve the recognition of audio. However, this structure only considers local context information, while ECAPA-TDNN considers global information, so it is more suitable for extracting voiceprint features in a period of time. However, the existing scheme does not focus on the difference in continuous change of audio, which leads to the fact that the existing model is not good enough in terms of accuracy and stability of voiceprint recognition, and there is still a lot of room for improvement. SUMMARY

[0004] The inventors of the present application realized in research that the difference features of continuous audio are very helpful to sequence models, so the difference features of audio signal time sequence are considered in the technical scheme of the present application. The inventors of the present application realized in research that the vocal organs of human body exist a continuous change process when they are making sound, which leads to the existence of continuous time sequence change in a continuous human voice signal, and the audio signals used for training algorithm in the existing algorithm do not distinguish the difference features in the continuous change in detail, which leads to the fact that the existing model still needs to be improved in the distinction degree of the voice signals of the same person at different times.

[0005] In order to solve the problem of insufficient accuracy and stability of the existing voiceprint recognition technology, the present application provides a time sequence voiceprint feature combination recognition method and device which distinguishes the voice of the same person at different times.

[0006] The technical scheme is as follows:

[0007] S1, obtaining the voice features of the audio signal;

[0008] S2, the speech feature is enhanced, and an enhanced speech feature C is obtained,

[0009] C=C t+1 +W·(C t+1 -C t );

[0010] Wherein, C t is the speech feature at t moment, W is the weight value of continuous change of speech feature, C t+1 is the speech feature at t+1 moment, the enhanced speech feature includes initial speech feature parameters and change speech parameters;

[0011] S3, the enhanced speech feature is input to the voiceprint recognition network model, the voiceprint recognition network model is trained until the voiceprint recognition network model converges.

[0012] The application can be further improved on the basis of the above scheme.

[0013] Further, the S1 further comprises: after selecting the enhanced speech feature to a fixed length, input to the voiceprint recognition network model.

[0014] Further, before the S1, it further comprises: carrying out noise reduction processing and dereverberation processing to the audio signal.

[0015] Further, the voiceprint recognition network model comprises: an input layer, a feature extraction layer and an output layer, the input layer is a time delay neural network structure, which is used for pre-input processing of the enhanced speech feature;

[0016] The feature extraction layer comprises an SE residual network layer and a feature fusion layer, after the input speech feature parameters are pre-input processed by the input layer, the SE residual network layer is used for feature dimension reduction and dimension recovery processing, and a speech feature vector is obtained, and the speech feature vector is calculated through the feature fusion layer to obtain a voiceprint feature vector;

[0017] The output layer comprises an attention statistical pooling layer and an AAM-Softmax layer, the voiceprint feature vector is pooled through the attention statistical pooling layer to obtain a voiceprint classification probability, and the AAM-Softmax layer is used for classifying the audio signal according to the voiceprint classification probability.

[0018] Further, the SE residual network layer comprises a plurality of SE residual network structures (SE-Res2Block),

[0019] When the initial speech feature parameters in the speech features are input to the SE-Res2Block, all the speech feature parameters are updated respectively by 3*3 convolution kernel operation, and the changed speech feature parameters are obtained;

[0020] When the changed speech feature parameters are input to the SE-Res2Block, the changed speech feature parameters and the initial speech feature parameters corresponding thereto are subjected to 3*3 convolution kernel operation, and after the changed speech feature parameters are updated, the speech feature vectors of the current SE-Res2Block are obtained and input to other SE-Res2Blocks, until the changed speech feature parameters and all the initial speech feature parameters are subjected to 3*3 convolution kernel operation, and the speech feature vectors of the multiple SE-Res2Blocks are subjected to multi-layer feature calculation by the feature fusion layer to obtain the voiceprint feature vector.

[0021] Further, the S4 is used to train the voiceprint recognition network model until the voiceprint recognition network model converges, specifically: the enhanced speech features are used for multiple iteration training of the voiceprint recognition network model, AAM-Softmax calculation is performed according to the voiceprint feature state quantity or the voiceprint feature state mean, and when the loss value is less than a preset threshold, it is judged that the voiceprint recognition network model of the preliminary training has converged.

[0022] Another object of the present application is to further provide a time sequence feature combination recognition device, comprising a memory and a processor, the memory stores at least one program, and the at least one program is executed by the processor to realize the time sequence voiceprint feature combination recognition method as described above.

[0023] Still another object of the present application is to further provide a computer readable storage medium, comprising a memory, the memory stores a computer program, and the computer program is executed by a processor to realize the time sequence voiceprint feature combination recognition method as described above.

[0024] The present application has the beneficial effect that the present application considers the voiceprint feature change in the time dimension, through the enhancement processing of the speech feature change in the time dimension in the audio signal, the model can more accurately identify the continuous change voiceprint feature state in a period of time, through the compression of the multi-time sequence feature, the algorithm model is associated in the signal processing of the speech and the time sequence, and the same person can be accurately identified in the continuous change audio signal at different time. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 The flowchart of the time sequence voiceprint feature combination recognition method of the present application;

[0026] Figure 2A network back propagation algorithm flow chart;

[0027] Figure 3 A voiceprint recognition network model structure schematic diagram;

[0028] Figure 4 A multi-layer residual network structure schematic diagram. DETAILED DESCRIPTION

[0029] The principles and characteristics of the present application are described below in conjunction with the accompanying drawings, and the examples are only used to explain the present application and are not used to limit the scope of the present application.

[0030] As Figure 1 shown, the present application provides a time sequence voiceprint feature combination recognition method and device, comprising the following steps,

[0031] S1, obtaining speech features of an audio signal;

[0032] S2, performing enhancement processing on the speech features to obtain enhanced speech features C,

[0033] C=C t+1 +W·(C t+1 -C t );

[0034] Wherein, C t is the speech feature at time t, W is the weight value of the continuous change of the speech feature, C t+1 is the speech feature at time t+1, the enhanced speech feature includes initial speech feature parameters and change speech parameters; wherein C t+1 -C t shows the speech feature change from time t to time t+1, and W reflects the importance of the speech feature change;

[0035] S3, inputting the enhanced speech features into a voiceprint recognition network model, training the voiceprint recognition network model until the voiceprint recognition network model converges.

[0036] The present application enhances the change of the speech features in the time dimension in the audio signal, inputs the speech features that have been enhanced in the time dimension into the voiceprint recognition model, so that the model can more accurately recognize the continuous change of the voiceprint feature state in a period of time, and through the compression of the multi-time sequence features, the algorithm model is associated in the signal processing of the speech and time sequence, and the same person can be accurately recognized in the continuous change of the audio signal at different times.

[0037] In this embodiment, the acquired audio signal can be processed by using an existing voice feature extraction method to extract voice features of the audio signal. Preferably, when the audio signal is extracted into voice features, the audio signal can be extracted after being subjected to noise reduction processing and dereverberation processing, so as to avoid the instability of the audio signal caused by force majeure factors and improve the quality of the audio signal. The voice features in the enhanced voice features are selected to have a fixed length and then input into the voiceprint recognition network model, and the fixed length is T. In this application, the MFCC parameters in the voice feature parameters are used to extract data from the audio signal to obtain a voice feature set. After the audio signal is acquired, the audio signal is extracted into voice features, and the audio signal is sequentially subjected to pre-emphasis processing, frame processing, window processing, fast Fourier transform processing, triangular band-pass filter, logarithmic operation and dynamic difference parameter operation to obtain the voice feature sequence set.

[0038] In this embodiment, the voiceprint recognition network model is iteratively trained by using the enhanced voice features, AAM-Softmax calculation is performed according to the number of voiceprint feature states or the average of the voiceprint feature states, and when the loss value is less than a preset threshold, it is judged that the voiceprint recognition network model of the preliminary training has converged.

[0039] The effect evaluation and optimization target of the network model make a prediction on the mapping of the network features of the full connection layer to the label space of the sample, and then compare it with the true value to measure the distance between the prediction value and the true sample label, and guide the network parameter learning and representation learning through error back propagation. For a multi-classification task, a cross-entropy loss function (Cross-entropy Loss Function) is generally selected, which is defined as the cross-entropy between the true value and the prediction value. The cross-entropy represents the distance between the actual output and the expected output, and the smaller the cross-entropy value, the closer the two are. For a given training set After all the samples are input into the network, the network output is The target function on the data set A is:

[0040]

[0041] In the formula, y n is the vector representation of the label corresponding to the nth data pair, and the learning parameters are iteratively updated and optimized by minimizing the target function. The smaller the Loss, and the closer the y n to the true value, the closer the prediction value is to the true value.

[0042] For example Figure 2As shown, the back propagation algorithm realizes an iterative process. At the beginning of each iteration, a small batch of training data needs to be selected, which is called a batch. Then, the batch is used to get the prediction results from the input layer through the hidden layer to the output layer using the current network initialized weights and thresholds. Since the training data and the real labels are one-to-one corresponding, the distance between the current model prediction value and the true value can be calculated, and the prediction results and the real values are used to form a cost function, which is the forward propagation process. Through the chain rule, the derivative of the loss function with respect to each parameter can be calculated, and the partial derivative of the objective function with respect to the net input of the first layer of neurons is defined as the error term, which is an intermediate variable. Since the error term of a neuron in the first layer is the weight sum of the error terms of all neurons in the 1+1 layer connected to it, and then multiplied by the gradient of the neuron activation function. The error can be propagated from the output layer to the hidden layer until it propagates to the input layer. In the process of back propagation, the values of the parameters are adjusted according to the error, and the above process is iterated until the target converges and the loss function reaches the optimal value. Based on the gap between the prediction value and the true value, the back propagation algorithm updates the model parameters accordingly, so that the prediction results of the model and the real labels of this batch are closer. When the target function network structure is determined, automatic differentiation can be used to calculate the gradient of the objective function with respect to each parameter, and then an optimization algorithm is selected to get the optimal value of the objective function and iteratively update the optimization parameters based on the gradient optimization algorithm.

[0043] In this embodiment, specifically, as Figure 3The input layer, the feature extraction layer and the output layer are shown, the input layer is a time delay neural network structure, that is, a one-dimensional neural network convolution layer, which is used for pre-input processing of the enhanced speech features, and includes an activation function, which is mainly used for nonlinear mapping operation of the convolution layer output result, and currently mainly has ReLU, sigmoid, tanh and other functions, and the application takes ReLU (Rectified Linear Unit) as a relatively optimal embodiment, which has nonlinear and linear characteristics in a specific range of deep neural networks, and has a constant derivative value for the corresponding gradient of all inputs greater than 0, so that the information integration capability is greatly enhanced, and the network training is accelerated and simplified. In addition, ReLU can effectively reduce the gradient vanishing and overfitting problems of the CNN model; the feature extraction layer includes an SE residual network layer and a feature fusion layer, wherein the SE residual network layer includes a plurality of SE residual network structures, and the SE residual network structure mainly includes Squeeze (compression) and Excitation (excitation) two parts, wherein the enhanced speech features are pre-input processed by the input layer, and then the input speech features are subjected to feature dimension reduction by the Squeeze (compression) part in the SE residual network layer, specifically, for example, W, H represents the feature width, height. C represents the number of channels, the input feature size is WxHxC, and the global average pooling (global average pooling) in the compression part of the SE residual network layer is subjected to compression operation, and the feature is compressed into a 1x1xC vector. Then the compressed feature vector is obtained by a Sigmoid (activation function) gate between 0 and 1 normalized weight, and finally a Scale (back to the original size) operation is performed to weight the normalized weight to each channel feature to restore the dimension, and then the speech feature vector is obtained, so that the feature has more nonlinearity, and the channel correlation can be better fitted. At the same time, the parameter amount and the calculation amount are greatly reduced.

[0044] The SE residual network layer in the application includes a plurality of SE residual network structures. The following describes an embodiment in which three SE-Res2Block layers are used. The first SE-Res2Block layer outputs a speech feature vector C1, the second SE-Res2Block network structure outputs a speech feature vector C2, and the third SE-Res2Block outputs a speech feature vector C3. When the initial speech feature parameters in the speech feature are input to the SE-Res2Block, all the speech feature parameters are updated by 3x3 convolution kernel operation, and the changed speech feature parameters are obtained. When the changed speech feature parameters are input to the SE-Res2Block, the changed speech feature parameters are updated by 3x3 convolution kernel operation with the corresponding initial speech feature parameters. After the changed speech feature parameters are updated, the speech feature vector of the current SE-Res2Block is obtained and input to other SE residual network structures, until the changed speech feature parameters are updated by 3x3 convolution kernel operation with all the initial speech feature parameters. The speech feature vectors of the plurality of SE-Res2Block layers are input to the feature fusion layer to obtain a voiceprint feature vector by multi-layer feature calculation.

[0045] The application is described by the following preferred embodiments. In the embodiment, three SE-Res2Block layers are used. In the embodiment, two types of speech features are obtained, one is the original speech feature parameter, and the other is the changed speech feature parameter. The initial speech feature parameters (x1, x2, x3, x4) are subjected to 3x3 convolution operation to obtain the corresponding changed speech feature parameters (y1, y2, y3, y4), as shown in the following formula: Figure 4 y5 is obtained by adding x2 to y1 and then performing 3x3 operation, y6 is obtained by adding x3 to y5 and then performing 3x3 operation, and y7 is obtained by adding x4 to y6 and then performing 3x3 operation. Then, (y1, y2, y3, y4, y5, y6, y7) are spliced to output the speech feature vector C1. C1 is input to the second SE-Res2Block layer network to perform the above feature network processing, and the speech feature vector C2 is output. Similarly, C2 is input to the second SE-Res2Block layer network, and the speech feature vector C3 is output. The first SE-Res2Block layer outputs the speech feature vector C1, the second SE-Res2Block network structure outputs the speech feature vector C2, and the third SE-Res2Block outputs the speech feature vector C3. The speech feature vectors C1, C2, and C3 are input to the feature fusion layer to obtain a voiceprint feature vector by multi-layer feature calculation.

[0046] The voiceprint feature vector is pooled by the attention statistical pooling layer for a voiceprint classification probability, the output layer includes an output layer and a classification layer, and the AAM-Softmax layer is used for voiceprint classification of the audio signal according to the voiceprint classification probability, that is, for a classification problem, and an expression thereof is that an activation function is replaced by an AAM-Softmax function on the basis of a full connection layer. The previous neural network layer activity value is taken as an input layer of the AAM-Softmax layer, there are K neurons corresponding to K sample labels in the output of the AAM-Softmax layer, a class score is calculated through the AAM-Softmax layer, a normalized exponential function is used to represent a probability distribution of different categories to which a current batch of samples belongs, and an output with the maximum classification probability is taken as a final prediction.

[0047] The processing of data in the time-delay neural network structure, the attention statistical pooling layer and the output layer also includes a batch normalization (BN) operation. The BN is an effective layer-by-layer normalization method. During the forward propagation of the model, the parameter distribution of each layer of the model changes, which leads to a reduction in the training speed, and it is very difficult to train a model with saturated nonlinearity, so it is necessary to perform the batch normalization operation on the hidden layer of the neural network. The batch normalization operation transforms and scales the input batch data to stable mean and standard deviation during each training of the model, so that each dimension of the net input is normalized to the standard normal distribution, thereby improving the optimization efficiency without special attention to the initialization of the parameters. Although the batch normalization operation can also be applied to the activity value of the neuron, the batch normalization net input is more conducive to network optimization. Therefore, in practical applications, the BN is generally used after the affine transformation and before the activation function. The application of the BN layer can enhance the adaptability of the model to different distributions of the input, has a slight regularization effect, accelerates the model convergence speed and the training speed, effectively improves the model convergence rate, and to a certain extent, alleviates the "gradient dispersion" effect during the training of the deep network, and plays a certain role in improving the network generalization performance, thereby making it easier and more stable to train the deep network model. In addition, the BN can also be used to accelerate the training speed of the model and even improve the model precision.

[0048] Another object of the present application is to also provide a multi-state voiceprint feature recognition device, comprising a memory and a processor, the memory stores at least one program, and the at least one program is executed by the processor to realize the time sequence voiceprint feature combination recognition method described above.

[0049] Still another object of the present application is to also provide a computer readable storage medium, comprising a memory, the memory stores a computer program, and the computer program is executed by a processor to realize the time sequence voiceprint feature combination recognition method described above.

[0050] The reader's attention is directed to the fact that the description of the application is illustrative only and that changes can be made to the embodiments described, without departing from the scope of the application. Accordingly, the application is not limited to that precisely as shown and described.

[0051] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented in other ways. For example, the above-described method embodiments are merely illustrative, for example, the division of steps is only a logical function division, and actual implementation can have another division manner, for example, multiple steps can be combined or integrated into another step, or some features can be ignored or not executed.

[0052] The above merely provides a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements shall be encompassed in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for recognizing a temporal-prosodic feature combination, characterized in that: The method comprises the following steps of: S1, obtaining a voice feature of an audio signal; S2, performing enhancement processing on the voice feature to obtain an enhanced voice feature C, wherein ; Wherein, C t is the speech feature at time t, W is the weight value of continuous change of the speech feature, C t+1 is the speech feature at time t+1, the enhanced speech feature includes the initial speech feature parameter and the changed speech parameter; S3, inputting the enhanced voice feature into a voiceprint recognition network model, and training the voiceprint recognition network model until the voiceprint recognition network model converges; The voiceprint recognition network model comprises a feature extraction layer; The feature extraction layer comprises an SE residual network layer and a feature fusion layer. After the enhanced voice feature is pre-input processed by an input layer, the SE residual network layer is used to perform feature dimension reduction and dimension restoration processing on the input voice feature parameters to obtain a voice feature vector, and the voice feature vector is input into the feature fusion layer to obtain a voiceprint feature vector through multi-layer feature calculation. The voiceprint recognition network model comprises an input layer and an output layer. The input layer is a time-delay neural network structure, which is used for pre-input processing of the enhanced voice feature. The output layer comprises an attention statistical pooling layer and an AAM-Softmax layer. The voiceprint feature vector is pooled through the attention statistical pooling layer to obtain a voiceprint classification probability, and the AAM-Softmax layer is used to classify the audio signal according to the voiceprint classification probability. The SE residual network layer comprises a plurality of SE residual network structures, which are referred to as SE-Res2Block. When the initial voice feature parameters in the voice feature are input into the SE-Res2Block, all the initial voice feature parameters are updated through 3*3 convolution kernel operation to obtain changed voice feature parameters. When the changed voice feature parameters are input into the SE-Res2Block, the changed voice feature parameters are updated through 3*3 convolution kernel operation with the corresponding initial voice feature parameters. After the updated changed voice feature parameters are input into other SE-Res2Blocks, the voice feature vectors of the plurality of SE-Res2Blocks are obtained through multi-layer feature calculation by the feature fusion layer to obtain the voiceprint feature vector.

2. The method of claim 1, wherein the method further comprises: The S3 further comprises the following step of: inputting the enhanced voice feature with a fixed length into the voiceprint recognition network model.

3. The method of claim 1, wherein the method further comprises: Before the S1, the audio signal is subjected to noise reduction processing and dereverberation processing.

4. The method of claim 1, wherein the method further comprises: In the S3, the voiceprint recognition network model is trained until the voiceprint recognition network model converges. Specifically, the voiceprint recognition network model is iteratively trained with the enhanced voice feature, AAM-Softmax calculation is performed according to the number of voiceprint feature states or the mean value of the voiceprint feature states, and when the loss value is less than a preset threshold, it is determined that the voiceprint recognition network model has been preliminarily trained and converged.

5. A temporal iatent feature combination recognition apparatus, characterized by, The computer program product comprises a memory and a processor, the memory stores at least one program, and the at least one program is executed by the processor to implement the time sequence voiceprint feature combination recognition method according to any one of claims 1 to 4.

6. A computer-readable storage medium, characterized in that: The computer program product comprises a memory, and the memory stores a computer program, and the computer program is executed by a processor to implement the time sequence voiceprint feature combination recognition method according to any one of claims 1 to 4.

Images