Voiceprint recognition method and system combining variational autoencoder and Markov random field

Abstract

The present invention discloses a voiceprint recognition method and system combining variational autoencoder and Markov random field, belonging to the technical field of voiceprint recognition, S1, using an artificial neural network model combining variational autoencoder and Markov random field to encode audio sampling data to obtain an initial probability mean vector and an initial probability variance vector; S2, for each initial probability mean vector and initial probability variance vector, k transfer probability operations are performed to obtain k transfer probabilities; the steps of each transfer probability operation are: S201, resampling the transfer probability feature of the initial probability mean vector and the initial probability variance vector to obtain the transfer probability; S202, encoding the transfer probability to obtain the transfer probability mean vector and the transfer probability variance vector; S3, using the k transfer probabilities to perform voiceprint recognition. The present invention can achieve efficient voiceprint recognition by combining the variational autoencoder and the artificial neural network model of Markov random field.

Description

Technical Field

[0001] The present invention belongs to the technical field of voiceprint recognition, and in particular relates to a voiceprint recognition method and system combining variational autoencoding and Markov random fields. Background Art

[0002] Voiceprint recognition technology is a recognition method based on biometrics, also known as speaker recognition technology. It is mainly divided into two sub-areas: speaker identification and speaker confirmation. The core process of voiceprint recognition is to convert sound signals into electrical signals, and then use computers to analyze and recognize them. Different voiceprint recognition technologies are used in different application scenarios and tasks. At present, in the process of voiceprint recognition, the information related to the speaker in a piece of speech is usually driven by a set of implicit features that are difficult to explain and initially changeable. After multiple rounds of random process iterations, these implicit features eventually form the speaker's unique voice characteristics.

[0003] When extracting latent variables, traditional voiceprint recognition methods often cannot determine which round of random processes these latent variables come from. Therefore, in order to improve the accuracy of recognition, researchers need to collect a large amount of sample data. However, even so, due to the difficulty of distinguishing latent variables, voiceprint recognition systems are still prone to errors and omissions. In other words, after collecting the speaker's voice data, the latent variables used to distinguish different speakers are still extremely difficult to accurately identify and distinguish, which greatly limits the accuracy and reliability of voiceprint recognition technology. Summary of the invention

[0004] The present invention provides a voiceprint recognition method and system combining variational autoencoder and Markov random field, which are used to distinguish implicit variables of a speaker.

[0005] To achieve the above-mentioned object of the invention, the first object of the present invention is to provide a voiceprint recognition method combining variational autoencoder and Markov random field, comprising:

[0006] S1. Encode the audio sampling data using an artificial neural network model combining variational autoencoder and Markov random field to obtain an initial probability mean vector and an initial probability variance vector;

[0007] S2. For each initial probability mean vector and initial probability variance vector, perform k transfer probability operations to obtain k transfer probabilities, where k is a natural number greater than 9; the steps of each transfer probability operation are:

[0008] S201, resampling the initial probability mean vector and the initial probability variance vector to obtain a transition probability;

[0009] S202, encoding the transition probability to obtain a transition probability mean vector and a transition probability variance vector;

[0010] S3. Using k transition probabilities to perform voiceprint recognition; specifically including:

[0011] S301, quantizing k transition probabilities one by one to obtain a transition probability matrix;

[0012] S302, using a convolutional neural network to perform a convolution operation on the transfer probability matrix, and output a convolution vector;

[0013] S303, using a multi-layer perceptron to process the convolution vector to obtain c perception outputs, where c is the number of voiceprint categories defined by the user;

[0014] S304, using the Softmax algorithm to process the c perception outputs to obtain the probability of the audio sampling data in each category, and selecting the category with the largest probability, which is the result of voiceprint recognition.

[0015] Preferably, the formula of the Softmax algorithm is:

[0016] ;

[0017] in, is the output of the multilayer perceptron, including c components, refers to its i-th component.

[0018] Preferably, the artificial neural network model includes:

[0019] An audio data encoder encodes the audio sampling data to obtain an initial probability mean vector and an initial probability variance vector;

[0020] The transition probability loop generation module performs k transition probability operations for each initial probability mean vector and initial probability variance vector to obtain k transition probabilities;

[0021] The transition probability encoder encodes each transition probability to obtain a transition probability mean vector and a transition probability variance vector;

[0022] The decoder performs implicit feature resampling and decoding on the transition probability mean vector and the transition probability variance vector to obtain the generated audio data.

[0023] Preferably, the training process of the artificial neural network model includes:

[0024] Prepare a data set; first, regularize the collected audio waveform data to obtain audio waveform data of uniform size; then, label the audio waveform data by category;

[0025] Design loss function; specifically include:

[0026] Calculate the variational autoencoder loss L1;

[0027] (1)

[0028] in, refers to the mean of the transition probabilities generated for the kth time, is the mean square error of the k-th transition probability, x is the input audio sampling data, is the generated audio data, n is the number of data in the dataset, and λ is the user-defined fine-tuning parameter of the loss term;

[0029] Calculate the classification loss L2;

[0030] (2)

[0031] in, is the one-hot encoding of the category to which the audio sample data x belongs, is the i-th component, is the output of the multilayer perceptron, including c components, refers to its i-th component;

[0032] Combined loss L;

[0033] (3)

[0034] Input the data in the data set into the artificial neural network model one by one, use the Adam optimization algorithm, run several epochs, take out the artificial neural network weight parameters, and complete the training process.

[0035] The second object of the present invention is to provide a voiceprint recognition system combining variational autoencoder and Markov random field, comprising:

[0036] An artificial neural network model combining variational autoencoder and Markov random field; encoding audio sampling data to obtain an initial probability mean vector and an initial probability variance vector; performing k transfer probability operations for each initial probability mean vector and initial probability variance vector to obtain k transfer probabilities, where k is a natural number greater than 9; the steps of each transfer probability operation are: firstly resampling the transfer probability features of the initial probability mean vector and the initial probability variance vector to obtain the transfer probability; then encoding the transfer probability to obtain a transfer probability mean vector and a transfer probability variance vector;

[0037] A voiceprint recognition model that uses k transition probabilities for voiceprint recognition; the specific recognition process includes: first, quantizing the k transition probabilities one by one to obtain a transition probability matrix; then using a convolutional neural network to perform a convolution operation on the transition probability matrix and output a convolution vector; then using a multi-layer perceptron to process the convolution vector to obtain c perception outputs, where c is the number of voiceprint categories defined by the user; finally, using the Softmax algorithm to process the c perception outputs to obtain the probability of audio sampling data in each category, and selecting the category with the largest probability, which is the result of voiceprint recognition.

[0038] Preferably, the formula of the Softmax algorithm is:

[0039] ;

[0040] in, is the output of the multilayer perceptron, including c components, refers to its i-th component.

[0041] Preferably, the artificial neural network model includes: an audio data encoder, a transition probability loop generation module, a transition probability encoder and a decoder.

[0042] Preferably, the training process of the artificial neural network model includes:

[0043] Prepare a data set; first, regularize the collected audio waveform data to obtain audio waveform data of uniform size; then, label the audio waveform data by category;

[0044] Design loss function; specifically include:

[0045] Calculate the variational autoencoder loss L1;

[0046] (1)

[0047] in, refers to the mean of the transition probabilities generated for the kth time, is the mean square error of the k-th transition probability, x is the input audio sampling data, is the generated audio data, n is the number of data in the dataset, and λ is the user-defined fine-tuning parameter of the loss term;

[0048] Calculate the classification loss L2;

[0049] (2)

[0050] in, is the one-hot encoding of the category to which the audio sample data x belongs, is the i-th component, is the output of the multilayer perceptron, including c components, refers to its i-th component;

[0051] Combined loss L;

[0052] (3)

[0053] Input the data in the data set into the artificial neural network model one by one, use the Adam optimization algorithm, run several epochs, take out the artificial neural network weight parameters, and complete the training process.

[0054] The third object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned voiceprint recognition method combining variational autoencoder and Markov random field.

[0055] A fourth object of the present invention is to provide a computer program product, comprising a computer program, which, when executed by a processor, implements the above-mentioned voiceprint recognition method combining variational autoencoding and Markov random field.

[0056] The advantages and positive effects of this application are:

[0057] The present invention realizes efficient voiceprint recognition by combining the artificial neural network model of variational autoencoder (VAE) and Markov random field (MRF). Specifically, the technical solution first uses the variational autoencoder to extract the initial latent variables, which can capture the key features in the sound signal. Subsequently, these latent variables are used to drive a Markov process, which further refines the representation of sound features. In this way, the model can effectively capture the dynamic changes of the sound signal and convert it into a feature representation that can be used for classification.

[0058] The advantage of this technical solution is that it can achieve a high voiceprint recognition accuracy even with a small number of samples. Traditional voiceprint recognition methods usually require a large number of training samples to achieve a high recognition rate. The technical solution of the present invention combines the advantages of VAE and MRF to extract discriminative features under limited sample conditions, thereby significantly improving recognition accuracy. This innovative model not only improves the efficiency of voiceprint recognition, but also reduces the dependence on large-scale data sets, making voiceprint recognition technology more feasible and flexible in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0060] Figure 1 A flowchart of a preferred embodiment of the present invention;

[0061] Figure 2 This is a flow chart of S2 in a preferred embodiment of the present invention;

[0062] Figure 3 This is a flow chart of S3 in a preferred embodiment of the present invention;

[0063] Figure 4 A structural diagram of an artificial neural network in a preferred embodiment of the present invention;

[0064] Figure 5 It is a reasoning flow chart of an artificial neural network in a preferred embodiment of the present invention;

[0065] Figure 6 It is a complete data processing flow chart in the preferred embodiment of the present invention. DETAILED DESCRIPTION

[0066] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0067] Variational Auto-Encoders (VAE) are a type of deep generative model proposed by Kingma et al. in 2014. They are based on a generative network architecture inferred by Variational Bayes (VB). The basic architecture of VAE follows the design of traditional autoencoders, mainly including two components: encoder and decoder. The encoder is responsible for mapping the input data to latent variables in a low-dimensional space, while the decoder is responsible for extracting samples from the latent variables to generate a specified image. However, the core difference between VAE and traditional autoencoders is that the hidden variables in VAE are random variables, and VAE describes the observation of the latent space in a probabilistic way.

[0068] The core concept of VAE is to map the original data to a known distribution (such as Gaussian distribution), and then randomly sample from the known distribution to obtain data close to the original data through the generative model. This is essentially fitting the data distribution, and then sampling from the distribution to generate new data. To achieve this goal, VAE uses two neural networks to build two probability density distribution models: one is used for variational inference of the original input data to generate the variational probability distribution of the latent variables, called the inference network; the other is based on the generated variational probability distribution of the latent variables, and restores the approximate probability distribution of the original data, called the generation network.

[0069] In VAE, the encoder outputs no longer a specific encoding value, but the mean and mean square error of the latent variable, which are used to describe the Gaussian distribution followed by the latent variable. In order to sample from the distribution of the latent variable, VAE uses the reparameterization technique, that is, sampling a random variable from a standard Gaussian distribution, and then transforming it through the mean and mean square error output by the encoder to obtain samples of the latent variable.

[0070] The decoder receives samples sampled from the latent variable distribution and tries to reconstruct the original input data. By continuously optimizing the parameters of the encoder and decoder, VAE can learn the mapping relationship from original data to latent variables and then to reconstructed data. VAE has shown a wide range of application value in image generation, data augmentation, anomaly detection and other fields. For example, in the field of image generation, VAE can generate new images that are similar to the original image but not exactly the same; in the field of data augmentation, VAE can generate new sample data by changing the feature values ​​in the latent space; in the field of anomaly detection, VAE can use reconstruction errors to detect abnormal data.

[0071] Markov Random Field (MRF) is a key probabilistic graphical model used to characterize the dependencies between random variables. MRF is represented by an undirected graph consisting of several random variables, where nodes represent random variables and edges represent interactions or dependencies between them. Each node has a state or value, and the state of each node is only affected by the states of its adjacent nodes, but not by other nodes. MRF has the Markov property, that is, given the value of a node, the values ​​of the node and its adjacent nodes are conditionally independent. Specifically, if the value of a node is given in a certain MRF, the state of the node is only related to the states of its adjacent nodes, but not to the states of other nodes in the graph.

[0072] MRF is usually composed of nodes and potential functions. Nodes represent random variables, while potential functions are used to describe the dependencies between nodes. The joint probability distribution of MRF can be expressed as the product of local potential functions (or local factors). MRF has a wide range of applications in natural language processing, computer vision, bioinformatics and other fields. For example, in the field of image processing, MRF can be used for tasks such as image segmentation, denoising and texture synthesis; in the field of computer vision, MRF can be applied to problems such as target recognition, face recognition and scene understanding. In addition, MRF also supports modeling complex nonlinear relationships between variables and can handle problems such as missing data and label noise. It has shown a wide range of applications in many fields and is a probabilistic calculation tool for solving complex problems.

[0073] See also Figures 1 to 6 ; The first embodiment, a voiceprint recognition method combining variational autoencoder and Markov random field, comprises the following steps:

[0074] S1. Encode the audio sampling data using an artificial neural network model combining variational autoencoder and Markov random field to obtain an initial probability mean vector and an initial probability variance vector;

[0075] By adopting an artificial neural network model that combines a variational autoencoder (VAE) and a Markov random field (MRF), the audio sample data can be effectively encoded. First, the artificial neural network model converts the audio sample data into an initial probability mean vector and an initial probability variance vector. Specifically, the variational autoencoder part is responsible for capturing the potential features in the audio data and generating the corresponding probability distribution parameters, namely the mean vector and the variance vector. Then, the Markov random field part further utilizes these probability distribution parameters to perform more sophisticated modeling and encoding of the audio data through its inherent graph structure and local dependencies. Through this combination, the model can not only capture the global features of the audio data, but also capture the local detail features, thereby achieving efficient encoding of the audio sample data.

[0076] S2. For each initial probability mean vector and initial probability variance vector, perform k transfer probability operations to obtain k transfer probabilities, where k is a natural number greater than 9;

[0077] For each initial probability mean vector and initial probability variance vector, k transition probability operations are performed to obtain k transition probabilities. Here, k is a natural number greater than 9, indicating that multiple transition probability operations are required to obtain enough data points.

[0078] When performing the specific steps of each transfer probability operation, you need to follow the following detailed process:

[0079] S201, resample the initial probability mean vector and the initial probability variance vector to obtain the transition probability. This means that it is necessary to extract features from the initial probability mean vector and the initial probability variance vector, and update these features through the resampling method to obtain a new transition probability. This process ensures that the transition probability can reflect the probability distribution of the current state and provides a basis for the next step of encoding.

[0080] S202, encode the transition probability obtained in the previous step to generate a transition probability mean vector and a transition probability variance vector. In this process, a coding technique is used to convert the transition probability into the form of a mean vector and a variance vector. The purpose of this step is to simplify the complex information of the transition probability into a vector form that is easier to process and analyze, thereby providing support for further probability models or decision-making processes.

[0081] Through these two steps, the transition probability can be effectively extracted and converted from the initial probability distribution, providing a solid foundation for subsequent probability analysis and decision-making.

[0082] S3. Using k transition probabilities to perform voiceprint recognition; specifically including:

[0083] S301, quantizing k transition probabilities one by one to obtain a transition probability matrix; the quantization process is to round off the transition probability to only one decimal place;

[0084] In S2, for each input data (transition probability), k transition probability operations are performed to obtain k transition probabilities, each of which is a vector. In this step, each transition probability is quantized and recorded to obtain a transition probability matrix.

[0085] S302, using a convolutional neural network to perform a convolution operation on the transfer probability matrix, and output a convolution vector;

[0086] The transfer probability matrix can be convolved via a conventional convolutional neural network (CNN) (the convolutional neural network is designed by the user). The output of the convolutional neural network is a vector, named the convolution vector.

[0087] S303, using a multi-layer perceptron to process the convolution vector to obtain c perception outputs, where c is the number of voiceprint categories defined by the user;

[0088] The convolution vector is processed by a conventional multi-layer perceptron (MLP) to obtain c perception outputs (c is the number of voiceprint categories defined by the user, and the multi-layer perceptron is designed by the user).

[0089] S304, using the Softmax algorithm to process the c perception outputs to obtain the probability of the audio sampling data in each category, and selecting the category with the largest probability, which is the result of voiceprint recognition.

[0090] In order to more deeply understand the technical solution of the present invention, the following non-limiting examples are given:

[0091] The formula of the Softmax algorithm is:

[0092] ;

[0093] in, is the output of the multilayer perceptron, including c components, refers to its i-th component.

[0094] Although both the traditional variational autoencoder and the present invention process the input data (the traditional variational autoencoder inputs the original data (mainly image data), and the present invention inputs the audio sampling data, such as recording the speaker's speech with a microphone on site and saving it as data in .WAV format. The internal processing during this period is usually PCM sampling, and then the waveform is recorded.), the encoding obtains the mean vector and the mean square error vector. However, the traditional variational autoencoder interprets the output as the mean and mean square error of the distribution of implicit features, while the present invention interprets the output as the mean and mean square error of the transition probability of the implicit feature distribution. The two have completely different meanings. In the present invention, the transition probability indicates a probability, which will generate a probability based on its own probability in subsequent processing. It is actually a transition probability feature, while the mean and mean square error of the distribution of the implicit features of the traditional variational autoencoder refer to the implicit features themselves, which directly generate the implicit features of the input data in subsequent processing.

[0095] The present invention resamples the transfer probability features of the mean vector and the mean square error vector to obtain the sampling results as the transfer probability of the subsequent step. The transfer probability is input into the transfer probability encoder and further encoded into another set of mean vectors and mean square error vectors, which represent the statistical characteristics of the transfer probability of the next stage and are named as the transfer probability mean vector and the transfer probability mean square error vector respectively. This process will be repeated k times (the k value is set by the user according to the application scenario, but it must be ensured that k is greater than or equal to 10) to continuously perform transfer probability feature resampling and subsequent processing.

[0096] The k-th obtained transfer probability mean vector and mean square error vector are regarded as the probabilistic statistical features of the implicit features of the input data (i.e., the audio sampling data). Based on them, subsequent implicit feature resampling and decoding operations are performed according to the traditional variational autoencoding method to obtain the generated data.

[0097] See also Figure 4, the artificial neural network model includes:

[0098] An audio data encoder encodes the audio sampling data to obtain an initial probability mean vector and an initial probability variance vector;

[0099] The transition probability loop generation module performs k transition probability operations for each initial probability mean vector and initial probability variance vector to obtain k transition probabilities;

[0100] The transition probability encoder encodes each transition probability to obtain a transition probability mean vector and a transition probability variance vector;

[0101] The decoder performs implicit feature resampling and decoding on the transition probability mean vector and the transition probability variance vector to obtain the generated audio data.

[0102] The training method of artificial neural network is:

[0103] 1) Prepare the dataset.

[0104] First, a series of preprocessing steps are required for the collected audio waveform data to ensure data consistency. Specifically, this includes adjusting all audio waveform data to the same length. For those audio files that are not of the original length, they will be filled with zero values ​​to ensure that they meet the required length standard. Conversely, for those audio files that are too long, appropriate truncation operations will be performed to remove the excess audio parts to ensure that the length of each audio file meets the preset requirements. This series of operations helps to standardize subsequent processing and analysis, ensuring the accuracy and efficiency of data processing.

[0105] Secondly, in order to better manage and use these audio waveform data, they need to be labeled by category. This means that a unique category number will be assigned to each audio waveform in the dataset. Specifically, a "data-category" pairing operation will be performed to ensure that any audio waveform in the dataset corresponds to a specific category number i. This labeling method will help with subsequent data classification, retrieval, and analysis, allowing us to perform targeted data processing and analysis based on different categories, thereby improving the efficiency and accuracy of data processing. Through this meticulous labeling process, these audio data can be better utilized for various machine learning and artificial intelligence applications.

[0106] 2) Design loss function.

[0107] A. First, calculate the variational autoencoder loss L1 according to formula (1);

[0108] (1)

[0109] in, refers to the mean of the transition probabilities generated for the kth time, is the mean square error of the k-th transition probability, x is the input audio sampling data, is the generated audio data, n is the number of data in the dataset, and λ is the user-defined loss term fine-tuning parameter, which is usually 1 and can be set by the user based on experience;

[0110] Calculate the classification loss L2;

[0111] (2)

[0112] in, is the one-hot encoding of the category to which the audio sample data x belongs, is the i-th component, is the output of the multilayer perceptron, including c components, refers to its i-th component;

[0113] Combined loss L;

[0114] (3)

[0115] Input the data in the data set into the artificial neural network model one by one, use the Adam optimization algorithm, run several epochs (in this embodiment, run about 40 epochs, the specific value is determined by the user), take out the artificial neural network weight parameters, and complete the training process.

[0116] The training process of the artificial neural network model includes:

[0117] Prepare a data set; first, regularize the collected audio waveform data to obtain audio waveform data of uniform size; then, label the audio waveform data by category;

[0118] Design loss function; specifically include:

[0119] Calculate the variational autoencoder loss L1;

[0120] (1)

[0121] in, refers to the mean of the transition probabilities generated for the kth time, is the mean square error of the k-th transition probability, x is the input audio sampling data, is the generated audio data, n is the number of data in the dataset, and λ is the user-defined fine-tuning parameter of the loss term;

[0122] Calculate the classification loss L2;

[0123] (2)

[0124] in, is the one-hot encoding of the category to which the audio sample data x belongs, is the i-th component, is the output of the multilayer perceptron, including c components, refers to its i-th component;

[0125] Combined loss L;

[0126] (3)

[0127] Input the data in the data set into the artificial neural network model one by one, use the Adam optimization algorithm, run several epochs, take out the artificial neural network weight parameters, and complete the training process.

[0128] The second embodiment is a voiceprint recognition system combining variational autoencoder and Markov random field, comprising:

[0129] An artificial neural network model combining variational autoencoder and Markov random field; encoding audio sampling data to obtain an initial probability mean vector and an initial probability variance vector; performing k transfer probability operations for each initial probability mean vector and initial probability variance vector to obtain k transfer probabilities, where k is a natural number greater than 9; the steps of each transfer probability operation are: firstly resampling the transfer probability features of the initial probability mean vector and the initial probability variance vector to obtain the transfer probability; then encoding the transfer probability to obtain a transfer probability mean vector and a transfer probability variance vector;

[0130] A voiceprint recognition model that uses k transition probabilities for voiceprint recognition; the specific recognition process includes: first, quantizing the k transition probabilities one by one to obtain a transition probability matrix; then using a convolutional neural network to perform a convolution operation on the transition probability matrix and output a convolution vector; then using a multi-layer perceptron to process the convolution vector to obtain c perception outputs, where c is the number of voiceprint categories defined by the user; finally, using the Softmax algorithm to process the c perception outputs to obtain the probability of audio sampling data in each category, and selecting the category with the largest probability, which is the result of voiceprint recognition.

[0131] The formula of the Softmax algorithm is:

[0132] ;

[0133] in, is the output of the multilayer perceptron, including c components, refers to its i-th component.

[0134] Although both the traditional variational autoencoder and the present invention process the input data (the traditional variational autoencoder inputs the original data, and the present invention inputs the audio sampling data), and encodes the mean vector and the mean square error vector. However, the traditional variational autoencoder interprets the output as the mean and mean square error of the distribution of the implicit feature, while the present invention interprets the output as the mean and mean square error of the transition probability of the implicit feature distribution. The two have completely different meanings. In the present invention, the transition probability indicates a probability, which will generate a probability according to its own probability in subsequent processing. It is actually a transition probability feature, while the mean and mean square error of the distribution of the implicit feature of the traditional variational autoencoder refers to the implicit feature itself, which directly generates the implicit feature of the input data in subsequent processing.

[0135] The present invention resamples the transfer probability features of the mean vector and the mean square error vector to obtain the sampling results as the transfer probability of the subsequent step. The transfer probability is input into the transfer probability encoder and further encoded into another set of mean vectors and mean square error vectors, which represent the statistical characteristics of the transfer probability of the next stage and are named as the transfer probability mean vector and the transfer probability mean square error vector respectively. This process will be repeated k times (the k value is set by the user according to the application scenario, but it must be ensured that k is greater than or equal to 10) to continuously perform transfer probability feature resampling and subsequent processing.

[0136] The k-th obtained transfer probability mean vector and mean square error vector are regarded as the probabilistic statistical features of the implicit features of the input data (i.e., the audio sampling data). Based on them, subsequent implicit feature resampling and decoding operations are performed according to the traditional variational autoencoding method to obtain the generated data.

[0137] See also Figure 4 , the artificial neural network model includes:

[0138] An audio data encoder encodes the audio sampling data to obtain an initial probability mean vector and an initial probability variance vector;

[0139] The transition probability loop generation module performs k transition probability operations for each initial probability mean vector and initial probability variance vector to obtain k transition probabilities;

[0140] The transition probability encoder encodes each transition probability to obtain a transition probability mean vector and a transition probability variance vector;

[0141] The decoder performs implicit feature resampling and decoding on the transition probability mean vector and the transition probability variance vector to obtain the generated audio data.

[0142] The training method of artificial neural network is:

[0143] 1) Prepare the dataset.

[0144] First, a series of preprocessing steps are required for the collected audio waveform data to ensure data consistency. Specifically, this includes adjusting all audio waveform data to the same length. For those audio files that are not of the original length, they will be filled with zero values ​​to ensure that they meet the required length standard. Conversely, for those audio files that are too long, appropriate truncation operations will be performed to remove the excess audio parts to ensure that the length of each audio file meets the preset requirements. This series of operations helps to standardize subsequent processing and analysis, ensuring the accuracy and efficiency of data processing.

[0145] Secondly, in order to better manage and use these audio waveform data, they need to be labeled by category. This means that a unique category number will be assigned to each audio waveform in the dataset. Specifically, a "data-category" pairing operation will be performed to ensure that any audio waveform in the dataset corresponds to a specific category number i. This labeling method will help with subsequent data classification, retrieval, and analysis, allowing us to perform targeted data processing and analysis based on different categories, thereby improving the efficiency and accuracy of data processing. Through this meticulous labeling process, these audio data can be better utilized for various machine learning and artificial intelligence applications.

[0146] 2) Design loss function.

[0147] A. First, calculate the variational autoencoder loss L1 according to formula (1);

[0148] (1)

[0149] in, refers to the mean of the transition probabilities generated for the kth time, is the mean square error of the k-th transition probability, x is the input audio sampling data, is the generated audio data, n is the number of data in the dataset, and λ is the user-defined loss term fine-tuning parameter, which is usually 1 and can be set by the user based on experience;

[0150] Calculate the classification loss L2;

[0151] (2)

[0152] in, is the one-hot encoding of the category to which the audio sample data x belongs, is the i-th component, is the output of the multilayer perceptron, including c components, refers to its i-th component;

[0153] Combined loss L;

[0154] (3)

[0155] Input the data in the data set into the artificial neural network model one by one, use the Adam optimization algorithm, run several epochs (in this embodiment, run about 40 epochs, the specific value is determined by the user), take out the artificial neural network weight parameters, and complete the training process.

[0156] The training process of the artificial neural network model includes:

[0157] Prepare a data set; first, regularize the collected audio waveform data to obtain audio waveform data of uniform size; then, label the audio waveform data by category;

[0158] Design loss function; specifically include:

[0159] Calculate the variational autoencoder loss L1;

[0160] (1)

[0161] in, refers to the mean of the transition probabilities generated for the kth time, is the mean square error of the k-th transition probability, x is the input audio sampling data, is the generated audio data, n is the number of data in the dataset, and λ is the user-defined fine-tuning parameter of the loss term;

[0162] Calculate the classification loss L2;

[0163] (2)

[0164] in, is the one-hot encoding of the category to which the audio sample data x belongs, is the i-th component, is the output of the multilayer perceptron, including c components, refers to its i-th component;

[0165] Combined loss L;

[0166] (3)

[0167] Input the data in the data set into the artificial neural network model one by one, use the Adam optimization algorithm, run several epochs, extract the artificial neural network weight parameters, and complete the training process;

[0168] Take out the trained audio encoder, transition probability decoder, convolutional neural network and multi-layer perceptron. Load the weights of the artificial neural network into Figure 5 The corresponding network is shown.

[0169] Pass an unknown audio sample data through Figure 5 The process shown can get the predicted voiceprint category.

[0170] A third embodiment is a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-mentioned voiceprint recognition method combining variational autoencoder and Markov random field.

[0171] A fourth embodiment is a computer program product, comprising a computer program, which, when executed by a processor, implements the above-mentioned voiceprint recognition method combining variational autoencoder and Markov random field.

[0172] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When the use is implemented in whole or in part in the form of a computer program product, the computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL) or wireless (e.g., infrared, wireless, microwave, etc.) mode) to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk SolidState Disk (SSD)), etc.

[0173] The above is only a preferred embodiment of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A voiceprint recognition method combining variational autoencoder and Markov random field, characterized in that: include: S1. Encode the audio sampling data using an artificial neural network model combining variational autoencoder and Markov random field to obtain an initial probability mean vector and an initial probability variance vector; S2. For each initial probability mean vector and initial probability variance vector, perform k transfer probability operations to obtain k transfer probabilities, where k is a natural number greater than 9; the steps of each transfer probability operation are: S201, resampling the initial probability mean vector and the initial probability variance vector to obtain a transition probability; S202, encoding the transition probability to obtain a transition probability mean vector and a transition probability variance vector; S3. Using k transition probabilities to perform voiceprint recognition; specifically including: S301, quantizing k transition probabilities one by one to obtain a transition probability matrix; S302, using a convolutional neural network to perform a convolution operation on the transfer probability matrix, and output a convolution vector; S303, using a multi-layer perceptron to process the convolution vector to obtain c perception outputs, where c is the number of voiceprint categories defined by the user; S304, using the Softmax algorithm to process the c perception outputs to obtain the probability of the audio sampling data in each category, and selecting the category with the largest probability, which is the result of voiceprint recognition.

2. The voiceprint recognition method combining variational autoencoder and Markov random field according to claim 1 is characterized in that: The formula of the Softmax algorithm is: ； in, is the output of the multilayer perceptron, including c components, refers to its i-th component.

3. The voiceprint recognition method combining variational autoencoder and Markov random field according to claim 1 is characterized in that: The artificial neural network model comprises: An audio data encoder encodes the audio sampling data to obtain an initial probability mean vector and an initial probability variance vector; The transition probability loop generation module performs k transition probability operations for each initial probability mean vector and initial probability variance vector to obtain k transition probabilities; The transition probability encoder encodes each transition probability to obtain a transition probability mean vector and a transition probability variance vector; The decoder performs implicit feature resampling and decoding on the transition probability mean vector and the transition probability variance vector to obtain the generated audio data.

4. The voiceprint recognition method combining variational autoencoder and Markov random field according to claim 3 is characterized in that: The training process of the artificial neural network model includes: Prepare a data set; first, regularize the collected audio waveform data to obtain audio waveform data of uniform size; then, label the audio waveform data by category; Design loss function; specifically include: Calculate the variational autoencoder loss L1; （1） in, refers to the mean of the transition probabilities generated for the kth time, is the mean square error of the k-th transition probability, x is the input audio sampling data, is the generated audio data, n is the number of data in the dataset, and λ is the user-defined fine-tuning parameter of the loss term; Calculate the classification loss L2; （2） in, is the one-hot encoding of the category to which the audio sample data x belongs, is the ith component, is the output of the multilayer perceptron, including c components, refers to its i-th component; Combined loss L; （3） Input the data in the data set into the artificial neural network model one by one, use the Adam optimization algorithm, run several epochs, take out the artificial neural network weight parameters, and complete the training process.

5. A voiceprint recognition system combining variational autoencoder and Markov random field, characterized in that: include: An artificial neural network model combining variational autoencoder and Markov random field; encoding audio sampling data to obtain an initial probability mean vector and an initial probability variance vector; performing k transfer probability operations for each initial probability mean vector and initial probability variance vector to obtain k transfer probabilities, where k is a natural number greater than 9; the steps of each transfer probability operation are: firstly resampling the transfer probability features of the initial probability mean vector and the initial probability variance vector to obtain the transfer probability; then encoding the transfer probability to obtain a transfer probability mean vector and a transfer probability variance vector; A voiceprint recognition model that uses k transition probabilities for voiceprint recognition; The specific recognition process includes: first, quantize the k transition probabilities one by one to obtain a transition probability matrix; then use the convolutional neural network to perform a convolution operation on the transition probability matrix and output a convolution vector; then use the multi-layer perceptron to process the convolution vector to obtain c perception outputs, where c is the number of voiceprint categories defined by the user; finally, use the Softmax algorithm to process the c perception outputs to obtain the probability of the audio sampling data in each category, and select the category with the largest probability, which is the result of voiceprint recognition.

6. The voiceprint recognition system combining variational autoencoder and Markov random field according to claim 5, characterized in that: The formula of the Softmax algorithm is: ； in, is the output of the multilayer perceptron, including c components, refers to its i-th component.

7. The voiceprint recognition system combining variational autoencoder and Markov random field according to claim 5, characterized in that: The artificial neural network model includes: an audio data encoder, a transition probability loop generation module, a transition probability encoder and a decoder.

8. The voiceprint recognition system combining variational autoencoder and Markov random field according to claim 7, characterized in that: The training process of the artificial neural network model includes: Prepare a data set; first, regularize the collected audio waveform data to obtain audio waveform data of uniform size; then, label the audio waveform data by category; Design loss function; specifically include: Calculate the variational autoencoder loss L1; （1） in, refers to the mean of the transition probabilities generated for the kth time, is the mean square error of the k-th transition probability, x is the input audio sampling data, is the generated audio data, n is the number of data in the dataset, and λ is the user-defined fine-tuning parameter of the loss term; Calculate the classification loss L2; （2） in, is the one-hot encoding of the category to which the audio sample data x belongs, is the ith component, is the output of the multilayer perceptron, including c components, refers to its i-th component; Combined loss L; （3） Input the data in the data set into the artificial neural network model one by one, use the Adam optimization algorithm, run several epochs, take out the artificial neural network weight parameters, and complete the training process.

9. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the voiceprint recognition method combining variational autoencoding and Markov random field as described in any one of claims 1 to 4 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the program is executed by a processor, the voiceprint recognition method combining variational autoencoding and Markov random field as described in any one of claims 1 to 4 is implemented.

Images