Voiceprint recognition method, device, electronic device and storage medium

Abstract

The present disclosure provides a voiceprint recognition method, device, electronic device and storage medium, which relate to the field of computer technology, and in particular to the field of artificial intelligence technology such as speech recognition and deep learning. The specific implementation scheme is: obtaining first audio data to be recognized; inputting the first audio data into a preset voiceprint recognition model to obtain the first voiceprint feature and the second voiceprint feature output by the voiceprint recognition model, wherein the preset voiceprint recognition model is generated based on a teacher model for identifying voiceprint distribution probability and training data marked with user labels; according to the first voiceprint feature and the second voiceprint feature, determining the target voiceprint feature corresponding to the first audio data. Thus, by performing voiceprint recognition on audio data from the two perspectives of voiceprint distribution probability and corresponding user labels, the accuracy of the acquired target voiceprint feature is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, in particular to artificial intelligence technologies such as speech recognition and deep learning, and specifically to a voiceprint recognition method, device, electronic device, and storage medium. Background Art

[0002] Voiceprints, like fingerprints, are unique to an individual. Different words spoken by a person should have consistent voiceprints. Therefore, voiceprint recognition can be used to identify the speaker. However, in practice, voiceprint recognition is often affected by multiple factors, including the speaker and the environment, which can affect its accuracy. Therefore, improving the accuracy of voiceprint recognition has become a key research topic. Summary of the Invention

[0003] The present disclosure provides a voiceprint recognition method, device, electronic device, and storage medium.

[0004] According to a first aspect of the present disclosure, a voiceprint recognition method is provided, comprising:

[0005] Acquire first audio data to be recognized;

[0006] Inputting the first audio data into a preset voiceprint recognition model to obtain a first voiceprint feature and a second voiceprint feature output by the voiceprint recognition model, wherein the preset voiceprint recognition model is generated by training based on a teacher model for identifying voiceprint distribution probability and training data labeled with user labels;

[0007] Determine a target voiceprint feature corresponding to the first audio data based on the first voiceprint feature and the second voiceprint feature.

[0008] According to a second aspect of the present disclosure, a voiceprint recognition device is provided, comprising:

[0009] A first acquisition module, configured to acquire first audio data to be recognized;

[0010] a second acquisition module, configured to input the first audio data into a preset voiceprint recognition model to obtain first and second voiceprint features output by the voiceprint recognition model, wherein the preset voiceprint recognition model is generated by training based on a teacher model for identifying voiceprint distribution probability and training data annotated with user labels;

[0011] The first determining module is configured to determine a target voiceprint feature corresponding to the first audio data according to the first voiceprint feature and the second voiceprint feature.

[0012] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the voiceprint recognition method as described in the first aspect.

[0016] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the voiceprint recognition method as described in the first aspect.

[0017] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising computer instructions, which, when executed by a processor, implement the steps of the voiceprint recognition method as described in the first aspect.

[0018] The voiceprint recognition method, device, electronic device, and storage medium provided by the present disclosure have the following beneficial effects:

[0019] In the disclosed embodiment, first audio data to be recognized is acquired. The first audio data is then input into a preset voiceprint recognition model to obtain the first and second voiceprint features output by the voiceprint recognition model. Finally, the target voiceprint features corresponding to the first audio data are determined based on the first and second voiceprint features. Thus, by performing voiceprint recognition on audio data from two perspectives, namely, voiceprint distribution probability and corresponding user labels, the accuracy of the acquired target voiceprint features is improved.

[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings are used to better understand the present invention and do not constitute a limitation of the present invention.

[0022] Figure 1 1 is a flow chart of a voiceprint recognition method provided according to an embodiment of the present disclosure;

[0023] Figure 2 is a flow chart of a voiceprint recognition method provided according to another embodiment of the present disclosure;

[0024] Figure 3 is a flow chart of a voiceprint recognition method provided according to another embodiment of the present disclosure;

[0025] Figure 4 is a flow chart of a voiceprint recognition method provided according to another embodiment of the present disclosure;

[0026] Figure 5 is a flow chart of a voiceprint recognition method provided according to another embodiment of the present disclosure;

[0027] Figure 6 is a flow chart of a voiceprint recognition method provided according to another embodiment of the present disclosure;

[0028] Figure 7 is a flow chart of a voiceprint recognition method provided according to another embodiment of the present disclosure;

[0029] Figure 8 1 is a schematic structural diagram of a voiceprint recognition device provided according to an embodiment of the present disclosure;

[0030] Figure 9 It is a block diagram of an electronic device used to implement the voiceprint recognition method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0032] The embodiments of the present disclosure relate to the fields of artificial intelligence technologies such as computer vision and deep learning.

[0033] Artificial Intelligence (AI) is a new technical science that studies and develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence.

[0034] Speech recognition, also known as automatic speech recognition, aims to convert the content of human language into computer-readable input, such as keystrokes, binary codes, or character sequences, thereby making it possible for humans and machines to interact through natural language.

[0035] Deep learning involves learning the inherent patterns and representational hierarchies of sample data. The information gained from this learning process is highly helpful in interpreting data such as text, images, and sounds. The ultimate goal of deep learning is to enable machines to have the same analytical and learning capabilities as humans, enabling them to recognize data such as text, images, and sounds.

[0036] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0037] Figure 1 The figure is a flow chart of a voiceprint recognition method provided according to an embodiment of the present disclosure.

[0038] It should be noted that the executor of the voiceprint recognition method of this embodiment is a voiceprint recognition device, which can be implemented by software and / or hardware. The device can be configured in an electronic device, which may include but is not limited to a terminal, a server, etc.

[0039] like Figure 1 As shown, the voiceprint recognition method includes:

[0040] S101: Acquire first audio data to be recognized.

[0041] The first audio data to be recognized is a continuous speech that needs to be recognized by voiceprint, such as a sentence, a paragraph, etc.

[0042] Optionally, the first audio data to be recognized can be audio data that is collected in real time by a recording device such as a recorder or microphone and that requires voiceprint recognition. For example, when a user wakes up an electronic device such as a mobile phone or tablet, the collection device in the electronic device collects audio data.

[0043] Alternatively, the first audio data to be identified may be pre-acquired audio data that needs to be subjected to voiceprint recognition, such as audio data downloaded from the Internet or from social software, etc. This disclosure does not limit this.

[0044] S102: Input the first audio data into a preset voiceprint recognition model to obtain a first voiceprint feature and a second voiceprint feature output by the voiceprint recognition model, wherein the preset voiceprint recognition model is generated based on a teacher model for identifying voiceprint distribution probability and training data marked with user labels.

[0045] The preset voiceprint recognition model is pre-trained and is used to perform voiceprint recognition on any audio data to obtain the voiceprint distribution probability and corresponding user label corresponding to the audio data. In the present disclosure, the first voiceprint feature can be output by the voiceprint distribution probability subnetwork in the preset voiceprint recognition model, and the second voiceprint feature can be output by the user label subnetwork in the preset voiceprint recognition model.

[0046] In an embodiment of the present disclosure, the first audio data can be processed using a preset voiceprint recognition model to determine the first voiceprint feature corresponding to the first audio data based on the output of the previous layer adjacent to the output layer in the voiceprint distribution probability subnetwork in the preset voiceprint recognition model, and determine the second voiceprint feature based on the output of the previous layer adjacent to the output layer in the user label subnetwork in the preset voiceprint recognition model.

[0047] It should be noted that the voiceprint probability distribution subnetwork in the voiceprint recognition model is primarily used to obtain the voiceprint distribution probability in the audio data. Therefore, the first voiceprint feature output by the voiceprint probability distribution subnetwork is more likely to reflect the voiceprint distribution characteristics of the audio data. The user label subnetwork in the voiceprint recognition model is primarily used to obtain the user label corresponding to the audio data. Therefore, the second voiceprint feature output by the user label subnetwork is more likely to reflect the user label corresponding to the audio data.

[0048] S103: Determine a target voiceprint feature corresponding to the first audio data according to the first voiceprint feature and the second voiceprint feature.

[0049] Optionally, the first voiceprint feature and the second voiceprint feature output by the voiceprint recognition model can be represented in the form of vectors. Therefore, after obtaining the first voiceprint feature and the second voiceprint feature, the average value of the first voiceprint feature and the second voiceprint feature can be used as the target voiceprint feature.

[0050] Alternatively, corresponding weights may be assigned to the first voiceprint feature and the second voiceprint feature, and then the weighted sum of the first and second voiceprint features may be used as the target voiceprint feature. That is, the sum of the product of the first voiceprint feature and its corresponding weight, and the product of the second voiceprint feature and its corresponding weight may be used as the target voiceprint feature.

[0051] For example, if the weight corresponding to the first voiceprint feature is 0.4 and the weight corresponding to the second voiceprint feature is 0.6, then the target voiceprint feature = first voiceprint feature * 0.4 + second voiceprint feature * 0.6.

[0052] It should be noted that the above exemplary knowledge is simply an illustration and cannot be used as a specific limitation on the weight corresponding to the first voiceprint feature and the weight corresponding to the second voiceprint feature in the present disclosure.

[0053] In the disclosed embodiment, first audio data to be recognized is acquired. The first audio data is then input into a preset voiceprint recognition model to obtain the first and second voiceprint features output by the voiceprint recognition model. Finally, the target voiceprint features corresponding to the first audio data are determined based on the first and second voiceprint features. Thus, by performing voiceprint recognition on audio data from two perspectives, namely, voiceprint distribution probability and corresponding user labels, the accuracy of the acquired target voiceprint features is improved.

[0054] Figure 2 FIG. 1 is a flow chart of a voiceprint recognition method according to another embodiment of the present disclosure. Figure 2 As shown, the voiceprint recognition method includes:

[0055] S201: Acquire first audio data to be recognized.

[0056] The specific implementation of step S201 can refer to the detailed description in other embodiments of the present disclosure and will not be described in detail here.

[0057] S202: Input the first audio data into an audio feature extraction network in a preset voiceprint recognition model to determine a first audio feature corresponding to the first audio data.

[0058] Optionally, before performing audio feature extraction on the first audio data, the first audio data may be framed, i.e., the first audio data may be divided into small segments of fixed length. Since Fourier transform is used to convert the audio data from a time domain signal to a frequency domain signal when performing audio feature extraction on the audio data, and Fourier transform is applicable to stable signals, in order to ensure the short-term stability of the audio data, 20 milliseconds (ms) to 40ms of audio is generally taken as one frame. For example, the length of each frame of audio may be 25ms, and this is not limited in the present disclosure.

[0059] Optionally, when framing the first audio data, to avoid missing the first audio data at the window boundary, the frames are offset by frame overlap (also known as frame shift). That is, each frame of audio data needs to overlap. Generally, half the frame length is used as the frame shift. For example, if the frame length is 25ms, the frame shift can be 10ms. This is not limited in this disclosure.

[0060] Optionally, Mel-frequency Cepstrum Coefficients (MFCC) may be used to extract audio features from each frame of audio in the first audio data. Alternatively, Perceptual Linear Predictive Coefficients (PLP) may be used to extract audio features from each frame of audio in the first audio data. Alternatively, a FilterBank (Fbank) may be used to extract audio features from each frame of audio in the first audio data. This disclosure does not limit this.

[0061] S203: Input the first audio feature into a content feature extraction network in a preset voiceprint recognition model to determine a first content feature corresponding to the first audio data.

[0062] Optionally, the content extraction network may be composed of a multi-layer time delay neural network (TDNN) or a multi-layer long short term memory (LSTM) network, which is not limited in this disclosure. The first content feature may be extracted from the penultimate hidden layer in the content extraction network.

[0063] Optionally, the content extraction network can also obtain the probability of each frame of audio data containing a phoneme. Wherein, a phoneme can be a pronunciation unit such as a vowel, a vowel, or a phonetic symbol. This disclosure does not limit this.

[0064] S204: Inputting the first audio feature and the first content feature into a voiceprint recognition network in a preset voiceprint recognition model to obtain a first voiceprint feature and a second voiceprint feature output by the voiceprint recognition model.

[0065] Optionally, the first audio feature and the first content feature may be fused and then input into a voiceprint recognition network in a preset voiceprint recognition model.

[0066] Optionally, the first audio feature and the first content feature may be concatenated to obtain a fused first feature. It should be noted that the disclosed embodiments do not limit the order in which the first audio feature and the second audio feature are concatenated. That is, the first content feature may be concatenated after the first audio feature, or after the first content feature.

[0067] Alternatively, corresponding weights may be assigned to the first audio feature and the first content feature, and then the weighted sum of the first audio feature and the first content feature may be used as the first feature. Specifically, the first feature may be the sum of the product of the first audio feature and its corresponding weight, and the product of the first content feature and its corresponding weight.

[0068] For example, if the weight corresponding to the first audio feature is 0.4 and the weight corresponding to the first content feature is 0.6, then the first feature = first audio feature * 0.4 + second content feature * 0.6.

[0069] In the disclosed embodiment, the first feature obtained by fusing the first content feature and the first audio feature corresponding to the first audio data is input into a preset voiceprint recognition model to obtain the first and second voiceprint features corresponding to the first audio data. Thus, the first and second voiceprint features corresponding to the first audio data are obtained in combination with the first content feature corresponding to the first audio data. This facilitates the voiceprint recognition model to perform voiceprint recognition on the first audio data based on the content corresponding to the first audio data, thereby reducing the influence of content factors on voiceprint feature extraction and improving the accuracy of the obtained first and second voiceprint features.

[0070] S205: Determine a target voiceprint feature corresponding to the first audio data according to the first voiceprint feature and the second voiceprint feature.

[0071] The specific implementation of step S205 can refer to the detailed description in other embodiments of the present disclosure and will not be described in detail here.

[0072] In the embodiment of the present disclosure, first audio data to be recognized is obtained, and then the first audio data is input into an audio feature extraction network in a preset voiceprint recognition model to determine a first audio feature corresponding to the first audio data. The first audio feature is then input into a content feature extraction network in the preset voiceprint recognition model to determine a first content feature corresponding to the first audio data. The first audio feature and the first content feature are then input into the voiceprint recognition network in the preset voiceprint recognition model to obtain a first voiceprint feature and a second voiceprint feature output by the voiceprint recognition model. Finally, based on the first voiceprint feature and the second voiceprint feature, a target voiceprint feature corresponding to the first audio data is determined. Thus, the first audio feature and the first content feature corresponding to the first audio data are input into the voiceprint recognition model together, which facilitates the voiceprint recognition model to perform voiceprint recognition on the first audio data based on the content corresponding to the first audio data, thereby reducing the influence of content factors on the voiceprint features extracted by the voiceprint recognition model, thereby improving the accuracy of the first voiceprint feature and the second voiceprint feature output by the voiceprint recognition model, and thereby improving the accuracy of the target voiceprint feature.

[0073] Figure 3 FIG. 1 is a flow chart of a voiceprint recognition method according to another embodiment of the present disclosure. Figure 3 As shown, the voiceprint recognition method includes:

[0074] S301: Acquire first audio data to be recognized.

[0075] The specific implementation of step S301 can refer to the detailed description in other embodiments of the present disclosure and will not be described in detail here.

[0076] S302: Input each frame of audio in the first audio data into a feature extraction network in a preset voiceprint recognition model in sequence to determine a second audio feature corresponding to each frame of audio.

[0077] Optionally, the feature extraction network may be a deep residual network (ResNet), or a deep convolutional neural network (VGG, AlexNet), etc. This disclosure does not limit this.

[0078] Optionally, before inputting the first audio data into the feature extraction network, the first audio data may be framed to obtain each frame of audio. It should be noted that the specific implementation of framing the first audio data can refer to the detailed steps in other embodiments of the present disclosure.

[0079] S303: Obtain a first reference voiceprint feature and a second reference voiceprint feature corresponding to each frame of audio.

[0080] Among them, the first reference voiceprint feature is the reference voiceprint feature corresponding to the first voiceprint feature of each frame of audio output by the voiceprint recognition model, and the second reference voiceprint feature is the reference voiceprint feature corresponding to the second voiceprint feature of each frame of audio output by the voiceprint recognition model.

[0081] It can be understood that in the embodiment of the present disclosure, the corresponding first voiceprint feature is determined in combination with the first reference voiceprint feature corresponding to each frame of audio, and the corresponding second voiceprint feature is determined in combination with the second reference voiceprint feature corresponding to each frame of audio, so that the first voiceprint feature and the second voiceprint feature corresponding to each frame of audio obtained are more accurate.

[0082] Optionally, when the current processing frame is the first frame of the first audio data, the first preset feature and the second preset feature are respectively determined as the first reference voiceprint feature and the second reference voiceprint feature corresponding to the current processing frame.

[0083] Alternatively, if the currently processed frame is not the first frame, the first voiceprint feature corresponding to the previous frame and the second voiceprint feature corresponding to the previous frame output by the preset voiceprint recognition model are respectively determined as the first reference voiceprint feature and the second reference voiceprint feature corresponding to the currently processed frame. That is, the first voiceprint feature corresponding to the previous frame of audio is used as the first reference voiceprint feature of the current frame, and the second voiceprint feature corresponding to the previous frame of audio is used as the second reference voiceprint feature of the current frame. This tightly combines the voiceprint features corresponding to consecutive frames of audio data, thereby improving the accuracy of the first and second voiceprint features obtained for the current frame.

[0084] S304: Input the second audio feature, the first reference voiceprint feature, and the second reference voiceprint feature into a feature processing network in a preset voiceprint recognition model to obtain the first voiceprint feature and the second voiceprint feature corresponding to each frame of audio.

[0085] Optionally, the feature processing network can be a multihead self attention network (MSA) or average-pooling. This disclosure does not limit this. It should be noted that average pooling is a simple averaging of the input second audio features, the first reference voiceprint features, and the second reference voiceprint features, while the multihead self attention network can reflect the importance relationship between the second audio features, the first reference voiceprint features, and the second reference voiceprint features.

[0086] S305: Determine a target voiceprint feature corresponding to the first audio data according to the first voiceprint feature and the second voiceprint feature corresponding to each frame of audio.

[0087] It can be understood that after obtaining the first voiceprint feature and the second voiceprint feature corresponding to each frame of audio, the first voiceprint feature and the second voiceprint feature corresponding to each frame of audio can be processed to obtain the target voiceprint feature corresponding to each frame of audio, and then the target voiceprint feature corresponding to the first audio data can be determined based on the target voiceprint feature corresponding to each frame of audio.

[0088] The specific implementation of step S305 can be referred to the detailed description in other embodiments of the present disclosure and will not be described in detail here.

[0089] In the embodiment of the present disclosure, first, the first audio data to be identified is obtained. Then, each frame of audio in the first audio data is sequentially input into a feature extraction network in a preset voiceprint recognition model to determine the second audio feature corresponding to each frame of audio. Then, the first reference voiceprint feature and the second reference voiceprint feature corresponding to each frame of audio are obtained. The second audio feature, the first reference voiceprint feature, and the second reference voiceprint feature are input into a feature processing network in the preset voiceprint recognition model to obtain the first voiceprint feature and the second voiceprint feature corresponding to each frame of audio. Finally, based on the first voiceprint feature and the second voiceprint feature corresponding to each frame of audio, the target voiceprint feature corresponding to the first audio data is determined. Thus, based on the second audio feature, the first reference voiceprint feature, and the second reference voiceprint feature corresponding to each frame of audio, the first voiceprint feature and the second voiceprint feature corresponding to the first audio data are jointly determined, thereby closely combining the voiceprint features of each frame of audio, further improving the accuracy of the first voiceprint feature and the second voiceprint feature output by the voiceprint recognition model, and thereby improving the accuracy of the target voiceprint feature.

[0090] Through the above analysis, it can be seen that the present disclosure can obtain the first voiceprint feature and the second voiceprint feature corresponding to the first audio data through the preset voiceprint recognition model. Figure 4 、 Figure 5 、 Figure 6 and Figure 7 The process of obtaining the preset voiceprint recognition model is described in detail.

[0091] Figure 4 FIG. 1 is a flow chart of a voiceprint recognition method according to another embodiment of the present disclosure. Figure 4 As shown, the voiceprint recognition method includes:

[0092] S401: Obtain a first training data set and a generated teacher model, wherein the first training data set is a text-related audio data set, and the teacher model is generated based on a text-independent second training data set.

[0093] Among them, text-independent means that voiceprint recognition is independent of the content of the speech, that is, the speaker's identity can be identified regardless of the speaker's speech content. Text-dependent means that the speaker must pronounce according to pre-specified content.

[0094] Optionally, the specific steps of obtaining the teacher model may include: performing audio feature extraction on the sample audio data in the second training data set to determine the sample audio features corresponding to the sample audio data; then performing content feature extraction on the sample audio features to determine the sample content features corresponding to the sample audio data; fusing the sample audio features and the sample content features, and inputting the fused features into the initial teacher model to obtain the predicted voiceprint distribution probability corresponding to the sample audio data output by the initial teacher model; and correcting the initial teacher model based on the difference between the predicted voiceprint distribution probability corresponding to the sample audio data and the labeled voiceprint distribution probability to generate a teacher model.

[0095] Optionally, a cross entropy loss function may be used to obtain the difference between the predicted voiceprint distribution probability and the annotated voiceprint distribution probability.

[0096] The second training data set may be a large amount of text-independent audio data obtained from open source data such as SRE04-16 and VoxCeleb.

[0097] S402: Input the second audio data in the first training data set into the teacher model and the initial student model respectively to obtain the first voiceprint distribution probability output by the teacher model, the second voiceprint distribution probability output by the initial student model, and the predicted label output by the initial student model.

[0098] Optionally, before inputting the second audio data into the teacher model and the initial student model, the following steps may be performed: first, audio feature extraction is performed on the second audio data to determine a third audio feature corresponding to the second audio data; then, content feature extraction is performed on the third audio feature to determine a second content feature corresponding to the second audio data; then, the third audio feature and the second content feature are fused to obtain a fused third feature; and finally, the fused third feature is input into the teacher model and the initial student model, respectively. Thus, by training the initial student model based on the content feature corresponding to the second audio data, the influence of content factors on voiceprint feature extraction can be reduced.

[0099] It should be noted that the specific implementation of fusing the third audio feature and the second content feature can refer to the detailed description of fusing the first audio feature and the first content feature in this disclosure, and will not be repeated here.

[0100] Optionally, the third audio data may be input into a preset content extraction network to obtain a second content feature corresponding to the third audio data.

[0101] Optionally, the specific steps of obtaining the content extraction network may include: obtaining a third training data set, wherein the third training data set may be audio data obtained from open source data such as WSJ (Wall Street Journal) and Switchboard; then framing each audio data in the third training data set, and performing feature extraction on each frame of audio data to obtain audio features corresponding to each frame of audio data, and then performing mean normalization on the audio features corresponding to each frame of audio data, that is, obtaining the mean of the audio features corresponding to a preset number of frames before and after the current frame of audio, and then inputting the mean into the initial content extraction network to obtain the probability that the current frame of audio data contains phonemes, and based on the difference between the probability output by the initial content extraction and the labeled probability, correcting the initial content extraction network to obtain the content extraction network.

[0102] Optionally, a cross entropy loss function can be used to determine the difference between the probability output by the initial content extraction network and the labeled probability.

[0103] S403: According to the difference between the predicted label and the annotated label corresponding to the first audio data, and the difference between the second voiceprint distribution probability and the first voiceprint distribution probability, the initial student model is modified to obtain a preset voiceprint recognition model.

[0104] It is understandable that in the embodiment of the present disclosure, the initial student model can be iteratively trained according to the multiple text-related audio data contained in the first training data set until the model converges, thereby improving the convergence and robustness of the voiceprint recognition model.

[0105] In the disclosed embodiments, because the teacher model is trained using a large amount of text-independent audio data, the teacher model has a strong ability to distinguish between speakers. In the disclosed embodiments, the teacher model can be used to guide the training of the initial student model, thereby imparting the teacher model's strong ability to distinguish between speakers to the initial student model. This allows the obtained preset voiceprint recognition model to have a strong ability to distinguish between speakers when the first training dataset contains a small amount of text-related audio data, thereby improving the accuracy of the voiceprint features obtained by the voiceprint recognition model.

[0106] In the disclosed embodiment, a first training data set and a generated teacher model are obtained, and then the second audio data in the first training data set are input into the teacher model and the initial student model, respectively, to obtain the first voiceprint distribution probability output by the teacher model, and the second voiceprint distribution probability and predicted label output by the initial student model. Finally, based on the difference between the predicted label and the labeled label corresponding to the first audio data, and the difference between the second voiceprint distribution probability and the first voiceprint distribution probability, the initial student model is corrected to obtain a preset voiceprint recognition model. Thus, the teacher model trained with a large amount of text-independent audio data guides the training of the initial student model, so that when the first training data set contains a small amount of text-related audio data, the generated voiceprint recognition model has a strong ability to distinguish the features of the audio data, thereby providing conditions for accurately obtaining the first voiceprint feature and the second voiceprint feature corresponding to the first audio data.

[0107] Figure 5 FIG. 1 is a flow chart of a voiceprint recognition method according to another embodiment of the present disclosure. Figure 5 As shown, the voiceprint recognition method includes:

[0108] S501: Obtain a first training data set and a generated teacher model, wherein the first training data set is a text-related audio data set, and the teacher model is generated based on a text-independent second training data set.

[0109] The specific implementation of step S501 can refer to the detailed description in other embodiments of the present disclosure and will not be described in detail here.

[0110] S502: Determine the number of random numbers to be generated according to the duration of the second audio data.

[0111] Optionally, the number of random numbers to be generated corresponding to each duration interval may be set in advance for each duration interval, and then the number of random numbers to be generated may be determined according to the duration interval in which the duration of the second audio data is located.

[0112] For example, the number of random numbers to be generated corresponding to the duration interval of 4000ms-5000ms is preset to 20. If the duration of the second audio data is 4500ms, the number of random numbers to be generated corresponding to the second audio data may be 20.

[0113] Alternatively, a range of the number of random numbers to be generated corresponding to each duration interval may be set in advance for each duration interval, and then the number of random numbers to be generated may be randomly determined according to the duration interval in which the duration of the second audio data is located.

[0114] For example, the number of random numbers to be generated corresponding to the duration interval of 4000ms-5000ms is preset to be 10-20. If the duration of the second audio data is 4800ms, the number of random numbers to be generated can be 12 or 15, and this disclosure does not limit this.

[0115] S503: Generate a first random erasure parameter group and a second random erasure parameter group based on the number of random numbers to be generated, wherein the first random erasure parameter group and the second random parameter group respectively include multiple random numbers and a duration parameter corresponding to each random number and a frequency parameter corresponding to each random number.

[0116] The duration parameter may be the duration of erasing the second audio data in the time domain.

[0117] The frequency point parameter may be a plurality of frequency points around each random number.

[0118] Optionally, the random numbers, duration parameters, and frequency parameters corresponding to each random number included in the first random erasure parameter group and the second random erasure parameter group can all be randomly generated. Therefore, the duration parameters corresponding to each random number can be the same or different, and the frequency parameters corresponding to each random number can be the same or different. This disclosure does not limit this.

[0119] S504: Input the second audio data into the random erasure network in the teacher model to process the second audio data based on the first random erasure parameter group to obtain processed third audio data.

[0120] Optionally, the time domain of the second audio data can be erased based on the multiple random numbers included in the first random parameter group and the duration parameters corresponding to each random number, and the frequency domain of the second audio data can be erased based on the multiple random numbers and the frequency parameters corresponding to each random number to obtain processed third audio data.

[0121] For example, the time point corresponding to each random number is used as the starting point of time domain erasure, and the value between each random number and the area formed by the time length parameter corresponding to each random number is set to 0, so that the area formed by each starting point and its corresponding time length parameter can be erased from the time domain of the second audio data.

[0122] The time point corresponding to each random number is used as the starting point for frequency domain erasure. The spectral region to be erased is determined based on the frequency point parameters corresponding to each random number, the frequency range to be erased, and the time range to be erased. The values ​​within the spectral region to be erased are set to 0, thereby erasing the spectral region to be erased corresponding to each random number from the frequency domain of the second audio data. For example, the spectrum within a 50 Hz and 40 ms region near multiple frequency points corresponding to each random number can be set to 0.

[0123] It should be noted that the randomly erased areas of the second audio data can be treated as noise interference, thereby enhancing the diversity of the audio data.

[0124] S505: Input the third audio data into the voiceprint distribution prediction network in the teacher model to obtain the first voiceprint distribution probability corresponding to the third audio data output by the teacher model.

[0125] The voiceprint probability distribution prediction network may include a feature extraction network, a feature processing network, a fully connected layer, and a logistic regression model (SoftMax).

[0126] In the embodiment of the present disclosure, the third audio data can be input into the voiceprint distribution prediction network, and the feature extraction network, feature processing network and fully connected layer in the voiceprint distribution prediction network process the third audio data in sequence, and finally SoftMax outputs the first voiceprint distribution probability.

[0127] S506: Input the second audio data into the random erasure network in the student model to process the second audio data based on the second random erasure parameter group to obtain processed fourth audio data.

[0128] It can be understood that the multiple random numbers in the first random erasure parameter group and the duration parameters and frequency parameters corresponding to each random number may be different from the multiple random numbers in the second random erasure parameter group and the duration parameters and frequency parameters corresponding to each random number. Therefore, the fourth audio data obtained by erasing the second audio data based on the second random erasure parameter group is different from the third audio data obtained by erasing the second audio data based on the first random erasure parameter group.

[0129] It should be noted that the specific implementation form of processing the second audio data based on the second random erasure parameter group can refer to the detailed description of processing the second audio data based on the first random erasure parameter group in the embodiment of the present disclosure, and will not be repeated here.

[0130] S507: Input the fourth audio data into the initial student model to obtain the second voiceprint distribution probability and prediction label corresponding to the fourth audio data output by the initial student model.

[0131] It can be understood that the input of the initial student model is the fourth audio data obtained by randomly erasing the second audio data, and the input of the teacher model is the third audio data obtained by randomly erasing the second audio data. The third audio data and the fourth audio data are not exactly the same. Therefore, even if the same second audio data is used, differences can be introduced into the input data of the teacher model and the initial student model.

[0132] S508: According to the difference between the predicted label and the annotated label corresponding to the first audio data, and the difference between the second voiceprint distribution probability and the first voiceprint distribution probability, the initial student model is modified to obtain a preset voiceprint recognition model.

[0133] The specific implementation of step S508 can be referred to the detailed description in other embodiments of the present disclosure and will not be described in detail here.

[0134] In the embodiment of the present disclosure, the randomly erased area of ​​the second audio data can be regarded as noise interference, and the initial student model is trained with the third audio data containing noise interference and the fourth audio data, thereby enhancing the anti-interference ability of the generated voiceprint recognition model.

[0135] In the disclosed embodiment, a first training data set and a generated teacher model are first obtained. The second audio data is then processed based on a first random erasure parameter set to obtain processed third audio data. The third audio data is input into the voiceprint distribution prediction network in the teacher model to obtain the first voiceprint distribution probability corresponding to the third audio data output by the teacher model. The second audio data is then processed based on a second random erasure parameter set to obtain processed fourth audio data. The fourth audio data is input into the initial student model to obtain the second voiceprint distribution probability and predicted label corresponding to the fourth audio data output by the initial student model. Finally, based on the difference between the predicted label and the annotated label corresponding to the first audio data, and the difference between the second voiceprint distribution probability and the first voiceprint distribution probability, the initial student model is modified to obtain a preset voiceprint recognition model. Thus, the second audio data is randomly erased, and the initial student model is trained based on the randomly erased third and fourth audio data to obtain a voiceprint recognition model, thereby improving the anti-interference ability of the voiceprint recognition model and providing conditions for accurately obtaining the first and second voiceprint features corresponding to the first audio data.

[0136] Figure 6 FIG. 1 is a flow chart of a voiceprint recognition method according to another embodiment of the present disclosure. Figure 6 As shown, the voiceprint recognition method includes:

[0137] S601: Obtain a first training data set and a generated teacher model, wherein the first training data set is a text-related audio data set, and the teacher model is generated based on a text-independent second training data set.

[0138] The specific implementation of step S601 can refer to the detailed description in other embodiments of the present disclosure and will not be described in detail here.

[0139] S602: Input the i-th frame of audio in the second audio data into the teacher model to obtain the third voiceprint feature and the first voiceprint distribution probability output by the teacher model, where i is a positive integer.

[0140] Optionally, the teacher model may also include a feature processing network. Therefore, after the i-th frame audio in the second audio data is input into the teacher model, the teacher model can output the third voiceprint feature through the feature processing network and output the first voiceprint distribution probability through the logistic regression model SoftMax in the teacher model.

[0141] S603: Input the i-th audio frame in the second audio data into the feature extraction network of the initial student model to obtain the fourth audio feature output by the feature extraction network.

[0142] S604: Obtain a third reference voiceprint feature corresponding to the i-th frame of audio.

[0143] Optionally, when the i-th frame is an initial frame in the second audio data, the third preset feature is determined as the third reference voiceprint feature.

[0144] Alternatively, when the i-th frame is not the initial frame in the first audio data, the fourth voiceprint feature corresponding to the i-1-th frame output by the initial student model is determined as the third reference voiceprint feature, so that the voiceprint features of each frame of audio can be closely combined.

[0145] S605: Input the fourth audio feature, the third voiceprint feature and the third reference voiceprint feature into the feature processing network in the initial student model to obtain the fourth voiceprint feature and the fifth voiceprint feature corresponding to the i-th frame audio.

[0146] Optionally, the feature processing network can be a multi-head attention network MSA. Therefore, the fourth audio feature, the third voiceprint feature and the third reference voiceprint feature can be input into the multi-head attention network MSA in the initial student model to obtain the fourth voiceprint feature and the fifth voiceprint feature corresponding to the i-th frame audio.

[0147] It should be noted that the use of the multi-head attention network MSA to process the fourth audio feature, the third voiceprint feature and the third reference voiceprint feature can reflect the importance relationship between the second audio feature, the first reference voiceprint feature and the second reference voiceprint feature.

[0148] Alternatively, the feature processing network can also be average-pooling. Therefore, the fourth audio feature, the third voiceprint feature and the third reference voiceprint feature can also be input into the average-pooling in the initial student model to obtain the fourth voiceprint feature and the fifth voiceprint feature corresponding to the i-th frame audio.

[0149] S606: Input the fourth voiceprint feature and the fifth voiceprint feature corresponding to the i-th frame audio into the first subnetwork and the second subnetwork in the initial student model respectively to obtain the second voiceprint distribution probability corresponding to the i-th frame audio output by the first subnetwork and the predicted label output by the second subnetwork.

[0150] Optionally, the fourth voiceprint feature is input into the first subnetwork in the initial student model to obtain the second voiceprint distribution probability corresponding to the i-th frame audio output by the first subnetwork, and the fifth voiceprint feature is input into the second subnetwork in the initial student model to obtain the predicted label corresponding to the i-th frame audio output by the first subnetwork.

[0151] S607: According to the difference between the predicted label and the annotated label corresponding to the first audio data, and the difference between the second voiceprint distribution probability and the first voiceprint distribution probability, the initial student model is modified to obtain a preset voiceprint recognition model.

[0152] Optionally, the initial student model may include a first sub-network and a second sub-network. Thus, the parameters of the first sub-network in the initial student model may be modified based on the difference between the second voiceprint distribution probability and the first voiceprint distribution probability to obtain a voiceprint distribution probability sub-network. The parameters of the second sub-network in the initial student model may be modified based on the difference between the predicted label and the annotated label corresponding to the first audio data to obtain a user label sub-network.

[0153] Optionally, a relative entropy loss function may be used to determine the difference between the second voiceprint distribution probability and the first voiceprint distribution probability; and a cross entropy loss function may be used to determine the difference between the predicted label and the annotated label corresponding to the first audio data.

[0154] In the embodiment of the present disclosure, a first training data set and a generated teacher model are first obtained, and then the i-th frame audio in the second audio data is input into the teacher model to obtain the third voiceprint feature and the first voiceprint distribution probability, and the i-th frame audio in the second audio data is input into the feature extraction network of the initial student model to obtain the fourth audio feature output by the feature extraction network, and then the fourth audio feature, the third voiceprint feature and the third reference voiceprint feature are input into the feature processing network in the initial student model to obtain the fourth voiceprint feature and the fifth voiceprint feature corresponding to the i-th frame audio, and the fourth voiceprint feature and the fifth voiceprint feature corresponding to the i-th frame audio are respectively input into the first sub-network and the second sub-network of the initial student model to obtain the second voiceprint distribution probability and the predicted label corresponding to the i-th frame audio, and finally the initial student model is corrected according to the difference between the predicted label and the labeled label corresponding to the first audio data, and the difference between the second voiceprint distribution probability and the first voiceprint distribution probability to obtain a preset voiceprint recognition model. Therefore, the third voiceprint feature and the first voiceprint distribution probability feature output by the teacher model participate in the training process of the initial student model, and the fourth voiceprint feature and the fifth voiceprint feature corresponding to the i-th frame audio are jointly determined by the fourth audio feature, the third voiceprint feature and the third reference voiceprint feature, so that not only the voiceprint features of each frame of audio can be closely combined, but also when the first training data set contains a small amount of text-related audio data, the generated voiceprint recognition model can also have a strong ability to distinguish the features of the audio data, thereby providing conditions for accurately obtaining the first voiceprint feature and the second voiceprint feature corresponding to the first audio data.

[0155] Figure 7 FIG. 1 is a flow chart of a voiceprint recognition method according to another embodiment of the present disclosure. Figure 7 As shown, the voiceprint recognition method includes:

[0156] First, the second audio data is framed, and the second audio features corresponding to each frame of audio are extracted. Then, based on the first random erasure parameter group, the second audio features corresponding to the i-th frame of audio are processed to obtain the processed third audio data; content features are extracted from the second audio features corresponding to the i-th frame of audio to determine the second content features corresponding to the i-th frame of audio; the third audio data and the second content features are fused and input into the feature extraction network in the teacher model, and the extracted audio features are input into the feature processing network in the teacher model to obtain the third voiceprint features output by the feature processing network; after the third voiceprint features are processed by the fully connected layer and SoftMax, the first voiceprint distribution probability is output.

[0157] Based on the second random erasure parameter group, the second audio feature corresponding to the i-th frame audio is processed to obtain the processed fourth audio data; the second audio feature corresponding to the i-th frame audio is extracted with content features to determine the second content feature corresponding to the i-th frame audio; the fourth audio data and the second content feature are fused and input into the feature extraction network of the initial student model to obtain the fourth audio feature corresponding to the i-th frame audio output by the feature extraction network; the fourth audio feature, the third voiceprint feature output by the teacher network, and the third reference voiceprint feature (which can be the fourth voiceprint feature corresponding to the i-1-th frame audio) are input into the feature processing network to obtain the fourth voiceprint feature and the fifth voiceprint feature corresponding to the i-th frame audio; the fourth audio feature and the third voiceprint feature output by the teacher network are input into the feature processing network to obtain the fourth voiceprint feature and the fifth voiceprint feature corresponding to the i-th frame audio; The voiceprint feature is input into the fully connected layer of the first sub-network in the initial student model, and the second voiceprint distribution probability is output by SoftMax of the first sub-network. The fifth voiceprint feature is input into the fully connected layer of the second sub-network in the initial student model, and the predicted label is output by SoftMax of the second sub-network. The difference between the second voiceprint distribution probability and the first voiceprint distribution probability is determined according to the relative entropy loss function, and the difference between the predicted label and the labeled label corresponding to the first audio data is determined according to the cross entropy loss function. Finally, according to the difference between the predicted label and the labeled label corresponding to the first audio data, and the difference between the second voiceprint distribution probability and the first voiceprint distribution probability, the initial student model is corrected to obtain the preset voiceprint recognition model.

[0158] Figure 8 FIG. 1 is a structural diagram of a voiceprint recognition device according to another embodiment of the present disclosure. Figure 8 As shown, the voiceprint recognition device 800 includes: a first acquisition module 810 , a second acquisition module 820 and a first determination module 830 .

[0159] The first acquisition module 810 is configured to acquire first audio data to be recognized;

[0160] A second acquisition module 820 is configured to input the first audio data into a preset voiceprint recognition model to obtain a first voiceprint feature and a second voiceprint feature output by the voiceprint recognition model, wherein the preset voiceprint recognition model is generated by training based on a teacher model for identifying voiceprint distribution probability and training data labeled with user labels;

[0161] The first determining module 830 is configured to determine a target voiceprint feature corresponding to the first audio data according to the first voiceprint feature and the second voiceprint feature.

[0162] Optionally, the second acquisition module 820 is specifically configured to:

[0163] Inputting the first audio data into an audio feature extraction network in a preset voiceprint recognition model to determine a first audio feature corresponding to the first audio data;

[0164] Inputting the first audio feature into a content feature extraction network in the preset voiceprint recognition model to determine a first content feature corresponding to the first audio data;

[0165] The first audio feature and the first content feature are input into a voiceprint recognition network in a preset voiceprint recognition model to obtain a first voiceprint feature and a second voiceprint feature output by the voiceprint recognition model.

[0166] Optionally, the second acquisition module 820 includes:

[0167] A first determining unit is configured to sequentially input each audio frame in the first audio data into a feature extraction network in a preset voiceprint recognition model to determine a second audio feature corresponding to each audio frame;

[0168] A first acquiring unit is configured to acquire a first reference voiceprint feature and a second reference voiceprint feature corresponding to each frame of audio;

[0169] The second acquisition unit is used to input the second audio feature, the first reference voiceprint feature and the second reference voiceprint feature into the feature processing network in the preset voiceprint recognition model to obtain the first voiceprint feature and the second voiceprint feature corresponding to each frame of audio.

[0170] Optionally, the first acquiring unit is specifically configured to:

[0171] When the current processing frame is the first frame, the first preset feature and the second preset feature are respectively determined as the first reference voiceprint feature and the second reference voiceprint feature corresponding to the current processing frame; and / or,

[0172] When the current processing frame is not the first frame, the first voiceprint feature corresponding to the previous frame and the second voiceprint feature corresponding to the previous frame output by the preset voiceprint recognition model are respectively determined as the first reference voiceprint feature and the second reference voiceprint feature corresponding to the current processing frame.

[0173] Optionally, also include:

[0174] A third acquisition module is configured to acquire a first training data set and a generated teacher model, wherein the first training data set is a text-related audio data set, and the teacher model is generated based on a text-independent second training data set;

[0175] a fourth acquisition module, configured to input the second audio data in the first training data set into the teacher model and the initial student model, respectively, to obtain the first voiceprint distribution probability output by the teacher model, the second voiceprint distribution probability output by the initial student model, and the predicted label output by the initial student model;

[0176] The fifth acquisition module is used to modify the initial student model according to the difference between the predicted label and the marked label corresponding to the first audio data, and the difference between the second voiceprint distribution probability and the first voiceprint distribution probability, so as to obtain a preset voiceprint recognition model.

[0177] Optionally, the fourth acquisition module is specifically configured to:

[0178] Inputting the second audio data into the random erasure network in the teacher model to process the second audio data based on the first random erasure parameter group to obtain processed third audio data;

[0179] Inputting the third audio data into the voiceprint distribution prediction network in the teacher model to obtain the first voiceprint distribution probability corresponding to the third audio data output by the teacher model;

[0180] Inputting the second audio data into the random erasure network in the initial student model to process the second audio data based on the second random erasure parameter group to obtain processed fourth audio data;

[0181] The fourth audio data is input into the initial student model to obtain the second voiceprint distribution probability and predicted label corresponding to the fourth audio data output by the initial student model.

[0182] Optionally, the fourth acquisition module is further specifically configured to:

[0183] determining the number of random numbers to be generated according to the duration of the second audio data;

[0184] Based on the number of random numbers to be generated, a first random erasure parameter group and a second random erasure parameter group are generated, wherein the first random erasure parameter group and the second random parameter group respectively include multiple random numbers and a duration parameter corresponding to each random number and a frequency parameter corresponding to each random number.

[0185] Optionally, the fourth acquisition module includes:

[0186] a third acquiring unit, configured to input the i-th audio frame in the second audio data into the teacher model to acquire a third voiceprint feature and a first voiceprint distribution probability output by the teacher model, where i is a positive integer;

[0187] A fourth acquisition unit is configured to input the i-th audio frame in the second audio data into the feature extraction network of the initial student model to obtain a fourth audio feature output by the feature extraction network;

[0188] A fifth acquiring unit, configured to acquire a third reference voiceprint feature corresponding to the i-th frame of audio;

[0189] a sixth acquisition unit, configured to input the fourth audio feature, the third voiceprint feature, and the third reference voiceprint feature into a feature processing network in the initial student model to obtain a fourth voiceprint feature and a fifth voiceprint feature corresponding to the i-th frame of audio;

[0190] The seventh acquisition unit is used to input the fourth voiceprint feature and the fifth voiceprint feature corresponding to the i-th frame audio into the first subnetwork and the second subnetwork in the initial student model, respectively, to obtain the second voiceprint distribution probability corresponding to the i-th frame audio output by the first subnetwork and the predicted label output by the second subnetwork.

[0191] Optionally, the sixth acquiring unit is specifically configured to:

[0192] The fourth audio feature, the third voiceprint feature and the third reference voiceprint feature are input into the multi-head attention network in the initial student model.

[0193] Optionally, the fifth acquiring unit is specifically configured to:

[0194] When the i-th frame is an initial frame in the second audio data, determining the third preset feature as a third reference voiceprint feature; and / or

[0195] In the case that the i-th frame is not the initial frame in the first audio data, the fourth voiceprint feature corresponding to the i-1-th frame output by the initial student model is determined as the third reference voiceprint feature.

[0196] It should be noted that the above explanation of the voiceprint recognition method is also applicable to the voiceprint recognition device of this embodiment and will not be repeated here.

[0197] In the disclosed embodiment, first audio data to be recognized is acquired. The first audio data is then input into a preset voiceprint recognition model to obtain the first and second voiceprint features output by the voiceprint recognition model. Finally, the target voiceprint features corresponding to the first audio data are determined based on the first and second voiceprint features. Thus, by performing voiceprint recognition on audio data from two perspectives, namely, voiceprint distribution probability and corresponding user labels, the accuracy of the acquired target voiceprint features is improved.

[0198] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0199] Figure 9 A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0200] like Figure 9 As shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the device 900 can also be stored in the RAM 903. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0201] Various components in the device 900 are connected to the I / O interface 905, including an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a magnetic disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0202] The computing unit 901 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as the voiceprint recognition method. For example, in some embodiments, the voiceprint recognition method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the voiceprint recognition method described above can be performed. Alternatively, in other embodiments, the computing unit 901 can be configured to perform the voiceprint recognition method in any other appropriate manner (e.g., via firmware).

[0203] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0204] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0205] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0206] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0207] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0208] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. This client-server relationship is established by computer programs running on the respective computers, establishing a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and VPS services ("Virtual Private Servers" or simply "VPS"). The server may also be a server in a distributed system or a server integrated with blockchain.

[0209] In the disclosed embodiment, first audio data to be recognized is acquired. The first audio data is then input into a preset voiceprint recognition model to obtain the first and second voiceprint features output by the voiceprint recognition model. Finally, the target voiceprint features corresponding to the first audio data are determined based on the first and second voiceprint features. Thus, by performing voiceprint recognition on audio data from two perspectives, namely, voiceprint distribution probability and corresponding user labels, the accuracy of the acquired target voiceprint features is improved.

[0210] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0211] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present disclosure, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In the description of the present disclosure, the words "if" and "if" used can be interpreted as "at the time of" or "when" or "in response to a determination" or "under the circumstances of".

[0212] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A voiceprint recognition method, comprising: Acquire first audio data to be recognized; Inputting the first audio data into a preset voiceprint recognition model to obtain a first voiceprint feature and a second voiceprint feature output by the voiceprint recognition model, wherein the preset voiceprint recognition model is generated by training based on a teacher model for identifying voiceprint distribution probability and training data labeled with user labels; determining a target voiceprint feature corresponding to the first audio data based on the first voiceprint feature and the second voiceprint feature; Before inputting the first audio data into a preset voiceprint recognition model, the method further includes: Obtaining a first training data set and a generated teacher model, wherein the first training data set is a text-related audio data set, and the teacher model is generated based on a text-independent second training data set; Input the second audio data in the first training data set into the teacher model and the initial student model, respectively, to obtain the first voiceprint distribution probability output by the teacher model, the second voiceprint distribution probability output by the initial student model, and the predicted label output by the initial student model; According to the difference between the predicted label and the marked label corresponding to the first audio data, and the difference between the second voiceprint distribution probability and the first voiceprint distribution probability, the initial student model is modified to obtain the preset voiceprint recognition model.

2. The method according to claim 1, wherein Inputting the first audio data into a preset voiceprint recognition model to obtain a first voiceprint feature and a second voiceprint feature output by the voiceprint recognition model includes: Inputting the first audio data into the audio feature extraction network in the preset voiceprint recognition model to determine a first audio feature corresponding to the first audio data; Inputting the first audio feature into a content feature extraction network in the preset voiceprint recognition model to determine a first content feature corresponding to the first audio data; The first audio feature and the first content feature are input into a voiceprint recognition network in the preset voiceprint recognition model to obtain a first voiceprint feature and a second voiceprint feature output by the voiceprint recognition model.

3. The method according to claim 1, wherein Inputting the first audio data into a preset voiceprint recognition model to obtain a first voiceprint feature and a second voiceprint feature output by the voiceprint recognition model includes: Inputting each frame of audio in the first audio data into a feature extraction network in a preset voiceprint recognition model in sequence to determine a second audio feature corresponding to each frame of audio; Obtaining a first reference voiceprint feature and a second reference voiceprint feature corresponding to each frame of the audio; The second audio feature, the first reference voiceprint feature and the second reference voiceprint feature are input into the feature processing network in the preset voiceprint recognition model to obtain the first voiceprint feature and the second voiceprint feature corresponding to each frame of the audio.

4. The method according to claim 3, wherein: The obtaining of the first reference voiceprint feature and the second reference voiceprint feature corresponding to each frame of the audio includes: When the current processing frame is the first frame, the first preset feature and the second preset feature are respectively determined as the first reference voiceprint feature and the second reference voiceprint feature corresponding to the current processing frame; and / or, When the current processing frame is not the first frame, the first voiceprint feature corresponding to the previous frame and the second voiceprint feature corresponding to the previous frame output by the preset voiceprint recognition model are respectively determined as the first reference voiceprint feature and the second reference voiceprint feature corresponding to the current processing frame.

5. The method according to claim 1, wherein The step of inputting the second audio data in the first training data set into the teacher model and the initial student model respectively includes: Inputting the second audio data into the random erasure network in the teacher model to process the second audio data based on the first random erasure parameter group to obtain processed third audio data; Inputting the third audio data into the voiceprint distribution prediction network in the teacher model to obtain the first voiceprint distribution probability corresponding to the third audio data output by the teacher model; Inputting the second audio data into the random erasure network in the initial student model to process the second audio data based on a second random erasure parameter group to obtain processed fourth audio data; The fourth audio data is input into the initial student model to obtain the second voiceprint distribution probability and predicted label corresponding to the fourth audio data output by the initial student model.

6. The method according to claim 5, wherein: Also includes: determining the number of random numbers to be generated according to the duration of the second audio data; Based on the number of random numbers to be generated, the first random erasure parameter group and the second random erasure parameter group are generated, wherein the first random erasure parameter group and the second random parameter group respectively include multiple random numbers and a duration parameter corresponding to each of the random numbers and a frequency parameter corresponding to each of the random numbers.

7. The method of claim 1, wherein: Inputting the second audio data in the first training data set into the teacher model and the initial student model respectively to obtain the first voiceprint distribution probability output by the teacher model and the second voiceprint distribution probability and predicted label output by the initial student model includes: Inputting the i-th frame of audio in the second audio data into the teacher model to obtain the third voiceprint feature and the first voiceprint distribution probability output by the teacher model, where i is a positive integer; Inputting the i-th audio frame in the second audio data into the feature extraction network of the initial student model to obtain a fourth audio feature output by the feature extraction network; Obtaining a third reference voiceprint feature corresponding to the i-th frame of audio; Inputting the fourth audio feature, the third voiceprint feature, and the third reference voiceprint feature into the feature processing network of the initial student model to obtain the fourth voiceprint feature and the fifth voiceprint feature corresponding to the i-th frame of audio; The fourth voiceprint feature and the fifth voiceprint feature corresponding to the i-th frame audio are respectively input into the first subnetwork and the second subnetwork in the initial student model to obtain the second voiceprint distribution probability corresponding to the i-th frame audio output by the first subnetwork and the predicted label output by the second subnetwork.

8. The method of claim 7, wherein: Inputting the fourth audio feature, the third voiceprint feature, and the third reference voiceprint feature into the feature processing network in the initial student model includes: The fourth audio feature, the third voiceprint feature and the third reference voiceprint feature are input into the multi-head attention network in the initial student model.

9. The method of claim 7, wherein: The obtaining of a third reference voiceprint feature corresponding to the i-th frame of audio includes: In a case where the i-th frame is an initial frame in the second audio data, determining a third preset feature as the third reference voiceprint feature; and / or In a case where the i-th frame is not the initial frame in the first audio data, the fourth voiceprint feature corresponding to the i-1-th frame output by the initial student model is determined as the third reference voiceprint feature.

10. A voiceprint recognition device, comprising: A first acquisition module, configured to acquire first audio data to be recognized; a second acquisition module, configured to input the first audio data into a preset voiceprint recognition model to obtain first and second voiceprint features output by the voiceprint recognition model, wherein the preset voiceprint recognition model is generated by training based on a teacher model for identifying voiceprint distribution probability and training data annotated with user labels; a first determining module, configured to determine a target voiceprint feature corresponding to the first audio data based on the first voiceprint feature and the second voiceprint feature; A third acquisition module is configured to acquire a first training data set and a generated teacher model, wherein the first training data set is a text-related audio data set, and the teacher model is generated based on a text-independent second training data set; a fourth acquisition module, configured to input the second audio data in the first training data set into the teacher model and the initial student model, respectively, to obtain the first voiceprint distribution probability output by the teacher model, the second voiceprint distribution probability output by the initial student model, and the predicted label output by the initial student model; The fifth acquisition module is used to correct the initial student model according to the difference between the predicted label and the marked label corresponding to the first audio data, and the difference between the second voiceprint distribution probability and the first voiceprint distribution probability, so as to obtain the preset voiceprint recognition model.

11. The device according to claim 10, wherein The second acquisition module is further configured to: Inputting the first audio data into the audio feature extraction network in the preset voiceprint recognition model to determine a first audio feature corresponding to the first audio data; Inputting the first audio feature into a content feature extraction network in the preset voiceprint recognition model to determine a first content feature corresponding to the first audio data; The first audio feature and the first content feature are input into a voiceprint recognition network in the preset voiceprint recognition model to obtain a first voiceprint feature and a second voiceprint feature output by the voiceprint recognition model.

12. The device of claim 10, wherein: The second acquisition module includes: a first determining unit, configured to sequentially input each frame of audio in the first audio data into a feature extraction network in a preset voiceprint recognition model to determine a second audio feature corresponding to each frame of audio; a first acquiring unit, configured to acquire a first reference voiceprint feature and a second reference voiceprint feature corresponding to each frame of the audio; The second acquisition unit is used to input the second audio feature, the first reference voiceprint feature and the second reference voiceprint feature into the feature processing network in the preset voiceprint recognition model to obtain the first voiceprint feature and the second voiceprint feature corresponding to each frame of the audio.

13. The device of claim 12, wherein: The first acquiring unit is further configured to: When the current processing frame is the first frame, the first preset feature and the second preset feature are respectively determined as the first reference voiceprint feature and the second reference voiceprint feature corresponding to the current processing frame; and / or, When the current processing frame is not the first frame, the first voiceprint feature corresponding to the previous frame and the second voiceprint feature corresponding to the previous frame output by the preset voiceprint recognition model are respectively determined as the first reference voiceprint feature and the second reference voiceprint feature corresponding to the current processing frame.

14. The apparatus of claim 10, wherein: The fourth acquisition module is further configured to: Inputting the second audio data into the random erasure network in the teacher model to process the second audio data based on the first random erasure parameter group to obtain processed third audio data; Inputting the third audio data into the voiceprint distribution prediction network in the teacher model to obtain the first voiceprint distribution probability corresponding to the third audio data output by the teacher model; Inputting the second audio data into the random erasure network in the initial student model to process the second audio data based on a second random erasure parameter group to obtain processed fourth audio data; The fourth audio data is input into the initial student model to obtain the second voiceprint distribution probability and predicted label corresponding to the fourth audio data output by the initial student model.

15. The apparatus of claim 14, wherein: The fourth acquisition module is further configured to: determining the number of random numbers to be generated according to the duration of the second audio data; Based on the number of random numbers to be generated, the first random erasure parameter group and the second random erasure parameter group are generated, wherein the first random erasure parameter group and the second random parameter group respectively include multiple random numbers and a duration parameter corresponding to each of the random numbers and a frequency parameter corresponding to each of the random numbers.

16. The apparatus of claim 10, wherein: The fourth acquisition module includes: a third acquiring unit, configured to input the i-th audio frame in the second audio data into the teacher model to acquire a third voiceprint feature and a first voiceprint distribution probability output by the teacher model, where i is a positive integer; a fourth acquisition unit, configured to input the i-th audio frame in the second audio data into the feature extraction network of the initial student model to obtain a fourth audio feature output by the feature extraction network; A fifth acquiring unit, configured to acquire a third reference voiceprint feature corresponding to the i-th frame of audio; a sixth acquiring unit, configured to input the fourth audio feature, the third voiceprint feature, and the third reference voiceprint feature into a feature processing network in the initial student model to acquire a fourth voiceprint feature and a fifth voiceprint feature corresponding to the i-th frame of audio; The seventh acquisition unit is used to input the fourth voiceprint feature and the fifth voiceprint feature corresponding to the i-th frame audio into the first subnetwork and the second subnetwork in the initial student model, respectively, to obtain the second voiceprint distribution probability corresponding to the i-th frame audio output by the first subnetwork and the predicted label output by the second subnetwork.

17. The apparatus of claim 16, wherein: The sixth acquiring unit is further configured to: The fourth audio feature, the third voiceprint feature and the third reference voiceprint feature are input into the multi-head attention network in the initial student model.

18. The apparatus of claim 16, wherein: The fifth acquiring unit is further configured to: When the i-th frame is an initial frame in the second audio data, determining a third preset feature as the third reference voiceprint feature; and / or In a case where the i-th frame is not the initial frame in the first audio data, the fourth voiceprint feature corresponding to the i-1-th frame output by the initial student model is determined as the third reference voiceprint feature.

19. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 9.

21. A computer program product comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 9.

Images