Speaker recognition method and system

Abstract

The present invention relates to a speaker recognition method and system. The method comprises obtaining a trained recurrent generative adversarial network; obtaining real-time audio data; extracting frame-level Mel-spectrogram features of the real-time audio data, performing voice activity detection on the extracted frame-level Mel-spectrogram features of the real-time audio data, and determining frame-level Mel-spectrogram features containing speech in the real-time audio data; mapping the frame-level Mel-spectrogram features containing speech in the real-time audio data and the frame-level Mel-spectrogram features containing speech in the registered audio data using a common speech frame-level Mel-spectrogram feature generator in the trained recurrent generative adversarial network, and determining a first output result and a second output result; and determining a recognition result based on the first and second output results and a neural network model of the speaker recognition system. The present invention can improve the recognition effect when recognizing a speaker using common pronunciation.

Description

Technical Field

[0001] The present invention relates to the field of speaker recognition, and in particular to a speaker recognition method and system based on the ordinary pronunciation of "hmm" sound. Background Art

[0002] "Ordinary pronunciations," such as "hmm," are common sounds in spoken conversations that are subject to little subjective control by the speaker. These sounds contain little content information, but they contain a wealth of speaker information. Current speaker recognition systems are mostly based on "ordinary speech," meaning sounds produced by human consciousness with clear phonetic content. These sounds are rich in speaker information and therefore highly suitable for speaker recognition. However, because ordinary speech can be flexibly controlled by the speaker, it exhibits a high degree of randomness.

[0003] However, due to physiological characteristics and pronunciation habits, speakers have limited control over ordinary pronunciation. For example, people use the "hmm" sound when expressing doubt or uncertainty, as well as the "um" sound when expressing affirmation. This results in less randomness in ordinary pronunciation, making it suitable for specialized scenarios such as forensic identification. However, datasets for ordinary pronunciation are typically much smaller than datasets for normal speech. For example, due to the small number of speakers and the small number of sentences per speaker, these datasets are insufficient for building and training a system. The performance of a speaker recognition system trained directly with normal speech datasets is equivalent to that of a speaker recognition system trained with normal speech datasets. Speaker recognition systems trained directly with normal speech datasets have poor recognition performance for ordinary pronunciation.

[0004] Based on the above problems, there is an urgent need to provide a speaker recognition method or system based on the ordinary pronunciation "hmm" sound, so as to improve the accuracy of speaker recognition using the ordinary pronunciation "hmm" sound. Summary of the Invention

[0005] The object of the present invention is to provide a speaker recognition method and system, which can improve the recognition effect when using the ordinary pronunciation "hmm" sound to recognize the speaker.

[0006] To achieve the above object, the present invention provides the following solutions:

[0007] A speaker recognition method, comprising:

[0008] Obtaining a trained cyclic generative adversarial network (CycleGAN); wherein a normal speech frame-level mel-spectrogram feature generator in the trained cyclic generative adversarial network (CycleGAN) takes as input a frame-level mel-spectrogram feature of speech containing a common pronunciation of the sound "hmm" and outputs a frame-level mel-spectrogram feature of speech containing a common speech;

[0009] Get real-time audio data;

[0010] Extracting frame-level Mel-spectrogram features of the real-time audio data, and performing voice activity detection on the extracted frame-level Mel-spectrogram features of the real-time audio data to determine whether the real-time audio data contains voice frame-level Mel-spectrogram features;

[0011] Mapping the frame-level Mel-spectrogram features containing speech in the real-time audio data and the frame-level Mel-spectrogram features containing speech in the registered audio data using the trained ordinary speech frame-level Mel-spectrogram feature generator in the recurrent generative adversarial network to determine a first output result and a second output result; the first output result is the frame-level Mel-spectrogram features containing speech in the real-time audio data based on ordinary speech; the second output result is the frame-level Mel-spectrogram features containing speech in the registered audio data based on ordinary speech;

[0012] A recognition result is determined based on the first output result and the second output result and a neural network model of the speaker recognition system; the recognition result indicates that the speaker of the real-time audio data and the speaker of the registered voice data are the same speaker or that the speaker of the real-time audio data and the speaker of the registered voice data are different speakers; the neural network model of the speaker recognition system uses frame-level Mel-spectrogram features containing voice based on ordinary voice as input and uses an embedding layer containing voice based on ordinary voice as output; the first output result and the second output result are respectively passed through the neural network model of the speaker recognition system to obtain a first output embedding layer and a second output embedding layer; cosine similarity is calculated between the two embedding layers; when the cosine similarity exceeds a set threshold, the speaker of the real-time audio data and the speaker of the registered voice data are the same speaker; otherwise, they are different speakers.

[0013] Optionally, the step of obtaining a trained cyclic generative adversarial network further includes:

[0014] Get the dataset dataset1 of normal speech and the dataset dataset2 of the ordinary pronunciation of "hmm" sound;

[0015] Extract the frame-level Mel spectrum features of dataset 1 and dataset 2;

[0016] Perform voice activity detection on the frame-level Mel-spectrogram features of dataset 1 and dataset 2, respectively, to determine the frame-level Mel-spectrogram features containing speech based on ordinary speech in dataset 1 and the frame-level Mel-spectrogram features containing speech based on the ordinary pronunciation of "hmm" in dataset 2;

[0017] Select the frame-level Mel spectrum containing speech whose mean distance between the frame-level Mel spectrum feature of the ordinary pronunciation "hmm" sound in dataset 1 and dataset 2 is less than the distance threshold;

[0018] Taking the frame-level Mel-spectrogram features of speech based on the ordinary pronunciation "hmm" sound in the dataset dataset2 as input and the frame-level Mel-spectrogram features of speech based on ordinary speech in the selected dataset dataset1 as output, the ordinary speech frame-level Mel-spectrogram feature generator in the cyclic generative adversarial network is trained, and then the trained cyclic generative adversarial network is determined.

[0019] Optionally, the trained cyclic generative adversarial network includes: two generative adversarial networks;

[0020] Each generative adversarial network consists of: a generator and a discriminator.

[0021] Optionally, the generator is U-Net.

[0022] Optionally, the neural network model of the speaker recognition system specifically includes: a plurality of stacked convolutional layers connected in sequence, a global average pooling layer, and a fully connected layer; the output of the neural network model of the speaker recognition system is an embedding layer.

[0023] Optionally, determining the recognition result according to the first output result and the second output result and a neural network model of the speaker recognition system specifically includes:

[0024] Determining a corresponding embedding layer according to the first output result and a neural network model of the speaker recognition system;

[0025] Determine and store a corresponding embedding layer based on the second output result and the neural network model of the speaker recognition system;

[0026] The embedded layer corresponding to the stored second output result is compared with the embedded layer corresponding to the first output result using cosine similarity to determine the recognition result.

[0027] A speaker recognition system, comprising:

[0028] A trained recurrent generative adversarial network acquisition module is used to acquire a trained recurrent generative adversarial network; the ordinary speech frame-level Mel-spectrogram feature generator in the trained recurrent generative adversarial network takes as input the frame-level Mel-spectrogram features containing speech based on the ordinary pronunciation of the sound "hmm" and outputs the frame-level Mel-spectrogram features containing speech based on ordinary speech;

[0029] A real-time audio data acquisition module is used to acquire real-time audio data;

[0030] A feature determination module for real-time audio data is used to extract frame-level Mel-spectrogram features of the real-time audio data and perform voice activity detection (VAD) on the extracted frame-level Mel-spectrogram features of the real-time audio data to determine whether the real-time audio data contains frame-level Mel-spectrogram features of speech;

[0031] An output result determination module is configured to map the frame-level Mel-spectrogram features of speech contained in the real-time audio data and the frame-level Mel-spectrogram features of speech contained in the registered audio data using a trained ordinary speech frame-level Mel-spectrogram feature generator in a recurrent generative adversarial network to determine a first output result and a second output result; the first output result is the frame-level Mel-spectrogram features of speech contained in the real-time audio data based on ordinary speech; the second output result is the frame-level Mel-spectrogram features of speech contained in the registered audio data based on ordinary speech;

[0032] A recognition result determination module is configured to determine a recognition result based on a first output result, a second output result, and a neural network model of a speaker recognition system; the recognition result indicates that the speaker of the real-time audio data and the speaker of the registered voice data are the same speaker, or that the speaker of the real-time audio data and the speaker of the registered voice data are different speakers; the neural network model of the speaker recognition system uses frame-level mel-spectrogram features containing speech based on ordinary speech as input and uses an embedding layer containing speech based on ordinary speech as output; the first output result and the second output result are respectively passed through the neural network model of the speaker recognition system to obtain a first output embedding layer and a second output embedding layer; cosine similarity is calculated between the two embedding layers; when the cosine similarity exceeds a set threshold, the speaker of the real-time audio data and the speaker of the registered voice data are the same speaker; otherwise, they are different speakers.

[0033] A speaker recognition system comprises: at least one processor, at least one memory and computer program instructions stored in the memory, wherein when the computer program instructions are executed by the processor, a speaker recognition method is implemented.

[0034] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0035] The present invention provides a speaker recognition method and system. Using a trained cyclic generative adversarial network (RGAN) frame-level mel-spectrogram feature generator for ordinary speech, the RGAN frame-level mel-spectrogram features of speech-containing sounds (hmm) are mapped to RGAN frame-level mel-spectrogram features of speech-containing sounds (hmm). These RGAN frame-level mel-spectrogram features are then mapped to the ordinary speech domain. The neural network model of the speaker recognition system is then used to recognize the mapped RGAN frame-level mel-spectrogram features. This solves the problem in existing speaker recognition systems that are trained directly using ordinary speech datasets and have poor recognition performance for ordinary speech. This improves recognition performance for ordinary speech. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0037] Figure 1 A flow chart of a speaker recognition method provided by the present invention;

[0038] Figure 2 This is the structure diagram of the generator U-Net;

[0039] Figure 3 Schematic diagram of the double convolutional layer module in the generator U-Net;

[0040] Figure 4 Schematic diagram for optimizing the generator and discriminator;

[0041] Figure 5 This is a schematic diagram of the structure of a speaker recognition system provided by the present invention. DETAILED DESCRIPTION

[0042] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0043] The object of the present invention is to provide a speaker recognition method and system, which can improve the recognition effect when using the ordinary pronunciation "hmm" sound to recognize the speaker.

[0044] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0045] Figure 1 A flow chart of a speaker recognition method provided by the present invention is shown in FIG. Figure 1 As shown, the present invention provides a speaker recognition method, comprising:

[0046] S101, obtaining a trained recurrent generative adversarial network; the ordinary speech frame-level Mel-spectrogram feature generator in the trained recurrent generative adversarial network takes the frame-level Mel-spectrogram features containing speech based on the ordinary pronunciation "hmm" sound as input, and takes the frame-level Mel-spectrogram features containing speech based on ordinary speech as output.

[0047] S101 and earlier also included:

[0048] Get the dataset dataset1 of normal speech and the dataset dataset2 of the ordinary pronunciation of "hmm" sound.

[0049] The dataset dataset1 should contain over 1,000 speakers, with an average sentence length of approximately 120 seconds. This dataset will be used to train the x-vector model in the speaker recognition neural network model and as the source domain dataset for CycleGAN. The dataset dataset1 can be CN-celebrity.

[0050] Dataset 2 has fewer speakers than dataset 1. It contains several sentences with the common "hmm" sound, with dozens to hundreds of speakers, and each speaker produces more than five "hmm" sounds. Dataset 2 will serve as the target domain dataset for CycleGAN. Dataset 2 can be CSLT-TRIVIAL-I.

[0051] Extract frame-level Mel-spectrogram features from dataset 1 and dataset 2. Set the frame length to 25ms, the frame shift to 10ms, and the number of Mel-spectrogram filters to 40. Combine each frame with the Mel-spectrograms of the four frames to the left and right to obtain the frame-level Mel-spectrogram features. Thus, one Mel-spectrogram frame consists of nine frames of data.

[0052] Speech activity detection was performed on the frame-level mel-spectrogram features of dataset 1 and dataset 2. The frame-level mel-spectrogram features containing speech based on normal speech in dataset 1 and the frame-level mel-spectrogram features containing speech based on the common pronunciation of the "hmm" sound in dataset 2 were determined. The frame-level mel-spectrogram features were processed using VAD (Vertical Addition) to retain the frame-level mel-spectrograms containing speech. VAD used an energy threshold detection method with an energy threshold of 5.5 and an energy scaling factor of 0.5.

[0053] Select the speech-containing frame-level mel-spectrograms from dataset 1 and dataset 2 whose mean distance between the frame-level mel-spectrogram features of the common "hmm" sound is less than a distance threshold. Cosine similarity can be used to compare distances. When the similarity score exceeds 0.5, the distances are considered close.

[0054] Taking the frame-level Mel-spectrogram features of speech based on the ordinary pronunciation "hmm" sound in the dataset dataset2 as input and the frame-level Mel-spectrogram features of speech based on ordinary speech in the selected dataset dataset1 as output, the ordinary speech frame-level Mel-spectrogram feature generator in the cyclic generative adversarial network is trained, and then the trained cyclic generative adversarial network is determined.

[0055] The trained CycleGAN includes two Generative Adversarial Networks. Each Generative Adversarial Network consists of a generator and a discriminator. The CycleGAN is trained using frame-level mel-spectrogram features containing speech from source and target domain datasets. The generator is trained using identity loss, generative adversarial loss, and cycle-consistency loss, while the discriminator is trained using only the discriminant loss. Both the generator and discriminator are trained using these loss functions.

[0056] The generator is used to map source domain data to target domain data, or vice versa. This is to achieve a mapping between the frame-level mel-spectrogram features of normal source domain speech and the frame-level mel-spectrogram features of the target domain ordinary pronunciation "hmm".

[0057] The generator is U-Net. Figure 2As shown in the figure, the left side shows a stack of downsampling modules, and the right side shows a stack of upsampling modules. After the data passes through the downsampling modules, the width and height of the input data decrease, while the number of channels increases. The downsampling modules consist of a max pooling layer and a double convolutional layer. The max pooling layer has a kernel size of 2×2, a stride s of 2, and a padding size of 0. The output channels of the double convolutional layer vary with the number of layers. After the data passes through the upsampling modules, the width and height of the data increase, while the number of channels decreases. Based on the data flow, the upsampling module consists of a deconvolutional layer, a channel-wise concatenation layer, and a double convolutional module. The deconvolutional layer has a kernel size of 2×2, a number of channels c, a stride s of 2, and a padding size of 0. After passing through all the downsampling modules, the size of the frame-level mel-spectrogram features changes from [1, 40, 9] to [512, 5, 1], where the first dimension represents the number of channels, the second dimension represents the number of mel-spectrogram dimensions, and the third dimension represents the number of time frames. After passing through all the upsampling modules, the data size returns to [1, 40, 9]. Figure 3 This is a double convolutional layer module in a U-Net. The input size is [input_c, h, w]. After the first convolutional layer (conv3×3), the kernel size is 3×3, the number of channels becomes output_c, h and w remain unchanged, and the convolutional layer is followed by a batch normalization (BN) layer and a Rectified Linear Unit (ReLU) layer. After the second convolutional layer (conv3×3), the kernel size is 3×3, followed by a batch normalization (BN) layer and a ReLU layer, and the size does not change. In each module, s represents the stride, pad represents the padding size, and c represents the number of output channels of the convolution.

[0058] The discriminator is used to determine whether the input sample comes from the current source domain or the target domain.

[0059] The discriminator structure is as follows: several layers of convolutional layers (followed by BN layers and ReLU layers) are stacked, then connected to a convolutional layer with an output channel of 1, and finally connected to global average pooling to output only one value.

[0060] The loss function required to train CycleGAN is as follows Figure 4 As shown in the figure, the box represents the generator or discriminator, the solid line represents the result after the data flow passes through the generator or discriminator, and the dotted line represents the data involved in calculating the loss function. Among them, x1 and x2 represent the frame-level Mel spectrum of speech from the ordinary speech dataset dataset1 and the ordinary pronunciation "hmm" sound dataset dataset2 respectively; G 1→2 and G 2→1They are the generators of frame-level Mel spectrum mapping from dataset1 to dataset2, and the generators of frame-level Mel spectrum mapping from dataset2 to dataset1; x′1 and x′2 represent x1 and x2 respectively through G 2→1 and G 1→2 The result of the mapping; and Respectively represent x1 and x2 through G 1→2 and G 2→1 The result of the mapping; and Represents G 2→1 (G 1→2 (x1)) and G 1→2 (G 2→1 (x2)) mapping result; D1 and D2 represent the discriminators that determine whether the input x belongs to the dataset1 or dataset2 domain; y1 and y2 represent the discrimination results of x1 and x2 respectively through the discriminators D1 and D2; and Respectively represent and Each passes the discrimination results of discriminators D1 and D2.

[0061] The loss functions used to train the generator include identity loss L iden , Generate adversarial loss L ady and cycle consistency loss L cycle .like Figure 4 As shown in part (a) of .

[0062] The role of identity loss is that when the input is the target domain data, the output of the generated network should remain unchanged as the target domain data. iden (G 2→1 , x1) as an example, the input is the frame level Mel spectrum x1 of dataset1, the generator G 2→1 For the mapping from dataset2 to dataset1 frame-level Mel spectrum, the output after mapping should remain the original data. L1 loss can be used, and the calculation is as follows:

[0063]

[0064] The role of the generative adversarial loss is that when the input is source domain data, after the generator mapping from the source domain to the target domain, the generated target domain data should be regarded as target domain data. adv (G 1→2 , D2, x1) as an example, that is, the input is the frame level Mel spectrum x1 of dataset1, which passes through the frame level Mel spectrum generator G from dataset1 to dataset21→2 Mapping, when judged by the target domain discriminator D2, it is expected to be judged as belonging to the target domain. MSE loss can be used as follows

[0065]

[0066] The goal of the cycle consistency loss is to use the generator to map the data to the target domain, and when it is remapped back to the source domain, it remains unchanged. Its role is to expect the mapped target domain data to retain some structural information of the source domain, such as semantic information, vocal tract information, etc. cycle (G 1→2 , G 2→1 , x1) as an example, using L1 loss implementation, the calculation is as follows:

[0067]

[0068] Finally, train two generators G 1→2 and G 1→2 The loss function is L(G 1→2 , G 2→1 , D1, D2, x1, x2), as shown below:

[0069] L(G 1→2 , G 2→1 , D1, D2, x1, x2) = λ iden L iden (G 2→1 ,x1)+λ iden L iden (G 1→2 , x2)+λ adv L adv (G 1→2 , D2, x1)+λ edv L adv (G 2→1 , D1, x2)+λ cycle L cyde (G 1→2 , G 2→1 ,x1)+λ cycle L cycle (G 2→1 , G 1→2 , x2)

[0070] where λ iden ,λ adv and λ cycle They are identity loss L iden , Generate adversarial loss L adv and cycle consistency loss L cycle The weight coefficient of . Optional is λ iden =5,λ adv=1,λ cycle =10.

[0071] When training using the above loss function, only the generator G is updated 1→2 and G 2→1 Parameters.

[0072] The role of the discriminator is to determine whether the current x comes from the source domain or from the source domain data generated by the generator. The process of updating the discriminator D1 and D2 is as follows: Figure 4 Part (b) and Figure 4 As shown in part (c), the loss function used for updating is L disc , using MSE loss. Taking D1 as an example, its loss function As shown below:

[0073]

[0074] Among them L disc The 1 subtracted from (D1, x1) represents the true label is 1, because x1 comes from dataset1; L disc (G 2→1 , D1, x2) represents the true label is 0, because x2 comes from dataset2 and passes through the generator G 2→1 Generated It is not really from dataset1. Loss function It is only used to update the discriminator D1, and does not update the generator involved. The update process of the discriminator D2 is similar to that of D1.

[0075] S102, acquiring real-time audio data.

[0076] S103 , extracting frame-level Mel-spectrogram features of the real-time audio data, and performing voice activity detection on the extracted frame-level Mel-spectrogram features of the real-time audio data to determine whether the real-time audio data contains voice frame-level Mel-spectrogram features.

[0077] S104, mapping the frame-level Mel-spectrogram features containing speech in the real-time audio data and the frame-level Mel-spectrogram features containing speech in the registered audio data respectively using the ordinary speech frame-level Mel-spectrogram feature generator in the trained cyclic generative adversarial network to determine a first output result and a second output result; the first output result is the frame-level Mel-spectrogram features containing speech in the real-time audio data based on ordinary speech; the second output result is the frame-level Mel-spectrogram features containing speech in the registered audio data based on ordinary speech.

[0078] S104 specifically includes:

[0079] For real-time audio data, the frame-level Mel spectrum features are extracted and VAD processing is performed, which is then input into the G-CNN trained cyclic generative adversarial network. 2→1 A generator, determining a first output result;

[0080] For the registered audio, the frame-level Mel spectrum features are extracted and VAD processed, which is then input into the G of the trained cyclic generative adversarial network. 2→1 Generator, determines the second output result.

[0081] S105: Determine a recognition result based on the first output result and the second output result and a neural network model of the speaker recognition system; the recognition result indicates that the speaker of the real-time audio data and the speaker of the registered voice data are the same speaker or that the speaker of the real-time audio data and the speaker of the registered voice data are different speakers; the neural network model of the speaker recognition system uses frame-level mel-spectrogram features containing speech based on ordinary speech as input and uses an embedding layer containing speech based on ordinary speech as output; the first output result and the second output result are respectively passed through the neural network model of the speaker recognition system to obtain a first output embedding layer and a second output embedding layer; calculate cosine similarity between the two embedding layers; when the cosine similarity exceeds a set threshold, the speaker of the real-time audio data and the speaker of the registered voice data are the same speaker; otherwise, they are different speakers.

[0082] The neural network model of the speaker recognition system specifically includes: multiple stacked convolutional layers, a global average pooling layer, and a fully connected layer connected in sequence; the output of the neural network model of the speaker recognition system is an embedding layer. The neural network model is trained using FC and softmax.

[0083] S105 specifically includes:

[0084] A corresponding embedding layer is determined according to the first output result and a neural network model of the speaker recognition system.

[0085] According to the second output result and the neural network model of the speaker recognition system, a corresponding embedding layer is determined and stored.

[0086] The embedded layer corresponding to the stored second output result is compared with the embedded layer corresponding to the first output result using cosine similarity to determine the recognition result.

[0087] Figure 5 This is a schematic diagram of the structure of a speaker recognition system provided by the present invention, such as Figure 5 As shown, the present invention provides a speaker recognition system, comprising:

[0088] The trained recurrent generative adversarial network acquisition module 501 is used to obtain a trained recurrent generative adversarial network; the ordinary speech frame-level Mel-spectrogram feature generator in the trained recurrent generative adversarial network takes the frame-level Mel-spectrogram features containing speech based on the ordinary pronunciation of the "hmm" sound as input and takes the frame-level Mel-spectrogram features containing speech based on ordinary speech as output.

[0089] The real-time audio data acquisition module 502 is used to acquire real-time audio data.

[0090] The real-time audio data feature determination module 503 is used to extract frame-level Mel-spectrogram features of the real-time audio data, perform voice activity detection on the extracted frame-level Mel-spectrogram features of the real-time audio data, and determine whether the real-time audio data contains voice frame-level Mel-spectrogram features.

[0091] The output result determination module 504 is used to map the frame-level Mel-spectrogram features containing speech in the real-time audio data and the frame-level Mel-spectrogram features containing speech in the registered audio data using the ordinary speech frame-level Mel-spectrogram feature generator in the trained recurrent generative adversarial network, respectively, to determine a first output result and a second output result; the first output result is the frame-level Mel-spectrogram features containing speech in the real-time audio data based on ordinary speech; the second output result is the frame-level Mel-spectrogram features containing speech in the registered audio data based on ordinary speech.

[0092] The recognition result determination module 505 is configured to determine a recognition result based on the first and second output results and a neural network model of the speaker recognition system. The recognition result indicates that the speaker of the real-time audio data and the speaker of the registered voice data are the same speaker, or that the speaker of the real-time audio data and the speaker of the registered voice data are different speakers. The neural network model of the speaker recognition system uses frame-level mel-spectrogram features containing speech based on ordinary speech as input and uses an embedding layer containing speech based on ordinary speech as output. The first and second output results are respectively passed through the neural network model of the speaker recognition system to obtain a first output embedding layer and a second output embedding layer. Cosine similarity is calculated between the two embedding layers. When the cosine similarity exceeds a set threshold, the speaker of the real-time audio data and the speaker of the registered voice data are the same speaker; otherwise, they are different speakers.

[0093] In order to execute the method corresponding to the above-mentioned embodiment 1 to achieve the corresponding functions and technical effects, the present invention also provides a speaker recognition system, including: at least one processor, at least one memory, and computer program instructions stored in the memory, when the computer program instructions are executed by the processor, a speaker recognition method is implemented.

[0094] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0095] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only intended to help understand the method and core concept of the present invention. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of the present invention. In summary, the contents of this specification should not be construed as limiting the present invention.

Claims

1. A speaker recognition method, characterized in that: include: Obtaining a trained recurrent generative adversarial network; wherein a normal speech frame-level Mel-spectrogram feature generator in the trained recurrent generative adversarial network takes as input a frame-level Mel-spectrogram feature containing speech based on a common pronunciation of the sound "hmm" and outputs a frame-level Mel-spectrogram feature containing speech based on normal speech; Get real-time audio data; Extracting frame-level Mel-spectrogram features of the real-time audio data, and performing voice activity detection on the extracted frame-level Mel-spectrogram features of the real-time audio data to determine whether the real-time audio data contains voice frame-level Mel-spectrogram features; Mapping the frame-level Mel-spectrogram features containing speech in the real-time audio data and the frame-level Mel-spectrogram features containing speech in the registered audio data using the normal speech frame-level Mel-spectrogram feature generator in the trained cyclic generative adversarial network to determine a first output result and a second output result; the first output result is the frame-level Mel-spectrogram features containing speech in the real-time audio data based on normal speech; The second output result is a frame-level Mel-spectrogram feature of the registered audio data containing speech based on ordinary speech; A recognition result is determined based on the first output result and the second output result and a neural network model of the speaker recognition system; the recognition result indicates that the speaker of the real-time audio data and the speaker of the registered voice data are the same speaker or that the speaker of the real-time audio data and the speaker of the registered voice data are different speakers; the neural network model of the speaker recognition system uses frame-level Mel-spectrogram features containing voice based on ordinary voice as input and uses an embedding layer containing voice based on ordinary voice as output; the first output result and the second output result are respectively passed through the neural network model of the speaker recognition system to obtain a first output embedding layer and a second output embedding layer; cosine similarity is calculated between the two embedding layers; when the cosine similarity exceeds a set threshold, the speaker of the real-time audio data and the speaker of the registered voice data are the same speaker; otherwise, they are different speakers.

2. A speaker recognition method according to claim 1, characterized in that: The step of obtaining the trained cyclic generative adversarial network also includes: Get the dataset dataset1 of normal speech and the dataset dataset2 of the ordinary pronunciation of "hmm" sound; Extract the frame-level Mel spectrum features of dataset 1 and dataset 2; Perform voice activity detection on the frame-level Mel-spectrogram features of dataset 1 and dataset 2, respectively, to determine the frame-level Mel-spectrogram features containing speech based on ordinary speech in dataset 1 and the frame-level Mel-spectrogram features containing speech based on the ordinary pronunciation of "hmm" in dataset 2; Select the frame-level Mel-spectrograms containing speech whose mean distance between the frame-level Mel-spectrogram features of the ordinary pronunciation "hmm" sound in dataset 1 and dataset 2 is less than the distance threshold; The frame-level Mel-spectrogram features of speech-containing sounds based on the ordinary pronunciation "hmm" in the dataset dataset2 are used as input, and the frame-level Mel-spectrogram features of speech-containing sounds based on ordinary speech in the selected dataset dataset1 are used as output. A cyclic generative adversarial network is trained to determine the trained cyclic generative adversarial network.

3. A speaker recognition method according to claim 2, characterized in that: The trained cyclic generative adversarial network includes: two generative adversarial networks; Each generative adversarial network consists of: a generator and a discriminator.

4. A speaker recognition method according to claim 3, characterized in that: The generator is U-Net.

5. A speaker recognition method according to claim 1, characterized in that: The neural network model of the speaker recognition system specifically includes: multiple stacked convolutional layers connected in sequence, a global average pooling layer and a fully connected layer; the output of the neural network model of the speaker recognition system is an embedding layer.

6. A speaker recognition method according to claim 5, characterized in that: Determining the recognition result according to the first output result, the second output result, and the neural network model of the speaker recognition system specifically includes: Determining a corresponding embedding layer according to the first output result and a neural network model of the speaker recognition system; Determine and store a corresponding embedding layer based on the second output result and the neural network model of the speaker recognition system; The embedded layer corresponding to the stored second output result is compared with the embedded layer corresponding to the first output result using cosine similarity to determine the recognition result.

7. A speaker recognition system, characterized in that: include: A trained cyclic generative adversarial network acquisition module is used to obtain a trained cyclic generative adversarial network; The ordinary speech frame-level Mel-spectrogram feature generator in the trained cyclic generative adversarial network takes the frame-level Mel-spectrogram feature containing speech based on the ordinary pronunciation "hmm" sound as input and takes the frame-level Mel-spectrogram feature containing speech based on ordinary speech as output; A real-time audio data acquisition module is used to acquire real-time audio data; a feature determination module for real-time audio data, configured to extract frame-level Mel-spectrogram features of the real-time audio data, perform voice activity detection on the extracted frame-level Mel-spectrogram features of the real-time audio data, and determine whether the real-time audio data contains voice frame-level Mel-spectrogram features; An output result determination module is configured to map the frame-level Mel-spectrogram features containing speech in the real-time audio data and the frame-level Mel-spectrogram features containing speech in the registered audio data using a trained ordinary speech frame-level Mel-spectrogram feature generator in a recurrent generative adversarial network to determine a first output result and a second output result; the first output result is the frame-level Mel-spectrogram features containing speech in the real-time audio data based on ordinary speech; The second output result is a frame-level Mel-spectrogram feature of the registered audio data containing speech based on ordinary speech; A recognition result determination module is configured to determine a recognition result based on a first output result, a second output result, and a neural network model of a speaker recognition system; the recognition result indicates that the speaker of the real-time audio data and the speaker of the registered voice data are the same speaker, or that the speaker of the real-time audio data and the speaker of the registered voice data are different speakers; the neural network model of the speaker recognition system uses frame-level mel-spectrogram features containing speech based on ordinary speech as input and uses an embedding layer containing speech based on ordinary speech as output; the first output result and the second output result are respectively passed through the neural network model of the speaker recognition system to obtain a first output embedding layer and a second output embedding layer; cosine similarity is calculated between the two embedding layers; when the cosine similarity exceeds a set threshold, the speaker of the real-time audio data and the speaker of the registered voice data are the same speaker; otherwise, they are different speakers.

8. A speaker recognition system, characterized in that: include: At least one processor, at least one memory, and computer program instructions stored in the memory, wherein when the computer program instructions are executed by the processor, a speaker recognition method according to any one of claims 1 to 6 is implemented.

Images