Method and device for voiceprint information extraction, model training and voiceprint recognition

Abstract

Disclosed is a scheme for voiceprint information extraction, correlation model training and voiceprint recognition. The voiceprint information extraction method comprises: obtaining first speech features of a first speech segment; calculating a similarity function representing the similarity between the first speech features and second speech features; and calculating enhanced voiceprint information incorporating the first speech features and the second speech features according to the similarity function. The present application can utilize multiple pieces of speech information with correlation to obtain enhanced voiceprint information based on their similarity, thereby improving the accuracy of voiceprint recognition. The above recognition can be performed using a voiceprint model trained by a neural network to improve the accuracy and efficiency of voiceprint extraction. The above scheme can be used by a speech interaction device, such as a smart speaker, in a multi-round dialogue in which the single-round dialogue time is too short, thereby effectively enhancing the voiceprint features of the subsequent dialogue using the voiceprint features of the previous round or rounds of speech in the continuous dialogue.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of speech processing, and in particular, to a method and device for voiceprint information extraction, model training and voiceprint recognition. BACKGROUND

[0002] With the rapid development of information technology and network technology, people's demand for identity recognition technology is increasing, and the requirement for its security and reliability is also becoming more and more strict. The identity recognition technology based on traditional password authentication has exposed many shortcomings in the actual information network application, while the identity recognition technology based on biological feature recognition has become mature in recent years and has shown great superiority in practical application. Among them, voiceprint recognition technology is one of the new and more effective identity recognition technologies developed in recent years.

[0003] Voiceprint refers to the information graph of the speaker's speech spectrum. Because the vocal organs of each person are different, the sound and its tone are different, therefore, voiceprint as a basic feature to realize the identity recognition of people has practical irreplaceability and stability, so that voiceprint recognition technology is widely used in various fields of information network. For example, voiceprint recognition can ensure that the smart speaker only reacts to instructions from a specific user. But in some application scenarios, for example, in the latter rounds of continuous dialogue, the user's voice will be relatively short, such as answering "yes" or "good", which increases the difficulty of voiceprint extraction.

[0004] Therefore, a processing technology capable of voiceprint enhancement is needed. SUMMARY

[0005] One of the technical problems to be solved by the present disclosure is to provide a voiceprint information extraction, related model training and voiceprint recognition scheme. The scheme can utilize multiple pieces of voice information with correlation, and obtain enhanced voiceprint information based on the similarity, thereby improving the accuracy of voiceprint recognition.

[0006] According to a first aspect of the present disclosure, a voiceprint information extraction method is provided, comprising: obtaining a first speech feature of a first speech segment; calculating a similarity function representing the similarity between the first speech feature and a second speech feature; and calculating enhanced voiceprint information incorporating the first speech feature and the second speech feature according to the similarity function.

[0007] According to a second aspect of the present disclosure, a voiceprint model training method is provided, comprising: obtaining a pair of training speech segments and corresponding label data, wherein the pair of speech features comprises a reference speech segment and at least two to-be-matched speech segments; performing feature extraction processing on the speech segment data in the pair of training speech segments to obtain a training sample feature set; and training based on the training sample feature set and the corresponding label data using a preset neural network structure to obtain the voiceprint model, wherein in the training process, the similarity of the corresponding speech features of the at least two to-be-matched speech segments is calculated, and enhanced voiceprint information is synthesized according to the similarity; and the matching result of the enhanced voiceprint information and the voiceprint information corresponding to the reference speech segment is taken as the output of the voiceprint model.

[0008] According to a third aspect of the present disclosure, a voiceprint recognition method is provided, comprising: receiving a current speech input of a user; extracting current speech features of the current speech input; calculating a similarity function representing the similarity of the current speech features and previous speech features; calculating enhanced voiceprint information according to the similarity function; and identifying the identity of the user based on the enhanced voiceprint information.

[0009] According to a fourth aspect of the present disclosure, a voiceprint information extraction apparatus is provided, comprising: a feature acquisition unit configured to acquire first speech features of a first speech segment; a similarity calculation unit configured to calculate a similarity function representing the similarity of the first speech features and second speech features; and an enhanced voiceprint calculation unit configured to calculate enhanced voiceprint information incorporating the first speech features and the second speech features according to the similarity function.

[0010] According to a fifth aspect of the present disclosure, a voiceprint model training apparatus is provided, comprising: a data acquisition unit configured to obtain a pair of training speech segments and corresponding label data, wherein the pair of speech features comprises a reference speech segment and at least two to-be-matched speech segments; a feature extraction unit configured to perform feature extraction processing on the speech segment data in the pair of training speech segments to obtain a training sample feature set; and a model training unit configured to train based on the training sample feature set and the corresponding label data using a preset neural network structure to obtain the voiceprint model, wherein in the training process, the similarity of the corresponding speech features of the at least two to-be-matched speech segments is calculated, and enhanced voiceprint information is synthesized according to the similarity; and the matching result of the enhanced voiceprint information and the voiceprint information corresponding to the reference speech segment is taken as the output of the voiceprint model.

[0011] According to a sixth aspect of the present disclosure, a voice interaction device is provided, comprising: a sound collecting module configured to collect sound information; a feature extracting module configured to extract voice features contained in the collected sound information; a voice recognizing module configured to recognize semantics in the voice features; and a voiceprint recognizing module configured to extract a voiceprint from the voice features for voiceprint recognition, wherein the sound collecting module collects sound information from at least two rounds of conversations for synthesizing enhanced voiceprint information based on similarity of the sound information, and the voiceprint recognizing module performs voiceprint recognition based on the enhanced voiceprint information.

[0012] According to a seventh aspect of the present disclosure, an Internet of Things device is provided, comprising: a sound collecting module configured to collect sound information; a voice recognizing module configured to recognize semantics in voice features; and an executing module configured to perform corresponding operations based on the recognized semantics, wherein the sound information collected by the sound collecting module and other sound information collected by other Internet of Things devices are used to synthesize enhanced voiceprint information based on similarity of the sound information for a voiceprint recognizing device to perform voiceprint recognition based on the enhanced voiceprint information.

[0013] According to an eighth aspect of the present disclosure, an Internet of Things device is provided, comprising: a sound collecting module configured to collect sound information from at least two rounds of conversations; a voice recognizing module configured to recognize semantics in voice features; an executing module configured to perform corresponding operations based on the recognized semantics; and a networking module configured to upload the sound information collected by the sound collecting module to a voiceprint recognizing device, wherein the voiceprint recognizing device performs voiceprint recognition according to enhanced voiceprint information synthesized based on similarity of the sound information.

[0014] According to a ninth aspect of the present disclosure, a voice interaction system is provided, comprising: a plurality of voice interaction devices according to the sixth aspect and / or a plurality of Internet of Things devices according to the seventh or eighth aspect; and a server having a voiceprint model trained based on a neural network arranged thereon.

[0015] According to a tenth aspect of the present disclosure, a computing device is provided, comprising: a processor; and a memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method according to the first to third aspects described above.

[0016] According to an eleventh aspect of the present disclosure, a non-transitory machine-readable storage medium is provided, having executable code stored thereon, which, when executed by a processor of an electronic device, causes the processor to perform the method according to the first to third aspects described above.

[0017] Thus, the present application can utilize multiple pieces of voice information with relevance, and obtain enhanced voiceprint information based on the similarity, thereby improving the accuracy of voiceprint recognition. The present application preferably uses a voiceprint model trained by a neural network to obtain enhanced voiceprint information, thereby further improving the accuracy and efficiency of voiceprint extraction. The above method can be used by a voice interaction device, such as a smart speaker, in a multi-round dialogue in which the single-round dialogue time is too short, thereby effectively enhancing the voiceprint features of the subsequent dialogue using the voiceprint features of the previous round or rounds of voice in the continuous dialogue. BRIEF DESCRIPTION OF DRAWINGS

[0018] The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which like reference characters refer to like parts throughout the figures, in which exemplary embodiments of the present disclosure are shown.

[0019] Figure 1 Classification of voiceprint recognition is shown.

[0020] Figure 2 A schematic flowchart of a voiceprint information extraction method according to one embodiment of the present application is shown.

[0021] Figure 3 A schematic flowchart of a voiceprint model training method according to one embodiment of the present application is shown.

[0022] Figure 4 An example of using a neural network to extract voiceprint information is shown.

[0023] Figure 5 An example of a voiceprint model of the present application is shown.

[0024] Figure 6 A schematic flowchart of a voiceprint recognition method according to one embodiment of the present application is shown.

[0025] Figure 7 A schematic composition diagram of a voiceprint information extraction device according to one embodiment of the present application is shown.

[0026] Figure 8 A schematic composition diagram of a voiceprint model training device according to one embodiment of the present application is shown.

[0027] Figure 9 A schematic composition diagram of a voice interaction device according to one embodiment of the present application is shown.

[0028] Figure 10 A structural schematic diagram of a computing device according to one embodiment of the present application, which can be used to implement the above voiceprint-related processing method, is shown. DETAILED DESCRIPTION

[0029] Preferred embodiments of the present disclosure will be described in greater detail below with reference to the accompanying drawings. While preferred embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure is thorough and complete and fully conveys the scope of the present disclosure to those skilled in the art.

[0030] A voiceprint is a sound wave spectrum carrying speech information displayed by an electroacoustic instrument. The production of human language is a complex physiological and physical process between the human language center and the vocal organs. The vocal organs used by a person when speaking (tongue, teeth, larynx, lungs, nasal cavity) differ greatly in size and shape from person to person, so the voiceprints of any two people are different.

[0031] Voiceprint information is information that can represent a speaker and has a high degree of distinction extracted by a filter, a model, etc. It can be used in a voiceprint recognition system. A voiceprint recognition system is an application system based on speech recognition or identification of a speaker. It is a technical system that automatically identifies or identifies the identity of a speaker based on the physiological and behavioral characteristics of the speaker represented by the speech information of the person. Based on different applications of the voiceprint recognition system, the technical implementation of the voiceprint recognition system can basically be divided into two categories. Figure 1 The classification of voiceprint recognition is shown. As shown in Figure 1 , voiceprint recognition technology can be divided into speaker verification technology and speaker identification technology. Speaker verification technology (a) is a technology for determining whether an unknown speaker is a specified person. For example, in the application scenario of voice interaction of a smart speaker, the smart speaker needs to determine whether the input voice input x is the same person as the registered speaker A1. Speaker identification technology (b) is a technology for identifying an unknown speaker as one of the recorded speakers. For example, in the case where a smart speaker supports multi-user switching, the smart speaker needs to determine which person among the registered speakers A1... A N Therefore, the technical problem to be solved by the voiceprint recognition system is embodied in the "one-to-one" matching judgment problem or the "multiple-to-one" comparison judgment problem.

[0032] Voiceprint information can also be used in a speaker adaptive system for speech recognition. Speaker adaptation transforms speaker-dependent speech features into speaker-independent speech features or speaker-independent voiceprint models into speaker-dependent voiceprint models to make the speech features of the target speaker to be tested more matched with the voiceprint model, thereby obtaining better recognition results. An important method for realizing these transformations is to extract the voiceprint information of the speaker's speech.

[0033] Generally, it takes a certain length of speech input of a speaker to extract the voiceprint information, for example, 3s. In some application scenarios, for example, in the last few rounds of continuous dialogue, the user's speech is relatively short, such as answering "yes" or "good". The duration of these speeches is usually less than 1s, which increases the difficulty of voiceprint extraction. The present application uses multiple pieces of speech information with correlation to obtain enhanced voiceprint information based on their similarity, thereby improving the accuracy of voiceprint recognition.

[0034] Figure 2 A schematic flowchart of a voiceprint information extraction method according to an embodiment of the present application is shown. The method can be performed by a device (for example, a smart speaker) that needs to perform voiceprint recognition, and the extracted voiceprint is used to perform speaker confirmation or identification as described above; it can also be used as part of the voiceprint model training as described below. Figure 3 The voiceprint model training part.

[0035] In step S210, the first speech feature of the first speech segment is obtained. Here, the "speech segment" refers to a continuous speech, for example, a sentence, a paragraph, or even a single-word answer. The speech segment needs to be processed by a certain feature extraction process to obtain a feature form that can be used for subsequent voiceprint information extraction. For example, the obtained original speech segment can be subjected to short-time Fourier transform (STFT) to frame and window the one-dimensional time-domain sound signal, and then perform Fourier transform (FFT) on each frame, and finally stack the results of each frame along another dimension to obtain a two-dimensional signal obtained by STFT expansion, i.e., the so-called spectrogram. The spectrogram is often a very large graph, and in order to obtain a sound feature of appropriate size, it is often transformed into a mel spectrum by a mel-scale filter bank. The mel-frequency cepstral analysis (taking the logarithm and performing DCT transform) on the mel spectrum obtains the mel-frequency cepstral coefficient (MFCC). In addition to the mel-frequency cepstral coefficient (MFCC) as described above, the speech feature obtained in the present application can also include perceptual linear prediction coefficient (PLP) or filter bank feature (Filter Bank Feature) and the like. Of course, in some extreme embodiments, the speech acoustic feature can also be the original speech data of the speech segment.

[0036] After the first speech feature of the first speech segment is obtained, a similarity function representing the similarity between the first speech feature and a second speech feature can be calculated at step S220. Here, the second speech feature can be a speech feature extracted from a second speech segment. Generally, the second speech segment is not an arbitrary speech segment, but is usually a segment associated with the first speech segment, i.e., a speech segment from the same speaker with a considerable probability. For example, the temporal association can be that the first speech segment includes the current speech segment and the second speech segment includes the previous speech segment. In particular, in a continuous speech interaction dialogue for a smart speaker or the like, multiple rounds of dialogue are mostly conducted by the same user. Therefore, it is highly probable that the correlation of the previous and subsequent rounds of sound is very high. At this time, the similarity can be further quantified by calculating the similarity function.

[0037] Subsequently, at step S230, enhanced voiceprint information incorporating the first speech feature and the second speech feature can be calculated according to the similarity function. For example, in the case of multiple rounds of dialogue, the speech feature of the previous round can be used to enhance the current speech feature, thereby improving the accuracy of voiceprint extraction. And the degree to which the speech feature of the previous round can enhance the current speech feature can be determined by the similarity function calculated as above. That is, the higher the similarity, the greater the proportion of the second speech feature.

[0038] Thus, the present application can use speech features with high similarity to enhance the current speech feature, thereby obtaining more accurate voiceprint information.

[0039] Here, it should be understood that "first" and "second" are only used to distinguish different instances of the same object, and do not imply their order or importance. Further, in the application scenario of the present application, the second speech feature is often obtained and calculated before the first speech feature.

[0040] In different embodiments, the fusion of the associated speech features can be performed at different stages. For example, in one embodiment, the respective voiceprint features can be obtained first, and the similarity function can be calculated based on the voiceprint features and then merged. In another embodiment, the similarity and merging of the speech features can be performed first, and the enhanced voiceprint feature can be obtained for the enhanced speech feature. In other embodiments, the similarity calculation and merging can also be performed for the intermediate features extracted from the speech features (e.g., MFCC) to the voiceprint features, and then the extraction of the voiceprint features can be completed to obtain the enhanced voiceprint features.

[0041] To this end, Figure 2The method can further include extracting the first voiceprint feature based on the first speech feature, extracting the second voiceprint feature based on the second speech feature, and computing the similarity function representing the similarity between the first speech feature and the second speech feature includes computing the similarity function of the first voiceprint feature and the second voiceprint feature, and computing the enhanced voiceprint information merging the first speech feature and the second speech feature according to the similarity function includes merging the first voiceprint feature and the second voiceprint feature according to the similarity function to obtain the enhanced voiceprint information.

[0042] Alternatively, the similarity computation can be performed first and then the voiceprint is obtained. To this end, the step S230 can include merging the first speech feature and the second speech feature according to the similarity function to obtain enhanced speech feature, and extracting the enhanced voiceprint information based on the enhanced speech feature.

[0043] Specifically, computing the similarity function representing the similarity between the first speech feature and the second speech feature can include computing the similarity function representing the similarity between the first speech feature and the second speech feature using inner product, for example, taking the similarity function as the weight of the second speech feature to obtain the enhanced voiceprint information. In one embodiment, the inner product of the first and second speech features can be computed as the similarity function, the merging of the speech features is performed, and then the enhanced voiceprint information is obtained based on the enhanced speech feature. In another embodiment, the inner product of the first and second voiceprint features can be computed, and the merging of the voiceprint features is directly performed to obtain the enhanced voiceprint information.

[0044] In one embodiment, the present application can extract the voiceprint information of the speech based on the neural network trained voiceprint model, thereby greatly simplifying the extraction and modeling process and improving the quality of voiceprint information extraction. For example, the voiceprint feature can be extracted from the speech feature based on the neural network, and the computation of the similarity function and the extraction of the voiceprint feature can be performed based on the neural network trained voiceprint model. In practical applications, different models can be used to complete different tasks, for example, a voiceprint extraction model can complete the extraction task from the speech feature to the voiceprint feature, a similarity computation and merging model can complete the computation of the similarity function and the extraction of the voiceprint feature, etc. In one embodiment, an end-to-end model can be used to realize the input of the speech feature to the final result output (for example, obtaining the enhanced voiceprint information, and further, identifying the speaker identity based on the enhanced voiceprint information). Since the first stage of the stage-by-stage learning cannot guarantee the optimal solution of the second stage problem, the end-to-end model can be preferred to ensure that the solution of the last stage is optimal.

[0045] As previously described, the voiceprint extraction method of the present application can further comprise identifying the speaker identity based on the enhanced voiceprint information. Specifically, the voiceprint model outputs a probability of whether the enhanced voiceprint information matches the reference voiceprint information based on the enhanced voiceprint information.

[0046] Further, the voiceprint model or other voiceprint recognition model other than the deep neural network can further comprise a down-sampling function. Thus, the voiceprint extraction method of the present application can further comprise: down-sampling the first speech feature to obtain a down-sampled first speech feature; and down-sampling the second speech feature to obtain a down-sampled second speech feature. At this time, the similarity function representing the similarity between the first speech feature and the second speech feature comprises a similarity function representing the similarity between the down-sampled first speech feature and the down-sampled second speech feature. Thus, the difficulty of extracting voiceprint information is reduced, and the computational efficiency is improved. The above-mentioned down-sampling function can be implemented by, for example, a convolutional neural network (CNN).

[0047] As previously described, the voiceprint model for extracting voiceprint information of the present application is preferably trained based on a neural network. Figure 3 A schematic flowchart of a voiceprint model training method according to an embodiment of the present application is shown.

[0048] In step S310, a pair of training speech segments and corresponding label data are obtained, wherein the pair of speech features comprises a reference speech segment and at least two to-be-matched speech segments. The at least two to-be-matched speech segments can be the second speech segment and the first speech segment input to the model in sequence. The corresponding label data can indicate whether the reference speech segment and the at least two to-be-matched speech segments come from the same speaker.

[0049] In step S320, feature extraction processing is performed on the speech segment data in the pair of training speech segments to obtain a training sample feature set.

[0050] In step S330, the preset neural network structure is trained based on the training sample feature set and the corresponding label data to obtain the voiceprint model.

[0051] Unlike conventional voiceprint extraction models, in the training process of the voiceprint model of the present application, the similarity of the corresponding speech features of the at least two to-be-matched speech segments is calculated, and the enhanced voiceprint information is synthesized according to the similarity; and the matching result of the enhanced voiceprint information and the voiceprint information corresponding to the reference speech segment is taken as the output of the voiceprint model.

[0052] In supervised learning, the training of a neural network requires the introduction of a loss function to minimize the loss between the prediction and the label. To this end, the training step S330 comprises adjusting the network parameters of the neural network with a first loss function (regular loss function), wherein the first loss function is used to describe the loss between the matching result and the label data.

[0053] Figure 4 An example of extracting voiceprint information using a neural network is shown. As shown on the left side of the figure, the original speech collected by, for example, a smart speaker can be converted into a spectrogram via a short-time Fourier transform (STFT), and then, via a speech feature processing operation, speech features such as MFCCs are obtained. The above-mentioned speech features are sent to a neural network for voiceprint information extraction.

[0054] As shown, the neural network can include three modules of down-sampling, feature extraction (Transformer), and voiceprint feature representation. Specifically, the neural network includes, from bottom to top, a convolutional neural network layer for down-sampling, a Transformer model, a pooling and fully connected layer (FC), and a normalization layer (Norm), that is, the data entering the neural network passes through the convolutional neural network layer for down-sampling, the Transformer, the pooling and fully connected layer, and then reaches the normalization layer. The fully connected layer includes at least one fully connected network.

[0055] The down-sampling module composed of a convolutional neural network layer (CNN) is an optional module. The addition of the down-sampling module can combine the speech acoustic features of multiple frames into a smaller number of frames, thereby reducing the frame rate so as to more quickly and effectively extract voiceprint features in the subsequent process.

[0056] The feature extraction module adopts the deep learning model Transformer released in 2017. The Transformer model is the mainstream model currently used in the field of natural language processing (NLP). Like a recurrent neural network (RNN), the Transformer is designed to process sequential data (such as natural language) to perform tasks such as translation and text summarization. However, unlike RNN, the Transformer does not need to process sequential data sequentially. For example, if the input data is a natural language sentence, the Transformer does not need to process the beginning of it before the end. Due to this function, the Transformer model facilitates more parallelization in the training process, so it can train on larger datasets compared to before.

[0057] As shown, the Transformer model includes three blocks: Multi-Head Self-Attention, which finds global connections in the input by introducing attention allocation coefficients. This is followed by Add and LayerNorm, a feed-forward layer, and Add and LayerNorm. The features extracted from the Transformer model can then be converted into a voice embedding via a voice feature representation module. The voice feature representation module can include an attention / pooling layer as described above and at least one fully connected layer (shown as three layers) followed by a normalization layer.

[0058] The model can input pairs of speech features as training samples, with data indicating whether the pairs of speech features come from the same speaker as training labels (e.g., 1 for the same person, 0 for different). The model can obtain a voice embedding as shown, and can output the probability that the input pairs of speech features come from the same person as a binary classification. The model can be optimized using a speaker end-to-end loss (e.g., using a cross-entropy loss function).

[0059] It should be understood that the model can also use other feature extraction models other than the Transformer model, such as a bidirectional recurrent neural network layer (bidirectional RNN), e.g., a BiLSTM network including a forward propagation layer and a backward propagation layer, so that the voice acoustic features of the current time frame can be analyzed together with the previous and subsequent time frames in the complete time sequence, making the feature extraction of the voice model more accurate.

[0060] Figure 4 The network structures and properties involved in each layer of the illustrated neural network architecture, such as the Transformer model, convolutional neural network, fully connected network, etc., are all well known to those skilled in the art through existing technology, and will not be described here.

[0061] Further, before extracting voiceprint information, the above neural network needs to be trained. For example, the neural network can be accelerated for voiceprint model training through a GPU or other neural network dedicated hardware, and the method includes: taking the voice acoustic features extracted from the training speech segment as the input data of the neural network; taking the classification label corresponding to the training speech segment as the output data of the neural network; classifying the input data through the neural network, and comparing the classification result with the corresponding relationship of the output data to train the voiceprint model.

[0062] In this case, the speaker can be used as a classification label, that is, one speaker corresponds to at least one training voice segment. The training of the neural network is as follows: a large number of training voice segments are input into the neural network, and the neural network classifies them. If the neural network classifies a certain training voice segment into the corresponding speaker, this classification is marked as 1 (that is, the classification result corresponds to the output data); if the neural network classifies a certain training voice segment into a non-corresponding speaker, this classification is marked as 0 (that is, the classification result does not correspond to the output data). After classifying all the training voice segments, the input data is input into the neural network again for analysis and training according to the classification mark, until the neural network can accurately classify (for example, using a first loss function to adjust the parameters at a certain learning rate), or the classification accuracy exceeds a predetermined threshold, which is used as the voiceprint model.

[0063] In order to obtain enhanced voiceprint information, the present application needs to incorporate the results of being able to calculate a similarity function and perform feature merging on the basis of the neural network shown in Figure 4 The similarity calculation module is incorporated into the voiceprint model.

[0064] To this end, the voiceprint model includes a similarity calculation module. Further, in order to enable the voiceprint model to learn how to obtain the similarity, a continuous similarity loss function can also be added. Thus, training based on the training sample feature set and the corresponding label data using a predetermined neural network structure includes adjusting the network parameters of the neural network using a continuous similarity loss function, wherein the continuous similarity loss function is used to describe the loss between the similarity function and the similarity label data. Here, "continuous similarity" refers to the iterative (continuous) learning of the similarity of the previous multiple rounds of voice features. Here, the continuous similarity loss function can be a cross-entropy loss function.

[0065] Further, the voiceprint model includes a merging module, and the voiceprint model needs to be able to describe the loss function of the enhanced voiceprint extraction effect as a whole. To this end, training based on the training sample feature set and the corresponding label data using a predetermined neural network structure includes adjusting the network parameters of the neural network using a similarity loss function, wherein the similarity loss function is used to describe the loss between the enhanced voiceprint information and the voiceprint label data. Here, the similarity loss function can also be a cross-entropy loss function.

[0066] To this end, Figure 5 An example of the voiceprint model of the present application is shown. As shown in the figure, Figure 5 The voiceprint model includes three modules: a voiceprint extraction module, a similarity calculation module, and a merging module. Here, the voiceprint extraction module is similar to the neural network structure for extracting the voiceprint vector of the current voice input shown in Figure 4 Figure 4 ​As shown, it also includes a downsampling network implemented by a convolutional neural network, a Transformer model for feature extraction, pooling, full connection and normalization layers, and generates a voiceprint vector e(norm) based on the current speech segment. However, as an alternative, in this example, the voiceprint model includes the speech feature extraction function, that is, the spectrogram can be converted into speech features such as MFCC. In other embodiments, the voiceprint model can even include an STFT module so that the original speech segment can be directly used as the model input. Those skilled in the art can choose to use a separate STFT module, MFCC extraction module, or incorporate the above modules into the voiceprint model as needed, and the present invention does not limit this. In addition, this example shows that multiple downsampled speech features (for example, via multiple channels) are fed into the Transformer model in parallel, that is, the multiple side-by-side boxes including ellipsis shown in the figure.

[0067] In addition to the voiceprint extraction model, the voiceprint model of the present invention also includes a similarity calculation module and a merging module, which are used to calculate the similarity and obtain enhanced voiceprint information e(t) based on the calculated similarity.

[0068] The following will be combined Figure 5 An example of a voiceprint enhancement solution based on the present invention is described.

[0069] 1) First, the current speech segment v(t) is transformed via a short-time Fourier transform (STFT) to obtain a spectrogram. This spectrogram is used as the model input, and initial features are extracted based on MFCCs. For example, a frame can be taken every 25ms, with each frame containing 23-dimensional features, and the output is n 23-dimensional features [x1, x2, ..., x23].

[0070] 2) Subsequently, the features extracted by MFCC are downsampled using a convolutional neural network (CNN) with a k*k convolution kernel to obtain n / (k*k*k) features. The dimension of each feature is 23 / (k*k*k)*64, where 64 is the number of convolution kernels.

[0071] 3) For the obtained downsampled features, perform Transformer encoding, pooling and full connection calculations, and finally normalize them to obtain the preliminary voiceprint feature e(norm).

[0072] 4) Using the voiceprint feature e(t-1) obtained in the previous round (i.e., step t-1), the similarity with the current voiceprint feature is calculated. Specifically, the inner product h(e(t-1), e(norm)) of e(norm) and e(t-1) can be calculated (where h is a cosine function as shown in the figure), and the inner product is mapped to [0, 1] by a sigmoid function (shown as Greek letter σ) and a bias b. Here, the inner product h(e(t-1), e(norm)) can be directly regarded as a similarity function, or the sigmoid(h(e(t-1), e(norm))+b) can be regarded as a similarity function. Both functions can describe the similarity between the voiceprint feature e(t-1) obtained in the previous round and the preliminary feature e(norm) obtained in the current round.

[0073] The sigmoid function can be multiplied with the voiceprint feature e(t-1) obtained in the previous round (i.e., step t-1), to obtain:

[0074] f(t) = sigmoid(h(e(t-1), e(norm))+b) * e(t-1)

[0075] Thus, based on the similarity between the two voiceprint features, the weight of the voiceprint feature added to the next round of voiceprint feature is determined, so as to achieve the effect of enhancing voiceprint extraction.

[0076] 4) The t-1 step feature and the current feature are added and normalized to obtain an enhanced voiceprint feature, i.e., the t step feature e(t):

[0077] e(t) = norm(f(t) + e(norm))

[0078] The enhanced voiceprint feature e(t) (i.e., enhanced voiceprint information) obtained can be subsequently used by the model for corresponding speaker confirmation or speaker recognition. For example, in a binary classification system, the probability that the above enhanced voiceprint information is the same as or different from the reference voiceprint information can be output, and thus it is determined whether the speaker of the speech segment is the same person as the registered speaker. In a multi-classification system, the probability that the above enhanced voiceprint information is the same as or different from multiple reference voiceprint information can be output, and thus it is determined which registered speaker the speaker of the speech segment belongs to.

[0079] It should be understood that when the length or the information contained in the input speech segment is sufficient (e.g., when a certain threshold is met), the above similarity calculation and merging function of the model can not be enabled, and the voiceprint feature e(norm) can be directly used for voiceprint recognition. When it is determined that the length or the information contained in the input speech segment is insufficient, the e(t-1) of the previous round itself can be obtained (i.e., e(t-1) = e(t-2) at this time), and the above similarity calculation and merging function of the model can be enabled. t-1(norm)) can also be accumulated as needed, for example, the e(t-1) of the previously accumulated features in the previous round is obtained (i.e., at this time, the e(t-1) is the enhanced voiceprint feature based on the previous 1 round or more of the voiceprint feature), and the enhanced voiceprint feature calculation is performed for the current input voiceprint feature.

[0080] In the voiceprint model as shown in Figure 5 In the voiceprint model as shown in

[0081] Further, as an alternative, the enhancement part (i.e., similarity calculation and merging) can be performed before the preliminary voiceprint feature extraction, for example, the enhancement part directly starts from the underlying speech MFCC feature to fuse the enhancement, and after obtaining the fused enhanced speech feature, the voiceprint extraction is performed on the above-mentioned feature.

[0082] The above-mentioned voiceprint extraction scheme of the present application is particularly suitable for occasions involving front and back associated speech input. For this purpose, the present application can also be implemented as a voiceprint recognition method. Figure 6 A schematic flowchart of a voiceprint recognition method according to an embodiment of the present application is shown. The method can be implemented by a device requiring voiceprint recognition, such as a smart speaker, and the method can particularly utilize the voiceprint model as described above for recognition.

[0083] In step S610, the current speech input of the user is received. In step S620, the current speech feature of the current speech input is extracted. In step S630, a similarity function representing the similarity between the current speech feature and the previous speech feature is calculated. In step S640, enhanced voiceprint information is calculated according to the similarity function. In step S650, the identity of the user is recognized based on the enhanced voiceprint information, for example, the enhanced voiceprint information is compared with the registered reference voiceprint information, and the identity of the user is recognized based on the comparison result.

[0084] The previous speech feature may be extracted based on the user's previous speech input. The previous speech input and the current speech input may be previous and next rounds of dialogue in the same multi-round dialogue.

[0085] In some embodiments, the voice features required for reliable voiceprint recognition can be continuously collected based on the natural progression of the conversation. In other embodiments, a conversation with the user can be proactively initiated to collect more voice features. In this case, the method may also include: if it is determined that the previous voice features cannot identify the user's identity, proactively conducting the next round of voice conversation to prompt the user to perform the current voice input.

[0086] In this way, the characteristics of continuous conversations can be fully combined, and the voiceprint features of the previous round of speech can be used to enhance the voiceprint features of the current round of speech, thereby effectively improving the accuracy of voiceprint feature extraction in continuous conversations and reducing the error rate of voiceprint recognition.

[0087] In one embodiment, the present invention can also be implemented as a voiceprint information extraction device. Figure 7 The figure shows a schematic diagram of the components of a voiceprint information extraction device according to an embodiment of the present invention. As shown in the figure, the device 700 may include a feature acquisition unit 710, a feature acquisition unit 720 and an enhanced voiceprint calculation unit 730.

[0088] The feature acquisition unit 710 is configured to acquire a first speech feature from a first speech segment. The similarity calculation unit 720 is configured to calculate a similarity function representing the similarity between the first speech feature and a second speech feature. The enhanced voiceprint calculation unit 730 is configured to calculate enhanced voiceprint information incorporating the first and second speech features based on the similarity function. Here, the feature acquisition unit 710 may have previously acquired the second speech feature from the second speech segment.

[0089] In one embodiment, the present invention can also be implemented as a voiceprint model training device. Figure 8 A schematic diagram of the components of a voiceprint model training device according to an embodiment of the present invention is shown. As shown in the figure, the device 800 may include a data acquisition unit 810, a feature extraction unit 820, and a model training unit 830.

[0090] The data acquisition unit 810 is used to acquire a training speech segment pair and its corresponding label data, wherein the speech feature pair includes a reference speech segment and at least two speech segments to be matched.

[0091] The feature extraction unit 820 is configured to perform feature extraction processing on the speech segment data in the training speech segment pair to obtain a training sample feature set.

[0092] The model training unit 830 is configured to train a preset neural network structure based on the training sample feature set and the corresponding label data, to obtain the voiceprint model.

[0093] In the training process, the similarity of the corresponding voice features of the at least two voice segments to be matched is calculated, and the enhanced voiceprint information is synthesized according to the similarity; and the matching result of the enhanced voiceprint information and the voiceprint information corresponding to the reference voice segment is taken as the output of the voiceprint model.

[0094] In one embodiment, the present application can also be implemented as a voice interaction device. The device can perform conventional voice interaction and can ensure interaction security through voiceprint extraction and comparison of acquired voice information.

[0095] Figure 9 A composition diagram of a voice interaction device according to one embodiment of the present application is shown. As shown in the figure, the voice interaction device 900 includes a sound collection module 910, a feature extraction module 920, a voice recognition module 930, and a voiceprint recognition module 940.

[0096] The sound collection module 910 is configured to collect sound information. The sound collection module 910 can be a microphone or a microphone array in the device 900.

[0097] The feature extraction module 920 is configured to extract voice features contained in the collected sound information. The voice recognition module 930 is configured to recognize semantics in the voice features. The voiceprint recognition module 940 is configured to extract a voiceprint from the voice features for voiceprint recognition. Here, the feature extraction module 920, the voice recognition module 930, and the voiceprint recognition module 940 can be separate chips arranged in the device 900, or different functional groups on the same chip, or can complete at least part of the functions via a server.

[0098] The sound collection module 910 collects sound information from at least two rounds of dialogues, to synthesize enhanced voiceprint information based on the similarity of the sound information, and the voiceprint recognition module 940 performs voiceprint recognition based on the enhanced voiceprint information.

[0099] Further, the device 900 can further include a feedback module configured to provide feedback based on the recognized semantics, wherein the at least two rounds of dialogues refer to user voice input during an interval of the feedback.

[0100] The feedback module includes a sound output module, a display module, and / or a remote control module. For example, the device 900 can be a smart speaker with or without a screen. The speaker of the smart speaker can be used for voice interaction with the user; the optional display screen can be used for visual feedback on the user's voice input (e.g., displaying the lyrics while playing the song requested by the user); and the remote control module (e.g., a Bluetooth module) can be used to control other devices bound to the device 900 based on the user's input.

[0101] Further, the device 900 can further include a networking module for uploading the collected sound information and / or extracted voice features to a server, and obtaining the voiceprint recognition result from the server.

[0102] In addition to the voice interaction device preferably implemented as a smart speaker as described above, the present application can also be implemented by other Internet of Things devices, such as other Internet of Things devices that do not perform voice control terminal tasks.

[0103] In one embodiment, the present application can be implemented as an Internet of Things device, including a sound collection module for collecting sound information, a voice recognition module for recognizing semantics in voice features, and an execution module for executing corresponding operations based on the recognized semantics. The sound information collected by the sound collection module and other sound information collected by other Internet of Things devices are used to synthesize enhanced voiceprint information based on the similarity of the sound information, for voiceprint recognition by a voiceprint recognition device based on the enhanced voiceprint information. For example, a user can directly perform voice operations on an Internet of Things device, such as inputting a "lower temperature" voice command to a smart refrigerator or a "shutdown" command to a smart television. The smart refrigerator can recognize limited voice commands, including the "lower temperature" command as described above, and the execution module can execute the operation of lowering the temperature. Similarly, the smart television can recognize limited voice commands, including the "shutdown" command as described above, and the execution module can execute the shutdown operation.

[0104] Since the length of the instructions received by a single device is limited, each Internet of Things device can upload the received instructions to a voiceprint recognition device for synthesis of enhanced voiceprint information and voiceprint recognition. The voiceprint recognition device can be a smart terminal of a family, such as a smart speaker, or can be implemented by a cloud. In one special implementation, the "other Internet of Things device" for providing "other sound information" can be the voiceprint recognition device itself, such as a smart speaker. In other words, the voiceprint recognition device can aggregate the sound information received by other devices individually for voiceprint recognition in the case where the voiceprint information obtained by the voiceprint recognition device itself is insufficient, or by default. This improves the overall security of, for example, a home Internet of Things.

[0105] In one similar embodiment, the Internet of Things device implemented by the present application can include: a sound collection module for collecting sound information from at least two dialogues; a voice recognition module for recognizing semantics in voice features; an execution module for executing corresponding operations based on the recognized semantics; and a networking module for uploading the sound information collected by the sound collection module to a voiceprint recognition device, which performs voiceprint recognition based on similarity of the sound information to synthesize enhanced voiceprint information. Here, multiple sound information collected by one Internet of Things device (for example, "turn on the TV" and "XX station" collected by a smart TV in succession) can be collected by a voiceprint recognition device such as a smart speaker to generate enhanced voiceprint and perform recognition. Thus, it is convenient to perform necessary voiceprint recognition when a user performs voice operation using a device without voiceprint recognition function.

[0106] When the sound information provided by the above Internet of Things device is recognized as non-matching voiceprint information, the operation based on the above sound information can be stopped under the control of the voiceprint recognition device, and an alarm can be sent to the binding account.

[0107] To this end, the present application can also be implemented as a voice interaction system, which includes: multiple voice interaction devices as described above; and / or an Internet of Things device as described above. The system can also include a server on which a voiceprint model trained based on a neural network is arranged.

[0108] The voice interaction device can upload voice information used by the user to register voiceprint. The system also includes a database for storing reference voiceprint information corresponding to multiple voice interaction devices. The voiceprint model on the server can extract voiceprint information and determine whether it matches the corresponding reference voiceprint information based on the current voiceprint information and / or enhanced voiceprint information collected by the voice interaction device.

[0109] Figure 10 A structural schematic diagram of a computing device that can be used to implement the above voiceprint-related processing method according to one embodiment of the present application is shown.

[0110] Referring to Figure 10 The computing device 1000 includes a memory 1010 and a processor 1020.

[0111] The processor 1020 can be a single core processor or a multiple core processor. In some embodiments, the processor 920 can include a general purpose processor and one or more special purpose co-processors such as graphics processors (GPUs), digital signal processors (DSPs), etc. In some embodiments, the processor 920 can be implemented using a custom circuit, such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0112] The memory 1010 can include various types of storage units, such as a system memory, a read-only memory (ROM), and a permanent storage device. The ROM can store static data or instructions that the processor 1020 or other modules of the computer need. The permanent storage device can be a read and write memory device. The permanent storage device can be a non-volatile storage device that does not lose stored instructions and data even if the computer is powered off. In some embodiments, the permanent storage device uses a mass storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device can be a removable storage device (such as a floppy disk, optical drive). The system memory can be a read and write memory device or a volatile read and write memory device, such as a dynamic random access memory. The system memory can store some or all of the instructions and data required by the processor during runtime. In addition, the memory 1010 can include a combination of any computer readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), magnetic disks and / or optical disks. In some embodiments, the memory 1010 can include a read and / or write removable storage device, such as a compact disc (CD), a read-only digital versatile disc (e.g., DVD-ROM, dual-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (e.g., SD card, min SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. The computer readable storage media does not include carrier waves and transient electronic signals transmitted through a wire or wireless transmission.

[0113] The memory 1010 stores executable code, which, when processed by the processor 920, can cause the processor 1020 to perform the voiceprint related processing method described above.

[0114] The method and device for voiceprint information extraction, model training and voiceprint recognition according to the present application have been described in detail above with reference to the accompanying drawings. The present application can utilize multiple pieces of speech information with correlation to obtain enhanced voiceprint information based on the similarity, thereby improving the accuracy of voiceprint recognition. The present application preferably uses a voiceprint model trained by a neural network to obtain the enhanced voiceprint information, thereby further improving the accuracy and efficiency of voiceprint extraction. The above method can be particularly used by a speech interaction device, such as a smart speaker, in a multi-round dialogue in which the single-round dialogue time is too short, thereby effectively enhancing the voiceprint features of the speech in the next dialogue by using the voiceprint features of the speech in the previous round or rounds of the continuous dialogue.

[0115] Furthermore, the method according to the present application can also be implemented as a computer program or computer program product comprising computer program code instructions for executing the above steps defined in the above method of the present application.

[0116] Alternatively, the present application can also be implemented as a non-transitory machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) having stored thereon executable code (or computer program, or computer instruction code) which, when executed by a processor of an electronic device (or computing device, server, etc.), causes the processor to perform the steps of the above method according to the present application.

[0117] Those skilled in the art will further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein can be implemented as electronic hardware, computer software, or combinations of both.

[0118] The flow diagrams and block diagrams in the drawings are presented to illustrate the architecture, functionality, and operations of possible implementations of systems and methods according to the present application. In this regard, each block in the flow diagrams and block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flow diagrams, and combinations thereof, can be implemented by special purpose hardware-based systems that perform the specified functions or operations, or combinations of special purpose hardware and computer instructions.

[0119] Having described various embodiments of the application, it is to be understood that the above description is meant to be illustrative only and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art, without departing from the scope and spirit of the described embodiments. The choice of words in this document is intended to best explain the principles of the embodiments, the practical application, or improvement over the technology in the art, or to enable other ordinary skilled in the art to understand the embodiments disclosed herein.

Claims

1. A voiceprint model training method, comprising: obtaining a training speech segment pair and corresponding label data, wherein the speech segment pair comprises a reference speech segment and at least two to-be-matched speech segments, the at least two to-be-matched speech segments are a second speech segment and a first speech segment inputted in sequence, and the second speech segment is a speech segment associated in time with the first speech segment and having a considerable probability of coming from the same speaker; performing feature extraction processing on speech segment data in the training speech segment pair to obtain a training sample feature set; and training based on the training sample feature set and corresponding label data using a preset neural network structure to obtain the voiceprint model, wherein, in the training process, a similarity of corresponding speech features of the at least two to-be-matched speech segments is calculated, and enhanced voiceprint information is obtained according to the similarity; a matching result of the enhanced voiceprint information and voiceprint information corresponding to the reference speech segment is taken as an output of the voiceprint model.

2. The method of claim 1, wherein, The voiceprint model comprises a similarity calculation module, and the training based on the training sample feature set and corresponding label data using a preset neural network structure comprises: adjusting network parameters of the neural network using a continuous similarity loss function, wherein the continuous similarity loss function is used to describe a loss between a similarity function and similarity label data.

3. The method of claim 2, wherein, The voiceprint model comprises a merging module, and the training based on the training sample feature set and corresponding label data using a preset neural network structure comprises: adjusting network parameters of the neural network using a similarity loss function, wherein the similarity loss function is used to describe a loss between the enhanced voiceprint information and voiceprint label data.

4. The method of claim 3, wherein, The training based on the training sample feature set and corresponding label data using a preset neural network structure comprises: adjusting network parameters of the neural network by minimizing a sum of the continuous similarity loss function and the similarity loss function. 5.The method of claim 1, further comprising: performing down-sampling on speech features contained in the training sample feature set using a convolutional neural network.

6. The method of claim 1, wherein, calculating a similarity of corresponding speech features of the at least two to-be-matched speech segments and obtaining enhanced voiceprint information according to the similarity comprises: obtaining voiceprint features of one to-be-matched speech segment as to-be-enhanced voiceprint features; calculating a similarity of the to-be-enhanced voiceprint features and preceding voiceprint features, wherein the preceding voiceprint features are obtained based on at least one to-be-matched speech segment of a preceding input model. 7.A voiceprint information extraction method, using a voiceprint model obtained by the training method of any one of claims 1-6 to extract voiceprint information, and the voiceprint information extraction method comprising: obtaining first speech features of a first speech segment; obtaining second speech features of a second speech segment, wherein the second speech segment is a speech segment associated in time with the first speech segment and having a considerable probability of coming from the same speaker; computing a similarity function representing similarity between the first speech feature and the second speech feature; and computing enhanced speaker information incorporating the first speech feature and the second speech feature according to the similarity function for recognizing speaker identity.

8. The method of claim 7, further comprising: extracting a first speaker feature based on the first speech feature; extracting a second speaker feature based on the second speech feature, and computing a similarity function representing similarity between the first speech feature and the second speech feature comprises: computing a similarity function of the first speaker feature and the second speaker feature, computing enhanced speaker information incorporating the first speech feature and the second speech feature according to the similarity function comprises: merging the first speaker feature and the second speaker feature according to the similarity to obtain the enhanced speaker information. computing enhanced speaker information incorporating the first speech feature and the second speech feature according to the similarity function comprises:

9. The method of claim 7, wherein, merging the first speech feature and the second speech feature according to the similarity to obtain enhanced speech feature; and extracting the enhanced speaker information based on the enhanced speech feature.

10. The method of claim 8 or 9, further comprising: wherein, the first speech segment comprises a current speech segment, and the second speech segment comprises a previous speech segment. the computing of the similarity function and the extracting of the speaker feature are based on a speaker model trained by a neural network.

11. The method of claim 8 or 9, wherein, 12. The method of claim 8 or 9, further comprising: recognizing speaker identity based on the enhanced speaker information. recognizing speaker identity based on the enhanced speaker information comprises:

13. The method of claim 11, wherein, the speaker model outputs a probability of whether the enhanced speaker information matches a reference speaker information based on the enhanced speaker information. computing a similarity function representing similarity between the first speech feature and the second speech feature comprises:

14. The method of claim 7, wherein, computing the similarity function representing similarity between the first speech feature and the second speech feature by using inner product. computing enhanced speaker information incorporating the first speech feature and the second speech feature according to the similarity function comprises:

15. The method of claim 7, wherein, using the computed similarity function as a weight of the second speech feature to compute the enhanced speaker information.

16. The method of claim 7, further comprising: down-sampling the first speech feature to obtain a down-sampled first speech feature; down-sampling the second speech feature to obtain a down-sampled second speech feature, and computing a similarity function representing similarity between the first speech feature and the second speech feature comprises: computing a similarity function representing similarity between the down-sampled first speech feature and the down-sampled second speech feature.

17. A speaker recognition method, using a speaker model trained by the training method of any one of claims 1-6 to perform recognition, and the speaker recognition method comprising: receiving a current speech input of a user; extracting a current speech feature of the current speech input; ​ extracting previous speech features based on previous speech input of the user, wherein the previous speech input is a speech segment that is temporally associated with the current speech input and has a considerable probability of coming from the same speaker; calculating a similarity function representing similarity between the current speech features and the previous speech features; calculating enhanced voiceprint information according to the similarity function; and identifying the identity of the user based on the enhanced voiceprint information.

18. The method of claim 17, wherein, The previous speech input and the current speech input belong to a previous round and a next round of a same multi-round dialogue.

19. The method of claim 18, comprising: in a case where it is determined that the previous speech features cannot identify the identity of the user, actively conducting a next round of speech dialogue to prompt the user to make the current speech input.

20. The method of claim 17, wherein, identifying the identity of the user based on the enhanced voiceprint information comprises: comparing the enhanced voiceprint information with registered reference voiceprint information; and identifying the identity of the user based on a comparison result.

21. A voiceprint information extraction apparatus, which extracts voiceprint information using a voiceprint model obtained by the training method of any one of claims 1-6, and comprises: a feature acquisition unit configured to acquire first speech features of a first speech segment and second speech features of a second speech segment, wherein the second speech segment is a speech segment that is temporally associated with the first speech segment and has a considerable probability of coming from the same speaker; a similarity calculation unit configured to calculate a similarity function representing similarity between the first speech features and the second speech features; and an enhanced voiceprint calculation unit configured to calculate enhanced voiceprint information incorporating the first speech features and the second speech features according to the similarity function for identifying a speaker identity.

22. A voiceprint model training apparatus, comprising: a data acquisition unit configured to acquire a pair of training speech segments and corresponding label data, wherein the pair of training speech segments comprises a reference speech segment and at least two to-be-matched speech segments, the at least two to-be-matched speech segments being a second speech segment and a first speech segment input in succession to a model, the second speech segment being a speech segment that is temporally associated with the first speech segment and has a considerable probability of coming from the same speaker; a feature extraction unit configured to perform feature extraction processing on speech segment data in the pair of training speech segments to obtain a training sample feature set; and a model training unit configured to train, based on the training sample feature set and the corresponding label data, using a preset neural network structure to obtain the voiceprint model, wherein, in the training process, similarity of corresponding speech features of the at least two to-be-matched speech segments is calculated, and enhanced voiceprint information is synthesized according to the similarity; a matching result of the enhanced voiceprint information and voiceprint information corresponding to the reference speech segment is taken as an output of the voiceprint model.

23. A voice interaction device, comprising: a sound acquisition module configured to acquire sound information; a feature extraction module configured to extract speech features contained in the acquired sound information; a speech recognition module configured to recognize semantics in the speech features; a voiceprint recognition module configured to extract a voiceprint from the speech feature for voiceprint recognition using a voiceprint model obtained by the training method of any one of claims 1-6, wherein the sound collection module is configured to collect sound information from at least two rounds of dialogues for synthesizing enhanced voiceprint information based on similarity of the sound information, and the voiceprint recognition module is configured to perform voiceprint recognition based on the enhanced voiceprint information. wherein the sound information collected from the at least two rounds of dialogues comprises a current speech input and a previous speech input, wherein the previous speech input is a speech segment associated in time with the current speech input and has a considerable probability of being from a same speaker.

24. The device of claim 23, comprising: a feedback module configured to provide feedback based on the recognized semantics, wherein the at least two rounds of dialogues refer to user speech inputs during which intervals the feedback is provided.

25. The apparatus of claim 24, wherein, the feedback module comprises: a sound output module; a display module; and / or a remote control module.

26. The device of claim 23, comprising: a networking module configured to upload the collected sound information and / or the extracted speech feature to a server and obtain the voiceprint recognition result from the server.

27. A computing device, comprising: a processor; and a memory having stored thereon executable code that, when executed by the processor, causes the processor to perform the method of any one of claims 1-20.

28. A non-transitory machine-readable storage medium having stored thereon executable code that, when executed by a processor of an electronic device, causes the processor to perform the method of any one of claims 1-20.

Images