Network model training method, device and storage medium for speaker recognition

Abstract

The present invention relates to the field of speech recognition technology, and in particular to a network model training method, device and storage medium for speaker recognition. The present invention first inputs a cross-domain speaker sample data set and a phoneme sample data set into a multi-task network model respectively, and uses a maximum mean difference algorithm to calculate the difference loss value between the two results according to the output results of the speaker classification subnet and the phoneme classification subnet, and uses the difference loss value as part of the total loss to continuously train the multi-task network model by increasing the total loss of the difference loss value, and finally obtains a trained model, and the trained speaker subnet model has a high generalization ability for phoneme sample data sets in different domains, that is, the trained model can weaken the influence of the difference between the cross-domain phoneme sample data set and the speaker sample data set on the accuracy of the network model in recognizing speakers.

Description

Technical Field

[0001] The present invention relates to the field of speech recognition technology, and in particular to a network model training method, device and storage medium for speaker recognition. Background Art

[0002] Speaker recognition is the task of verifying whether the input utterance (speech signal) belongs to a specific speaker. Speech signals are composed of multiple intrinsic components such as speaker emotion, accent (phonemes) and language. The uncertainty of these intrinsic factors, especially the uncertainty of speech content, will affect the recognition performance of the system. Therefore, the extraction of speaker embedding cannot only consider the speaker label. Inspired by the application of speaker adaptation technology in automatic speech recognition (ASR), multi-task learning (MTL) strategy was proposed to learn speech information contained in multiple related tasks to help improve the generalization ability of the main task (speaker recognition). In recent years, strategies such as adversarial learning have been introduced into the multi-task framework, which can play the advantages of both in terms of speech information usage. Many studies have shown that encouraging phoneme information at the frame level and suppressing phoneme information at the segment level are both effective. However, in many practical situations, it is very expensive and inflexible to obtain an ideal dataset in the domain with both speaker labels and phoneme labels. However, when a cross-domain ASR dataset or a cross-language ASR dataset is introduced into the phoneme recognition subnet (the phoneme recognition subnet is used to assist the speaker network model in speaker classification training), these methods usually do not provide more useful information for the speaker recognition subnet (speaker network model). Especially when performing speaker recognition of a minority language, generating new manually transcribed phoneme labels will result in longer training time. Therefore, since the phoneme-aware network model used to identify speakers is not suitable for the differences brought about by phoneme datasets and speaker datasets from different sources, the generalization ability of the phoneme-aware network model used for speaker classification training is poor.

[0003] In summary, the generalization ability of existing phoneme-aware network models used for speaker classification training is poor.

[0004] Therefore, the prior art still needs to be improved and enhanced. Summary of the invention

[0005] To solve the above technical problems, the present invention provides a network model training method, device and storage medium for speaker recognition, which solves the problem of poor generalization ability of the existing phoneme-aware network model used for speaker classification training.

[0006] To achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a network model training method for speaker recognition, which comprises:

[0008] Inputting a speaker sample data set into a multi-task network model, extracting a first result output by a set layer in a speaker classification subnet model in the multi-task network model, wherein the speaker classification subnet model is used for speaker classification training;

[0009] Inputting a cross-domain phoneme sample data set corresponding to the speaker sample data set into the multi-task network model, extracting a second result output by a set layer in a phoneme classification subnet model in the multi-task network, wherein the set layer in the phoneme classification subnet model corresponds to the set layer in the speaker classification subnet model, and the phoneme classification subnet model is used for phoneme classification training;

[0010] Applying a maximum mean difference algorithm to the first result and the second result to obtain a difference loss value corresponding to the first result and the second result;

[0011] According to the total loss to which the difference loss value is added, the multi-task network model where the speaker classification subnetwork model and the phoneme classification subnetwork model are located is trained to obtain the trained network model.

[0012] In one implementation, the multi-task network model where the speaker classification subnetwork model and the phoneme classification subnetwork model are located is trained according to the total loss to which the difference loss value is added to obtain the trained network model, including:

[0013] Obtaining a speaker sample identity label corresponding to the speaker sample data set;

[0014] Obtaining a trained speaker embedding model based on the speaker subnet model;

[0015] Obtaining a predicted speaker identity label based on the speaker sample dataset and the speaker embedding model;

[0016] Calculating an identity tag difference between the speaker sample identity tag and the speaker identity tag;

[0017] Obtaining the phoneme sample label corresponding to the phoneme sample data set;

[0018] Obtaining the predicted phoneme label according to the phoneme sample data set and the phoneme classification subnet model;

[0019] Calculating a phoneme difference between the phoneme sample label and the predicted phoneme label;

[0020] The identity tag difference, the difference loss value, and the phoneme difference are used as the new total loss to train the multi-task network model where the speaker classification subnet model and the phoneme classification subnet model are located, so as to obtain the trained network model.

[0021] In one implementation, the multi-task network model in which the speaker classification subnet model and the phoneme classification subnet model are located is trained according to the identity tag difference, the difference loss value, and the phoneme difference to obtain the trained network model, including:

[0022] Performing weighted calculation on the identity tag difference, the difference loss value, and the phoneme difference to obtain a total loss value corresponding to the network model;

[0023] The network model is trained according to the total loss value to obtain the trained network model.

[0024] In one implementation, the multi-task network model in which the speaker classification subnet model and the phoneme classification subnet model are located is trained according to the identity tag difference, the difference loss value, and the phoneme difference to obtain the trained network model, including:

[0025] According to the phoneme classification subnetwork model, a frame-level phoneme classification subnetwork and a segment-level phoneme classification subnetwork in the phoneme classification subnetwork model are obtained;

[0026] Obtaining a predicted first phoneme label in the phoneme label according to the phoneme sample data set and the frame-level phoneme classification subnetwork;

[0027] According to the phoneme sample data set and the segment-level phoneme classification subnetwork, obtain a predicted phoneme second label in the phoneme label;

[0028] calculating a phoneme first difference between the phoneme sample label and the phoneme first label in the phoneme difference;

[0029] calculating a phoneme second difference between the phoneme sample label and the phoneme second label in the phoneme difference;

[0030] The multi-task network model where the speaker classification subnet model and the phoneme classification subnet model are located is trained according to the identity tag difference, the difference loss value, the phoneme first difference, and the phoneme second difference to obtain the trained network model.

[0031] In one implementation, the method further includes:

[0032] Obtaining a speaker sample identity label corresponding to the speaker sample data set;

[0033] Obtaining the phoneme sample label corresponding to the phoneme sample data set;

[0034] Obtaining a predicted speaker identity label based on the speaker sample data set and the trained speaker classification subnet model;

[0035] Obtaining the phoneme label according to the phoneme sample data set and the trained phoneme classification subnet model;

[0036] Recording the speaker sample whose identity label is different from the speaker sample corresponding to the speaker identity label as a speaker misclassification sample;

[0037] Applying the misclassification vector guidance loss function to the speaker sample data set to obtain a loss function value for a new speaker;

[0038] According to the loss function value of the new speaker, the total loss of the multi-task network after training is updated.

[0039] In one implementation, the phoneme classification subnet model is a phoneme neural network model for identifying phoneme labels, the speaker classification subnet model is a speaker neural network model for identifying speaker identities, the set layer in the phoneme classification subnet model is a layer above the layer where the phoneme neural network model is used to output phoneme labels, and the set layer in the speaker classification subnet model is a layer above the layer where the speaker neural network model is used to output speaker identity labels.

[0040] In a second aspect, an embodiment of the present invention further provides a device for a network model training method for speaker recognition, wherein the device comprises the following components:

[0041] A first result calculation module, used to input a speaker sample data set into a multi-task network model, extract a first result output by a set layer in a speaker classification subnet model in the multi-task network model, and the speaker classification subnet model is used for speaker classification training;

[0042] A second result calculation module, configured to input a cross-domain phoneme sample data set corresponding to the speaker sample data set into the multi-task network model, and extract a second result output by a set layer in a phoneme classification subnet model in the multi-task network, wherein the set layer in the phoneme classification subnet model corresponds to the set layer in the speaker classification subnet model, and the phoneme classification subnet model is used for phoneme classification training;

[0043] a difference loss value calculation module, used for applying a maximum mean difference algorithm to the first result and the second result to obtain a difference loss value corresponding to the first result and the second result;

[0044] The model training module is used to train the multi-task network model where the speaker classification subnet model and the phoneme classification subnet model are located according to the total loss to which the difference loss value is added, so as to obtain the trained network model.

[0045] In a third aspect, an embodiment of the present invention further provides a terminal device, wherein the terminal device comprises a memory, a processor, and a network model training program for speaker recognition stored in the memory and executable on the processor, and when the processor executes the network model training program for speaker recognition, the steps of the above-mentioned network model training method for speaker recognition are implemented.

[0046] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a network model training program for speaker recognition is stored. When the network model training program for speaker recognition is executed by a processor, the steps of the above-mentioned network model training method for speaker recognition are implemented.

[0047] Beneficial effect: The present invention first inputs the speaker sample data set and the phoneme sample data set into the multi-task phoneme network model respectively, and uses the maximum mean difference algorithm to calculate the difference loss value between the two results according to the results output by the speaker classification subnet model and the phoneme classification subnet model, and continuously trains the phoneme perception model through the total loss of the added difference loss value, and finally obtains the trained model. The trained phoneme perception model has a high generalization ability for phoneme sample data sets in different domains, that is, the trained model can weaken the influence of the differences between the phoneme sample data sets in different domains and the speaker sample data sets on the accuracy of speaker recognition by the network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is the overall flow chart of the present invention;

[0049] Figure 2 The network model of the present invention includes three submodules;

[0050] Figure 3 The network model of the present invention includes four submodules;

[0051] Figure 4 This is a block diagram of the internal structure principle of the terminal device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The following is a clear and complete description of the technical solution of the present invention in combination with the embodiments and the accompanying drawings. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] Research has found that speaker recognition is the task of verifying whether the input speech (speech signal) belongs to a specific speaker. Speech signals are composed of multiple intrinsic components such as speaker emotion, accent (phonemes) and language. The uncertainty of these intrinsic factors, especially the uncertainty of speech content, will affect the recognition performance of the system. Therefore, the extraction of speaker embedding cannot only consider the speaker label. Inspired by the application of speaker adaptation technology in automatic speech recognition (ASR), multi-task learning (MTL) strategies have been proposed to learn speech information contained in multiple related tasks to help improve the generalization ability of the main task (speaker recognition). In recent years, strategies such as adversarial learning have been introduced into the multi-task framework, which can play the advantages of both in terms of the use of speech information. Many studies have shown that encouraging phoneme information at the frame level and suppressing phoneme information at the segment level are both effective. However, in many practical situations, it is very expensive and inflexible to obtain an ideal dataset with both speaker labels and phoneme labels. However, when a cross-domain ASR dataset or a cross-language ASR dataset is introduced into the phoneme recognition subnet (the phoneme recognition subnet is used to assist the speaker classification subnet model in speaker classification training), these methods usually do not provide more useful information for the speaker recognition subnet (speaker subnet model). Especially when performing speaker recognition of a minority language, generating new manually transcribed phoneme labels will result in longer training time. Therefore, since the phoneme-aware network model used for speaker classification training is not suitable for the differences brought about by phoneme datasets and speaker datasets from different sources, the generalization ability of the phoneme-aware network model used for speaker classification training is poor.

[0054] In order to solve the above technical problems, the present invention provides a network model training method, device and storage medium for speaker recognition, which solves the problem of poor generalization ability of the existing phoneme-aware network model used for speaker classification training. In specific implementation, the present invention first inputs the speaker sample data set and the phoneme sample data set into the multi-task phoneme network model respectively, and uses the maximum mean difference algorithm to calculate the difference loss value between the two results according to the results output by the speaker classification subnet model and the phoneme classification subnet model, and continuously trains the phoneme-aware model by adding the total loss of the difference loss value, and finally obtains the trained model. The present invention can improve the generalization ability of the trained model.

[0055] For example, the speaker sample data set comes from domain A, and the phoneme sample data set comes from domain B. The speaker sample data set and the phoneme sample data set come from different domains (for example, the two data sets come from different databases, and the data distribution of different databases is different, or the data encoding method is also different), so there are differences between the speaker sample data set and the phoneme sample data set, and the existence of such differences will cause the network model where the speaker classification subnet model and the phoneme classification subnet model are located to be unable to use phoneme information to improve the speaker classification training performance, and the existing network model cannot adapt to such differences. In order to enable the network model to adapt to such differences, this embodiment uses the maximum mean difference algorithm to calculate the difference loss value corresponding to the results output by the two subnet models, and trains the network model based on the total loss with the difference loss value added, so that the trained network model can adapt to speaker sample data sets and phoneme sample data sets from different domains.

[0056] Exemplary Methods

[0057] A network model training method for speaker recognition in this embodiment can be applied to a terminal device, which can be a terminal product with computing capabilities, such as a computer. Figure 1 As shown in , the network model training method for speaker recognition specifically includes the following steps:

[0058] S100, inputting a speaker sample data set into a multi-task network model, extracting a first result output by a set layer in a speaker classification subnet model in the multi-task network model, and the speaker classification subnet model is used for speaker classification training.

[0059] In this embodiment, the speaker classification subnet model is a speaker embedding model, which is a neural network model M s . Neural network model M s like Figure 2 As shown, in this embodiment, M s The setting layer is located above the layer for outputting speaker labels. In this embodiment, the setting layer is M s The seventh floor.

[0060] S200, inputting a cross-domain phoneme sample data set corresponding to the speaker sample data set into the multi-task network model, extracting a second result output by a set layer in a phoneme classification subnet model in the multi-task network, wherein the set layer in the phoneme classification subnet model corresponds to the set layer in the speaker classification subnet model, and the phoneme classification subnet model is used for phoneme classification training.

[0061] In this embodiment, the phoneme classification subnet model is a neural network model Mp . Neural network model M p like Figure 2 As shown, in this implementation, M p The setting layer is located above the layer where the phoneme label is output. The setting layer of this embodiment is M p The seventh floor.

[0062] S300, applying a maximum mean difference algorithm to the first result and the second result to obtain a difference loss value corresponding to the first result and the second result.

[0063] In this embodiment, the maximum mean difference algorithm (MMD) is used to calculate the difference loss value L mmd The principle is as follows:

[0064] Assumptions and is a sample set of distribution S and P, and the difference loss value of S and P is calculated using the following formula:

[0065]

[0066] Where H represents the feature space, i.e., the regenerated Hilbert space, and φ(·) represents the mapping function. Since the Gaussian kernel function can map infinite-dimensional space, the Gaussian kernel function is used. To represent the inner product of the mapping function, σ is the bandwidth parameter used to control the radial range of action. The original formula after the introduction of the Gaussian kernel function is decomposed into:

[0067]

[0068] When formula (2) is used to calculate the difference loss value L corresponding to the first result output by the seventh layer of the speaker classification subnet model of this embodiment and the second result output by the seventh layer of the phoneme classification subnet model, mmd When , we only need to replace Replace with the first result, Just replace it with the second result.

[0069] S400, training the multi-task network model where the speaker classification subnetwork model and the phoneme classification subnetwork model are located according to the total loss to which the difference loss value is added, to obtain the trained network model.

[0070] According to the difference loss value L mmd , identity label difference L s , phoneme difference L p These three jointly train the network model.

[0071] In this embodiment, the total loss value L corresponding to the weighted sum of the three total (Total loss value Ltotal That is, the total loss) is back-propagated to train the network model.

[0072] L total =L s +α·L p +β·L mmd (3)

[0073] Where α represents the phoneme subnetwork weight, and β represents the maximum mean difference loss weight.

[0074] Calculate the identity label difference L s When, the process includes the following steps S401, S402, S403, and S404:

[0075] S401, obtaining a speaker sample identity label corresponding to the speaker sample data set.

[0076] S402: Obtain a trained speaker embedding model based on the speaker subnet model.

[0077] S403: Obtain a predicted speaker identity label according to the speaker sample dataset and the speaker embedding model.

[0078] S404: Calculate the identity label difference between the speaker sample identity label and the predicted speaker identity label.

[0079] Phonemic Difference L p When, the process includes the following steps S405, S406, and S407:

[0080] S405: Obtain the phoneme sample label corresponding to the phoneme sample data set.

[0081] S406: Obtain the predicted phoneme label according to the phoneme sample data set and the phoneme classification subnet model.

[0082] S407: Calculate the phoneme difference between the phoneme sample label and the predicted phoneme label.

[0083] Next, we introduce L s and L p The detailed calculation process is:

[0084] The network model of this embodiment includes not only the speaker classification subnet model and the phoneme classification subnet model, but also a shared frame-level network module M f , where the shared frame-level network module M f Used to mine the common features of speaker information and phoneme information.

[0085] like Figure 2 As shown, given a training segment pair [Xs ,X p ] and the corresponding speaker identity label y s and phoneme sample labels y p , s is used to indicate the speaker, and p is used to indicate the phoneme. It consists of M frames. For X s In the i-th frame, It consists of N frames.

[0086] L s =CE(M s (M f (X s )),y s ) (4)

[0087]

[0088] Where CE represents the cross entropy loss function (Coss Entropy loss).

[0089] In order to improve the generalization ability of the entire network model, this embodiment divides the phoneme classification subnetwork model into the following Figure 3 The frame-level phoneme classification subnetwork M shown ps and segment-level phoneme classification subnetwork M pf , then formula (3) is transformed into formula (6):

[0090] L total =L s +α·L pf +β·L ps +γ·L mmd (6)

[0091] In the formula, α represents the weight of the frame-level phoneme subnetwork, β represents the weight of the segment-level phoneme subnetwork, γ represents the maximum mean difference loss weight between the speaker subnetwork and the frame-level phoneme subnetwork, and L pf is the second phoneme difference, L ps For the first phoneme difference, calculate L pf and L ps The specific process includes: obtaining the frame-level phoneme classification subnetwork M in the phoneme classification subnetwork model according to the two phoneme classification subnetwork models; ps and segment-level phoneme classification subnetwork M pf ; Based on the phoneme sample data set and the frame-level phoneme classification subnetwork, obtain the predicted phoneme first label in the phoneme label; Based on the phoneme sample data set and the segment-level phoneme classification subnetwork, obtain the predicted phoneme second label in the phoneme label; Calculate the phoneme first difference L between the phoneme sample label and the phoneme first label in the phoneme difference psCalculate the phoneme second difference L between the phoneme sample label and the phoneme second label in the phoneme difference pf .

[0092] The trained network model is obtained through steps S100, S200, S300, and S400. The present embodiment further updates the trained network model. Updating the trained network model includes the following steps S501, S502, S503, S504, S505, S506, and S507:

[0093] S501, obtaining a speaker sample identity label corresponding to the speaker sample data set.

[0094] The speaker sample identity tag is an identity tag pre-set for the speaker. The identity tag in this embodiment can be analogous to the speaker's ID number, so the identity tag in this embodiment is a unique tag.

[0095] S502: Obtain a phoneme sample label corresponding to the phoneme sample data set.

[0096] The phoneme sample label is a label pre-set for the phoneme. The phoneme label of this embodiment can be analogous to the phonetic symbol of each word.

[0097] S503: Obtain a predicted speaker identity label according to the speaker sample data set and the trained speaker classification subnet model.

[0098] The speaker identity label is the label obtained by the trained speaker classification subnet model based on the speaker sample dataset.

[0099] S504: Obtain the phoneme label according to the phoneme sample data set and the trained phoneme classification subnet model.

[0100] S505: Record the speaker sample whose identity label is different from the speaker sample corresponding to the speaker identity label as a speaker misclassification sample.

[0101] If the speaker sample identity label corresponding to a speaker sample data set is A, but after the speaker classification subnet model in the training process identifies the speaker sample data set, the predicted speaker identity label is B, and A and B are different, the speaker sample data set is recorded as a speaker misclassification sample.

[0102] S506, applying the misclassification vector guidance loss function to the speaker misclassification sample to obtain a loss function value L mv .

[0103] S507, updating the total loss and training the network model according to the loss function value of the new speaker.

[0104] In this embodiment, the loss function value L is calculated mv The principle is as follows:

[0105] There are two problems with the margin-based softmax, which is widely used in the speaker field: 1) these loss functions do not consider the importance of difficult samples caused by misclassification during training; 2) the interval between classes cannot be adaptively changed during training. This embodiment introduces the loss function guided by the misclassification vector (mv-softmax), and the formula is as follows:

[0106]

[0107] Where I k The formula is as follows:

[0108]

[0109] When I k =1, it means that the sample is currently a difficult sample, and we will emphasize this misclassification vector:

[0110]

[0111] Where t≥0 is a hyperparameter, f(·) is the distance formula of different margin-based softmax, m represents the parameter interval margin, ω y is the weight corresponding to class y, and x is the learned feature vector. Experiments on the dataset Voxceleb2, which is augmented with noise and reverberation data, show that in the field of speaker recognition, the mv-softmax that simultaneously emphasizes the weight of misclassified samples and interval adaptation reduces the EER by 5.5% compared to am-softmax.

[0112] In summary, the present invention first inputs the speaker sample dataset and the phoneme sample dataset into the multi-task phoneme perception network model respectively, and according to the results output by the speaker classification subnet model and the phoneme classification subnet model, uses the maximum mean difference algorithm to calculate the difference loss value between the two results, and continuously trains the phoneme perception model by adding the difference loss value as the total loss, and finally obtains the trained model. The trained speaker classification subnet model has a high generalization ability for phoneme sample datasets in different domains, that is, the trained model can weaken the influence of the differences between the cross-domain phoneme sample datasets and the speaker sample datasets on the accuracy of the network model in recognizing speakers.

[0113] In addition, the present invention applies the maximum mean difference (MMD) difference minimization to the phoneme-aware network for the first time, improving the generalization ability of the speaker embedding generated by the speaker subnetwork under the condition of phoneme information in different domains. The maximum mean difference (MMD) is used to measure the distance between two distributions in the regenerated Hilbert space.

[0114] The misclassification vector guided loss function mv-softmax combines the advantages of sample interval loss (margin-based softmax) and difficult sample mining loss (mining-base softmax) for the first time. The present invention applies it to the speaker recognition network for the first time, which can make full use of the misclassified sample information in the training stage and enable the interval between samples to be adaptively changed in the training stage.

[0115] Exemplary Devices

[0116] This embodiment also provides a device for a network model training method for speaker recognition, the device comprising the following components:

[0117] A first result calculation module, used to input a speaker sample data set into a multi-task network model, extract a first result output by a set layer in a speaker classification subnet model in the multi-task network model, and the speaker classification subnet model is used for speaker classification training;

[0118] A second result calculation module, configured to input a cross-domain phoneme sample data set corresponding to the speaker sample data set into the multi-task network model, and extract a second result output by a set layer in a phoneme classification subnet model in the multi-task network, wherein the set layer in the phoneme classification subnet model corresponds to the set layer in the speaker classification subnet model, and the phoneme classification subnet model is used for phoneme classification training;

[0119] a difference loss value calculation module, used for applying a maximum mean difference algorithm to the first result and the second result to obtain a difference loss value corresponding to the first result and the second result;

[0120] The model training module is used to train the multi-task network model where the speaker classification subnet model and the phoneme classification subnet model are located according to the total loss to which the difference loss value is added, so as to obtain the trained network model.

[0121] Based on the above embodiments, the present invention further provides a terminal device, whose principle block diagram can be shown as follows: Figure 4As shown. The terminal device includes a processor, a memory, a network interface, a display screen, and a temperature sensor connected via a system bus. Among them, the processor of the terminal device is used to provide computing and control capabilities. The memory of the terminal device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the terminal device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a network model training method for speaker recognition is implemented. The display screen of the terminal device can be a liquid crystal display screen or an electronic ink display screen, and the temperature sensor of the terminal device is pre-set inside the terminal device to detect the operating temperature of the internal device.

[0122] Those skilled in the art will understand that Figure 4 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the terminal device to which the scheme of the present invention is applied. The specific terminal device may include more or less components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0123] In one embodiment, a terminal device is provided. The terminal device includes a memory, a processor, and a network model training program for speaker recognition stored in the memory and executable on the processor. When the processor executes the network model training program for speaker recognition, the following operation instructions are implemented:

[0124] Inputting a speaker sample data set into a multi-task network model, extracting a first result output by a set layer in a speaker classification subnet model in the multi-task network model, wherein the speaker classification subnet model is used for speaker classification training;

[0125] Inputting a cross-domain phoneme sample data set corresponding to the speaker sample data set into the multi-task network model, extracting a second result output by a set layer in a phoneme classification subnet model in the multi-task network, wherein the set layer in the phoneme classification subnet model corresponds to the set layer in the speaker classification subnet model, and the phoneme classification subnet model is used for phoneme classification training;

[0126] Applying a maximum mean difference algorithm to the first result and the second result to obtain a difference loss value corresponding to the first result and the second result;

[0127] According to the total loss to which the difference loss value is added, the multi-task network model where the speaker classification subnetwork model and the phoneme classification subnetwork model are located is trained to obtain the trained network model.

[0128] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0129] In summary, the present invention discloses a network model training method, device and storage medium for speaker recognition, the method comprising: firstly inputting a speaker sample data set and a phoneme sample data set into a multi-task phoneme network model respectively, and using a maximum mean difference algorithm to calculate the difference loss value between the two results according to the output results of the speaker classification subnet model and the phoneme classification subnet model, and continuously training the phoneme perception model by adding the total loss of the difference loss value, and finally obtaining the trained model. The speaker subnet model after training of the present invention has a high generalization ability for cross-domain phoneme sample data sets, that is, the trained model can weaken the influence of the difference between the phoneme sample data sets and the speaker sample data sets in different domains on the accuracy of the network model in recognizing speakers.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A network model training method for speaker recognition, characterized in that: include: Inputting a speaker sample data set into a multi-task network model, extracting a first result output by a set layer in a speaker classification subnet model in the multi-task network model, wherein the speaker classification subnet model is used for speaker classification training; Inputting a cross-domain phoneme sample data set corresponding to the speaker sample data set into the multi-task network model, extracting a second result output by a set layer in a phoneme classification subnet model in the multi-task network, wherein the set layer in the phoneme classification subnet model corresponds to the set layer in the speaker classification subnet model, and the phoneme classification subnet model is used for phoneme classification training; Applying a maximum mean difference algorithm to the first result and the second result to obtain a difference loss value corresponding to the first result and the second result; According to the total loss to which the difference loss value is added, the multi-task network model where the speaker classification subnetwork model and the phoneme classification subnetwork model are located is trained to obtain the trained network model; The method of training the multi-task network model where the speaker classification subnet model and the phoneme classification subnet model are located according to the total loss to which the difference loss value is added, to obtain the trained network model, comprises: Obtaining a speaker sample identity label corresponding to the speaker sample data set; Obtaining a trained speaker embedding model based on the speaker classification subnet model; Obtaining a predicted speaker identity label based on the speaker sample dataset and the speaker embedding model; Calculating an identity tag difference between the speaker sample identity tag and the predicted speaker identity tag; Obtaining the phoneme sample label corresponding to the phoneme sample data set; Obtaining predicted phoneme labels based on the phoneme sample data set and the phoneme classification subnet model; Calculating a phoneme difference between the phoneme sample label and the predicted phoneme label; The identity tag difference, the difference loss value, and the phoneme difference are used as the new total loss to train the multi-task network model where the speaker classification subnet model and the phoneme classification subnet model are located, so as to obtain the trained network model.

2. The network model training method for speaker recognition according to claim 1, characterized in that: The multi-task network model where the speaker classification subnet model and the phoneme classification subnet model are located is trained according to the identity tag difference, the difference loss value, and the phoneme difference to obtain the trained network model, including: Performing weighted calculation on the identity tag difference, the difference loss value, and the phoneme difference to obtain a total loss value corresponding to the network model; The network model is trained according to the total loss value to obtain the trained network model.

3. The network model training method for speaker recognition according to claim 1, characterized in that: The multi-task network model where the speaker classification subnet model and the phoneme classification subnet model are located is trained according to the identity tag difference, the difference loss value, and the phoneme difference to obtain the trained network model, including: According to the two phoneme classification subnetwork models, a frame-level phoneme classification subnetwork and a segment-level phoneme classification subnetwork in the phoneme classification subnetwork model are obtained; According to the phoneme sample data set and the frame-level phoneme classification subnetwork, obtain a predicted first phoneme label in the phoneme label; According to the phoneme sample data set and the segment-level phoneme classification subnetwork, obtain a predicted phoneme second label in the phoneme label; Calculating a phoneme first difference between the phoneme sample label and the predicted phoneme first label in the phoneme difference; calculating a phoneme second difference between the phoneme sample label and the predicted phoneme second label in the phoneme difference; The multi-task network model where the speaker classification subnet model and the phoneme classification subnet model are located is trained according to the identity tag difference, the difference loss value, the phoneme first difference, and the phoneme second difference to obtain the trained network model.

4. The network model training method for speaker recognition according to claim 1, characterized in that: Also includes: Obtaining a speaker sample identity label corresponding to the speaker sample data set; Obtaining the phoneme sample label corresponding to the phoneme sample data set; Obtaining a predicted speaker identity label based on the speaker sample data set and the trained speaker classification subnet model; Obtaining the predicted phoneme label according to the phoneme sample data set and the trained phoneme classification subnet model; The speaker sample whose identity label is different from the predicted speaker identity label is recorded as a speaker misclassification sample; Applying the misclassification vector guidance loss function to the speaker sample data set to obtain a loss function value for a new speaker; According to the loss function value of the new speaker, the total loss of the multi-task network after training is updated.

5. The network model training method for speaker recognition according to claim 1, characterized in that: The phoneme classification subnet model is a phoneme neural network model for identifying phoneme labels, the speaker classification subnet model is a speaker neural network model for identifying speaker identities, the set layer in the phoneme classification subnet model is a layer above the layer where the phoneme neural network model outputs phoneme labels, and the set layer in the speaker classification subnet model is a layer above the layer where the speaker neural network model outputs speaker identity labels.

6. A device for a network model training method for speaker recognition, characterized in that: The device comprises the following components: A first result calculation module, used to input a speaker sample data set into a multi-task network model, extract a first result output by a set layer in a speaker classification subnet model in the multi-task network model, and the speaker classification subnet model is used for speaker classification training; A second result calculation module, configured to input a cross-domain phoneme sample data set corresponding to the speaker sample data set into the multi-task network model, and extract a second result output by a set layer in a phoneme classification subnet model in the multi-task network, wherein the set layer in the phoneme classification subnet model corresponds to the set layer in the speaker classification subnet model, and the phoneme classification subnet model is used for phoneme classification training; a difference loss value calculation module, used for applying a maximum mean difference algorithm to the first result and the second result to obtain a difference loss value corresponding to the first result and the second result; A model training module, used for training the multi-task network model where the speaker classification subnet model and the phoneme classification subnet model are located according to the total loss to which the difference loss value is added, to obtain the trained network model; The method of training the multi-task network model where the speaker classification subnet model and the phoneme classification subnet model are located according to the total loss to which the difference loss value is added, to obtain the trained network model, comprises: Obtaining a speaker sample identity label corresponding to the speaker sample data set; Obtaining a trained speaker embedding model based on the speaker classification subnet model; Obtaining a predicted speaker identity label based on the speaker sample dataset and the speaker embedding model; Calculating an identity tag difference between the speaker sample identity tag and the predicted speaker identity tag; Obtaining the phoneme sample label corresponding to the phoneme sample data set; Obtaining predicted phoneme labels based on the phoneme sample data set and the phoneme classification subnet model; Calculating a phoneme difference between the phoneme sample label and the predicted phoneme label; The identity tag difference, the difference loss value, and the phoneme difference are used as the new total loss to train the multi-task network model where the speaker classification subnet model and the phoneme classification subnet model are located, so as to obtain the trained network model.

7. A terminal device, characterized in that: The terminal device includes a memory, a processor, and a network model training program for speaker recognition stored in the memory and executable on the processor. When the processor executes the network model training program for speaker recognition, the steps of the network model training method for speaker recognition as described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a network model training program for speaker recognition. When the network model training program for speaker recognition is executed by the processor, the steps of the network model training method for speaker recognition as described in any one of claims 1-5 are implemented.

Images