A method and system for identifying ordinary pronunciation speakers based on knowledge distillation

Abstract

This invention discloses a method and system for identifying ordinary pronunciation speakers based on knowledge distillation. The method includes: obtaining an embedding layer network for ordinary pronunciation speakers; acquiring the speaker's registered audio and extracting its spectral features, inputting the extracted registered audio spectrum into the ordinary pronunciation speaker embedding layer network to obtain the output of the registered audio spectrum in the ordinary pronunciation speaker embedding layer network; obtaining real-time audio data and detecting whether the real-time audio data is ordinary pronunciation; if so, extracting features from the real-time audio data and inputting the extracted real-time audio spectrum into the ordinary pronunciation speaker embedding layer network to obtain the output of the real-time audio spectrum in the ordinary pronunciation speaker embedding layer network; and calculating the cosine similarity between the output of the registered audio spectrum in the ordinary pronunciation speaker embedding layer network and the output of the real-time audio spectrum in the ordinary pronunciation speaker embedding layer network to determine whether they are the same speaker.

Description

Technical Field

[0001] This application relates to the field of speech recognition technology, and in particular to a method and system for identifying ordinary speakers based on knowledge distillation. Background Technology

[0002] Ordinary pronunciations refer to pronunciations that frequently occur in spoken dialogue and are subject to little subjective control by the speaker. They contain virtually no semantic content but are rich in sound information, such as coughs, laughter, the "hello" sound commonly used on the phone, the "um" sound indicating affirmation, and the sound of clearing one's throat. They are suitable for speaker recognition tasks in specific scenarios, such as in forensic identification when complete sentences cannot be obtained.

[0003] Although ordinary pronunciation can reflect some physical characteristics of the speaker's vocal tract, its short pronunciation time (average duration within 0.5 seconds) and limited speech content mean that speaker recognition systems developed using ordinary pronunciation are not robust enough to ordinary pronunciation. When using ordinary pronunciation to recognize speakers, the performance will drop significantly.

[0004] Furthermore, speaker recognition systems are often crucial for enabling other voice services on edge devices. However, due to the limited resources of edge devices, there is a high demand for fast inference and low power consumption. Current speaker recognition systems, however, typically have millions of model parameters, resulting in high model complexity, slow inference speed, and high computational power consumption, making them difficult to deploy on resource-constrained edge devices. However, directly reducing model complexity would further decrease the robustness of speaker recognition systems to mundane pronunciations.

[0005] Therefore, existing speaker recognition systems can only focus on the robustness of lightweight or ordinary pronunciation, and cannot take both into account. Summary of the Invention

[0006] To address the aforementioned issues, this patent proposes a method and system for identifying ordinary speakers based on knowledge distillation. This method utilizes knowledge distillation to obtain a lightweight embedding layer network for ordinary speakers, thereby improving the inference speed of the speaker recognition system, reducing the required storage and computational resources, and enhancing the system's robustness in identifying ordinary speakers.

[0007] To address the aforementioned technical problems, a first aspect of the present invention discloses a method for identifying ordinary pronunciation speakers based on knowledge distillation, the method comprising:

[0008] A speaker embedding layer network for ordinary pronunciation is obtained; wherein the speaker embedding layer network is obtained by knowledge distillation of a pre-constructed teacher network; the knowledge distillation includes: performing lightweight processing on the teacher network to construct the network structure of a lightweight student network to which the speaker embedding layer network belongs, and using the embedding layer output and posterior probability of the teacher network to guide the output of the lightweight student network to train the lightweight student network, and extracting the speaker embedding layer network from the lightweight student network after training is completed;

[0009] The speaker's registered audio is collected and its spectral features are extracted. The extracted registered audio spectrum is input into the ordinary pronunciation speaker embedding layer network to obtain the output of the registered audio spectrum in the ordinary pronunciation speaker embedding layer network; wherein, the registered audio is an ordinary pronunciation.

[0010] Obtain real-time audio data and detect whether the real-time audio data is a common pronunciation;

[0011] If so, feature extraction is performed on the real-time audio data, and the extracted real-time audio spectrum is input into the ordinary pronunciation speaker embedding layer network to obtain the output of the real-time audio spectrum in the ordinary pronunciation speaker embedding layer network;

[0012] The cosine similarity of the output of the registered audio spectrum in the ordinary pronunciation speaker embedding layer network and the output of the real-time audio spectrum in the ordinary pronunciation speaker embedding layer network is calculated to determine whether they are from the same speaker.

[0013] Preferably, the method for obtaining the embedding layer network of ordinary pronunciation speakers specifically includes:

[0014] Obtain an audio training set, wherein each audio sentence in the audio training set has a corresponding speaker label;

[0015] Construct and train the teacher network described above;

[0016] The lightweight student network is constructed and trained; wherein, during the construction process, the teacher network is lightweighted to obtain the lightweight student network; during the training process, the embedding layer output and posterior probability of the teacher network are used to guide the output of the lightweight student network.

[0017] Obtain the second speaker embedding layer network S of the lightweight student network after training is completed. e This serves as the embedding layer network for the so-called ordinary pronunciation speaker.

[0018] Preferably, the network structure of the teacher network includes: a first speaker embedding layer network T eAnd a first speaker classification layer T implemented with a fully connected layer. clf ;in,

[0019] The first speaker embedding layer network T e It is mainly composed of several convolutional layers connected in sequence and a fully connected layer. Each convolutional layer carries a batch normalized (BN) layer and a linear rectified unit (ReLU) layer.

[0020] Preferably, training the teacher network specifically includes:

[0021] The audio training set is divided into several rounds to train the teacher network until convergence; each round contains several batches, each batch has several different speakers, and each speaker corresponds to a long-term speech spectrum.

[0022] The long-term speech spectrum is sequentially input into the first speaker embedding layer network T. e and the first speaker classification layer T clf The first speaker classification layer T clf The output is passed through a normalized exponential function to obtain the posterior probability p for each speaker. 教师 Using speaker labels and posterior probability p 教师 Calculate the cross-entropy to obtain the loss L of the teacher network. 教师 Using the loss L 教师 Backpropagation updates the parameters of the teacher network.

[0023] Preferably, the long-time speech spectrum is obtained through the following steps:

[0024] Compare the speaker's corresponding audio spectrum with the set number of frames;

[0025] If the set number of long frames is exceeded, the spectrum corresponding to the set number of consecutive long frames is arbitrarily selected from the audio spectrum as the long-term speech spectrum;

[0026] If the set number of frames is not met, repeat the audio spectrum content until the set number of frames is met.

[0027] Preferably, the lightweight student network structure includes: a second speaker embedding layer network S e And a second speaker classification layer S implemented with a fully connected layer. clf ;in,

[0028] The second speaker embedding layer network S eIt is mainly composed of several convolutional layers connected in sequence and one fully connected layer. Each convolutional layer carries a batch normalized (BN) layer and a linear rectified unit (ReLU) layer. Among the several convolutional layers, except for the first layer which is a convolutional layer, the remaining layers are separable convolutional layers.

[0029] Preferably, training the lightweight student network specifically includes:

[0030] The audio training set is divided into several rounds to train the lightweight student network until convergence; each round contains several batches, each batch has several different speakers, and each speaker corresponds to the spectrum X of a short speech or trivial pronunciation. 短时 / 平凡 And an adversarial spectrum X based on short-term speech or mundane pronunciation. 对抗 ;

[0031] Loss function L for training lightweight student networks 学生 The expression is as follows:

[0032] L 学生 =KL 短时 / 平凡 (p 长时 p 短时 / 平凡 )+KL 对抗 (p 长时 p 对抗 )+L 短时 / 平凡 (e 长时 e 短时 / 平凡 )+L 对抗 (e 长时 e 对抗 )

[0033] Where KL represents relative entropy; KL 短时 / 平凡 (p 长时 p 短时 / 平凡 )=-p 长时 log(p 短时 / 平凡 ), p 长时 For long-term speech spectrum X 长时 The posterior probability in the teacher network is given by the spectrum X of the short-time speech or trivial pronunciation. 短时 / 平凡 The long-time speech spectrum X corresponding to the sentence 长时 After passing through the teacher network and applying a normalization function, p is obtained. 短时 / 平凡 The spectrum X of the short-time speech or trivial pronunciation 短时 / 平凡 The posterior probability in the lightweight student network is input as the spectrum X of the short-time speech or trivial pronunciation. 短时 / 平凡 The KL algorithm is obtained through the lightweight student network and using a normalization function. 对抗 (p 长时 p 对抗 )=-p 长时log(p 对抗 ), p 对抗 The adversarial spectrum X based on short-time speech or trivial pronunciation 对抗 The posterior probability in the lightweight student network is input as the adversarial spectrum X based on short-time speech or trivial pronunciation. 对抗 The result is obtained through the lightweight student network and using a normalization function; L represents the cosine distance loss. e 长时 The long-term speech spectrum X 长时 First speaker embedding layer network T e The output is the spectrum X of the short-time speech or trivial pronunciation, which is the input. 短时 / 平凡 The long-time speech spectrum X corresponding to the sentence 长时 Through the first speaker embedding layer network T in the teacher network e Get; e 短时 / 平凡 The spectrum X of the short-time speech or trivial pronunciation 短时 / 平凡 In the second speaker embedding layer network S e The output is the spectrum X of the short-time speech or trivial pronunciation, which is the input. 短时 / 平凡 Through the second speaker embedding layer network S of the lightweight student network e get; e 对抗 The adversarial spectrum X based on short-time speech or trivial pronunciation 对抗 In the second speaker embedding layer network S e The output is the adversarial spectrum X based on short-time speech or trivial pronunciation, and the input is the adversarial spectrum X. 对抗 Through the second speaker embedding layer network S of the lightweight student network e get;

[0034] Each batch of audio training sets is input into the lightweight student network, and the first speaker embedding layer network T of the teacher network is utilized. e The output and posterior probability are combined with the loss function L 学生 The spectrum X of the short-term speech or mundane pronunciation in the lightweight student network is guided. 短时 / 平凡 The output and the adversarial spectrum X based on short-time speech or trivial pronunciation 对抗 The output is used to train the lightweight student network.

[0035] Preferably, the spectrum X of the short-duration speech or mundane pronunciation 短时 / 平凡 It is obtained through the following steps:

[0036] Compare the speaker's corresponding audio spectrum with a set short frame rate;

[0037] If the set number of short frames is exceeded, the spectrum corresponding to the set number of consecutive short frames is arbitrarily selected from the audio spectrum as the spectrum X of the short-time speech or ordinary pronunciation. 短时 / 平凡 ;

[0038] If the set number of short frames is not met, repeat the audio spectrum content until the set number of short frames is met.

[0039] Preferably, the adversarial spectrum X based on short-time speech or mundane pronunciation 对抗 It is obtained through the following steps:

[0040] The spectrum X of the short-time speech or trivial pronunciation 短时 / 平凡 The initial adversarial spectrum is obtained by adding background noise according to the random signal-to-noise ratio.

[0041] Whether to continue is determined based on a set probability.

[0042] If not continuing, output the initial adversarial spectrum as the adversarial spectrum X based on short-time speech or trivial pronunciation. 对抗 ;

[0043] If we continue, based on the initial adversarial spectrum, we use the lightweight student network to obtain the initial adversarial spectrum in the second speaker embedding layer network S. e Output e 初始对抗 , and the posterior probability p of the initial adversarial spectrum in the lightweight student network 初始对抗 ;

[0044] Calculate e 初始对抗 The long-term speech spectrum X of the sentence corresponding to the initial adversarial spectrum 长时 In the first speaker embedding layer network T e Output e 长时 Cosine distance loss L 初始对抗 (e 长时 e 初始对抗 ), and p 初始对抗 The long-term speech spectrum X of the sentence corresponding to the initial adversarial spectrum 长时 The posterior probability p in the teacher network 长时 KL relative entropy KL 初始对抗 (p 长时 p 初始对抗 );

[0045] Obtain L 初始对抗 With KL 初始对抗 The gradient of the initial adversarial spectrum after addition;

[0046] After linearly adding the initial adversarial spectrum to the corresponding gradient by a set coefficient, the adversarial spectrum X based on short-time speech or trivial pronunciation is obtained. 对抗 .

[0047] Preferably, obtaining real-time audio data and detecting whether the real-time audio data is a common pronunciation specifically includes:

[0048] Obtain the real-time audio data and extract its spectral features;

[0049] The extracted spectral features are input into a trivial pronunciation detector to detect whether the real-time audio data is a trivial pronunciation; wherein, the trivial pronunciation detector is a neural network model trained with trivial pronunciations in the audio training set as positive samples and other audio as negative samples;

[0050] When the real-time audio data is not a mundane sound, continue to acquire and process other real-time audio data;

[0051] When the real-time audio data is a common pronunciation, the following steps are performed: "Extract features from the real-time audio data and input the extracted real-time audio spectrum into the common pronunciation speaker embedding layer network to obtain the output of the real-time audio spectrum in the common pronunciation speaker embedding layer network".

[0052] Preferably, the neural network model mainly consists of several convolutional layers and one binary classification fully connected layer stacked together; wherein,

[0053] The binary fully connected layer has two output nodes: one representing a common pronunciation and the other representing a negative sample. When the real-time audio data contains common pronunciations, the value of the output node belonging to the common pronunciation is higher than the value of the negative sample.

[0054] A second aspect of the present invention discloses a knowledge distillation-based system for identifying mundane pronunciation speakers, the system comprising:

[0055] An acquisition module is used to acquire a speaker embedding layer network for ordinary pronunciation; wherein the speaker embedding layer network is obtained by knowledge distillation of a pre-constructed teacher network; the knowledge distillation includes: performing lightweight processing based on the teacher network to construct the network structure of a lightweight student network to which the speaker embedding layer network belongs, and using the embedding layer output and posterior probability of the teacher network to guide the output of the lightweight student network for training the lightweight student network, and extracting the speaker embedding layer network from the lightweight student network after training is completed;

[0056] The first processing module is used to collect the registered audio of the speaker and extract its spectral features, input the extracted registered audio spectrum into the ordinary pronunciation speaker embedding layer network, and obtain the output of the registered audio spectrum in the ordinary pronunciation speaker embedding layer network; wherein, the registered audio is an ordinary pronunciation;

[0057] The judgment module is used to obtain real-time audio data and detect whether the real-time audio data is a common pronunciation;

[0058] The second processing module is used to extract features from the real-time audio data if the condition is met, and input the extracted real-time audio spectrum into the ordinary pronunciation speaker embedding layer network to obtain the output of the real-time audio spectrum in the ordinary pronunciation speaker embedding layer network.

[0059] The calculation module is used to perform cosine similarity calculation on the output of the registered audio spectrum in the ordinary pronunciation speaker embedding layer network and the output of the real-time audio spectrum in the ordinary pronunciation speaker embedding layer network to determine whether they are from the same speaker.

[0060] A third aspect of the present invention discloses a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the above-described method.

[0061] A fourth aspect of the present invention discloses a voice recognition device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described method.

[0062] Through one or more technical solutions of the present invention, the present invention has the following beneficial effects or advantages:

[0063] This invention discloses a method and system for speaker recognition based on knowledge distillation of mundane pronunciations. By utilizing knowledge distillation, the original large teacher network structure is lightweighted, resulting in a lightweight student network structure to which the speaker embedding layer network belongs. The embedded layer output and posterior probability of the teacher network guide the output of the lightweight student network for training. After training, the speaker embedding layer network is extracted from the lightweight student network. This makes the speaker embedding layer network structure even lighter. Because the lightweight student network is trained using the embedded layer output and posterior probability of the teacher network, its separability is comparable to that of the teacher network. Therefore, the speaker embedding layer network can simultaneously achieve both lightweight design and robustness in speaker recognition of mundane pronunciations, making it well-suited for deployment on edge devices. It is evident that, compared to existing speaker recognition schemes that only focus on lightweight design or robustness of mundane pronunciations, the method proposed in this invention can simultaneously meet both requirements.

[0064] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0065] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0066] Figure 1 A flowchart illustrating a method for identifying mundane pronunciation speakers based on knowledge distillation according to an embodiment of the present invention is shown.

[0067] Figure 2 A flowchart illustrating a specific method for creating a non-pronunciation speaker embedding layer network according to an embodiment of the present invention is shown.

[0068] Figure 3A A schematic diagram of the structure of a teacher network according to an embodiment of the present invention is shown;

[0069] Figure 3B A schematic diagram of a lightweight student network according to an embodiment of the present invention is shown;

[0070] Figure 4 An adversarial spectrum X according to an embodiment of the present invention is shown. 对抗 A schematic diagram of the generation process;

[0071] Figure 5 A schematic diagram of the structure of a loss function according to an embodiment of the present invention is shown;

[0072] Figure 6 A schematic diagram of a knowledge distillation-based mundane pronunciation speaker recognition system according to an embodiment of the present invention is shown. Detailed Implementation

[0073] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0074] To address the limitation of existing speaker recognition systems that can only focus on the robustness of either lightweight or mundane pronunciations and cannot simultaneously achieve both, this invention discloses a method and system for speaker recognition of mundane pronunciations based on knowledge distillation. By utilizing knowledge distillation, the originally large teacher network structure is lightweighted, resulting in a lightweight student network structure to which the speaker embedding layer network belongs. The embedded layer output and posterior probability of the teacher network guide the output of the lightweight student network for training. After training, the speaker embedding layer network is extracted from the lightweight student network. This makes the speaker embedding layer network structure even lighter. However, because the lightweight student network is trained using the embedded layer output and posterior probability of the teacher network, its separability is comparable to that of the teacher network. Therefore, the speaker embedding layer network can simultaneously achieve robustness in speaker recognition of both lightweight and mundane pronunciations, making it well-suited for deployment on edge devices.

[0075] To further illustrate and explain the present invention, please refer to the following: Figure 1 This invention discloses a method for identifying speakers of ordinary pronunciations based on knowledge distillation. The main purpose of this method is to identify the entity to which an ordinary pronunciation belongs. The entity to which an ordinary pronunciation belongs is not limited to humans; it can be applied to identifying various animals, such as parrots and dolphins. The method includes the following steps:

[0076] Step 1: Obtain the embedding layer network of ordinary pronunciation speakers.

[0077] The speaker embedding layer network is obtained by knowledge distillation of a pre-constructed teacher network. Furthermore, this embodiment improves upon knowledge distillation in two ways. First, it performs lightweighting on the teacher network to construct the network structure of a lightweight student network to which the speaker embedding layer network belongs. Second, it uses the embedding layer outputs and posterior probabilities of the teacher network to guide the outputs of the lightweight student network, thereby training the lightweight student network and extracting the speaker embedding layer network from it after training. Lightweighting refers to reducing the number of channels in the teacher network without changing its structure. For example, the number of channels in each convolutional layer of the teacher network is reduced by 1 / 4 to obtain the number of channels in each convolutional layer of the lightweight student network.

[0078] To illustrate and explain this embodiment, please refer to the following: Figure 2 This is a specific method for creating an embedding layer network for ordinary pronunciation speakers:

[0079] Step 101: Obtain the audio training set. The audio training set is used to train the teacher network and lightweight student network described below. Specifically, the audio training set needs to meet one or more of the following conditions: a) The number of speakers in the audio training set should exceed 1000, with a balanced gender ratio; b) The length of each audio sentence should be similar, for example, around 120 seconds; c) The number of ordinary pronunciations in each audio sentence should meet a set requirement, for example, 10 sentences; d) There should be no silent segments in the audio training set; e) Each audio sentence has a corresponding speaker label. Further, for the audio training set, extract the audio spectrum corresponding to each audio sentence; that is, each speaker corresponds to one audio spectrum, with the spectrum size being [spectral dimension, time frame number]. The frame length used for spectrum extraction can be selected as 25ms, and the frame shift is 10ms.

[0080] Step 102: Build and train a network of teachers.

[0081] The network structure of the teacher network is as follows: Figure 3A As shown, it includes: a first speaker embedding layer network T e And a first speaker classification layer T implemented with a fully connected layer. clf ;in,

[0082] First speaker embedding layer network T e It mainly consists of several sequentially connected convolutional layers and one fully connected layer stacked together. Each convolutional layer carries a BN (batch normalization) layer and a ReLU (Rectified Linear Unit) layer. Optionally, the teacher network is a VGG network model.

[0083] The teacher network in this embodiment is trained in the following manner:

[0084] First, the audio training set is divided into several rounds to train the teacher network until convergence. Each round contains several batches, each batch has several different speakers, and each speaker corresponds to a long-time speech spectrum. The long-time speech spectrum is obtained from the audio training set in step 101.

[0085] The long-term speech spectrum of this embodiment and the spectrum X of short-term speech or trivial pronunciation described in subsequent embodiments. 短时 / 平凡 Correspondingly, the number of frames in the spectrum of long-term speech is greater than the number of frames in the spectrum of short-term speech or ordinary articulation. 短时 / 平凡 The number of frames. Specifically, the long-term speech spectrum is obtained through the following steps: Since each speaker corresponds to an audio spectrum, the audio spectrum corresponding to the speaker is compared with the set number of long-term frames; the length of the set number of long-term frames should include as many frames as possible of all ordinary pronunciation spectra, for example, 200 frames. If the audio spectrum exceeds the set number of long-term frames, the spectrum corresponding to the set number of consecutive long-term frames is arbitrarily selected from the audio spectrum as the long-term speech spectrum; if the audio spectrum is insufficient for the set number of long-term frames, the audio spectrum content is repeated until the set number of long-term frames is met.

[0086] Secondly, during training, the long-term speech spectrum is sequentially input into the first speaker embedding layer network T. e And the first speaker classification layer T clf First speaker classification layer T clf The output is then passed through a normalized exponential function, softmax, to obtain the posterior probability p for each speaker. 教师 Using speaker labels and posterior probability p 教师 Calculate the cross-entropy to obtain the loss L of the teacher network. 教师 Using loss L 教师 Backpropagation updates the parameters of the teacher network until it meets the convergence condition. The trained teacher network performs well in speaker recognition for a set number of frames. The spectrum is processed through the first speaker embedding layer network T. e The output embedding layer data is informative and well-separated, making it highly suitable for lightweight student network learning; the first speaker classification layer T clf The probability output obtained through softmax is a soft label suitable for lightweight student network learning, which can represent the true distribution of speaker probabilities in the training set to a certain extent.

[0087] Step 103: Construct and train a lightweight student network.

[0088] In the construction process, the teacher network is lightweighted to obtain a lightweight student network; during training, the embedded layer output and posterior probability of the teacher network are used to guide the output of the lightweight student network.

[0089] Specifically, lightweight student network structures include... Figure 3B As shown, it includes: a second speaker embedding layer network S e And a second speaker classification layer S implemented with a fully connected layer. clf Among them, the second speaker embedding layer network S e It mainly consists of several convolutional layers stacked sequentially and one fully connected layer. Each convolutional layer carries a batch normalization (BN) layer and a ReLU (Rectified Localization) layer. Among the several convolutional layers, except for the first layer, the remaining layers are separable convolutional layers. Furthermore, the number of channels in each convolutional layer in the lightweight student network is less than the number of channels in the corresponding convolutional layer in the teacher network, for example, the number of channels is reduced by 1 / 4.

[0090] contrast Figure 3A and Figure 3B It can be seen that the lightweight student network and the teacher network have similar structures, with only two differences. One is that the lightweight student network uses more separable convolutional layers. The other is that each convolutional layer in the lightweight student network has fewer channels than the corresponding convolutional layer in the teacher network. Therefore, compared to the structurally dissimilar teacher-student networks, the structurally similar student network is better able to learn the first speaker embedding layer T from the teacher network. e And the first speaker classification layer T clf Characteristics of the distribution.

[0091] The lightweight student network is trained as follows:

[0092] First, the audio training set is divided into several rounds to train a lightweight student network until convergence. Each round contains several batches, and each batch has several different speakers, with each speaker corresponding to the spectrum X of a short speech or a trivial pronunciation. 短时 / 平凡 And an adversarial spectrum X based on short-term speech or mundane pronunciation. 对抗 The two spectra have the same number of time frames, which can be selected as 33 frames. The spectrum of short-duration speech or ordinary pronunciation X 短时 / 平凡 Obtained from the audio training set in step 101.

[0093] Among them, the spectrum X of short-duration speech or ordinary pronunciation 短时 / 平凡 It is obtained through the following steps:

[0094] Since each speaker corresponds to a single audio spectrum, the audio spectrum corresponding to that speaker is compared with a set number of short frames, for example, 33 frames. If the set number of short frames is exceeded, the spectrum corresponding to any consecutive set number of short frames is arbitrarily selected from the audio spectrum as the spectrum X of the short speech or ordinary pronunciation. 短时 / 平凡If the set short frame count is insufficient, repeat the audio spectrum content until the set short frame count is met.

[0095] To further illustrate and explain this embodiment, please refer to the following: Figure 4 Adversarial spectrum X based on short-time speech or mundane pronunciation 对抗 The input is the spectrum X of short-duration speech or ordinary pronunciation. 短时 / 平凡 It is obtained through the following steps:

[0096] Step 1031, analyze the spectrum X of short-duration speech or mundane articulation. 短时 / 平凡 Background noise is added according to a random signal-to-noise ratio to obtain an initial adversarial spectrum. The selectable range for the signal-to-noise ratio is [0dB, 20dB].

[0097] Step 1032: Determine whether to continue based on the set probability. The set probability can be set according to actual needs; for example, this embodiment specifies a set probability of 0.5, but this does not constitute a limitation.

[0098] If not proceeding, execute step 1033, outputting the initial adversarial spectrum as the adversarial spectrum X based on short-time speech or trivial pronunciation. 对抗 .

[0099] If we continue, we execute step 1034, and based on the initial adversarial spectrum, we use a lightweight student network to obtain the initial adversarial spectrum in the second speaker embedding layer network S. e Output e 初始对抗 And the posterior probability p of the initial adversarial spectrum in a lightweight student network 初始对抗 It is worth noting that the number of frames in the initial adversarial spectrum is related to the spectrum X of short-duration speech or trivial pronunciation. 短时 / 平凡 The frame rate is consistent.

[0100] Step 1035, calculate e 初始对抗 Long-term speech spectrum X of the sentence corresponding to the initial adversarial spectrum 长时 First speaker embedding layer network T e Output e 长时 Cosine distance loss L 初始对抗 (e 长时 e 初始对抗 ), and p 初始对抗 Long-term speech spectrum X of the sentence corresponding to the initial adversarial spectrum 长时 The posterior probability p in the teacher network 长时 KL relative entropy KL 初始对抗 (p 长时 p 初始对抗 ).

[0101] Step 1036, obtain L 初始对抗 (e 长时 e初始对抗 ) and KL 初始对抗 (p 长时 p 初始对抗 The gradient of the initial adversarial spectrum after addition.

[0102] Step 1037: After linearly adding the initial adversarial spectrum and the corresponding gradient by a set coefficient, the adversarial spectrum X based on short-time speech or trivial pronunciation is obtained. 对抗 The coefficients of the initial adversarial spectrum can be selected as 1, and the coefficients of the gradient can be randomly selected from (0, 1e-3).

[0103] Loss function L for training lightweight student networks 学生 The different loss functions involved are as follows: Figure 5 As shown in the dashed box, the expression is as follows:

[0104] L 学生 =KL 短时 / 平凡 (p 长时 p 短时 / 平凡 )+KL 对抗 (p 长时 p 对抗 )+L 短时 / 平凡 (e 长时 e 短时 / 平凡 )+L 对抗 (e 长时 e 对抗 )

[0105] Where KL represents relative entropy; KL 短时 / 平凡 (p 长时 p 短时 / 平凡 )=-p 长时 log(p 短时 / 平凡 ), p 长时 For long-term speech spectrum X 长时 In the teacher network, the posterior probability is given by the spectrum X of a short-duration speech or trivial pronunciation. 短时 / 平凡 Long-term speech spectrum X of the corresponding sentence 长时 After passing through the teacher network and using a normalization function, p is obtained. 短时 / 平凡 The spectrum X of short-duration speech or ordinary pronunciation 短时 / 平凡 The posterior probability in a lightweight student network, where the input is the spectrum X of a short-duration speech or a trivial pronunciation. 短时 / 平凡 This was obtained by simplifying the student network and using a normalization function; KL 对抗 (p 长时 p 对抗 )=-p 长时 log(p 对抗 ), p 对抗 For adversarial spectrum X based on short-time speech or trivial pronunciation 对抗The posterior probability in a lightweight student network, with input being an adversarial spectrum X based on short-time speech or trivial pronunciation. 对抗 The result is obtained by simplifying the student network and using a normalization function; L represents the cosine distance loss. e 长时 For long-term speech spectrum X 长时 First speaker embedding layer network T e The output is the spectrum X of a short-duration speech or trivial pronunciation, which is the input. 短时 / 平凡 Long-term speech spectrum X of the corresponding sentence 长时 Through the first speaker embedding layer network T in the teacher network e Get; e 短时 / 平凡 The spectrum X of short-duration speech or ordinary pronunciation 短时 / 平凡 Second speaker embedding layer network S e The output is the spectrum X of a short-duration speech or trivial pronunciation, which is the input. 短时 / 平凡 Through a lightweight student network, the second speaker embedding layer network S e get; e 对抗 For adversarial spectrum X based on short-time speech or trivial pronunciation 对抗 Second speaker embedding layer network S e The output is the adversarial spectrum X based on short-time speech or trivial pronunciation. 对抗 Through a lightweight student network, the second speaker embedding layer network S e get;

[0106] Each batch of audio training data is fed into a lightweight student network, and the first speaker embedding layer network T of the teacher network is utilized. e The output and posterior probability are combined with the loss function L 学生 Spectrum X of short-duration or ordinary pronunciation in student online learning is used to guide lightweight learning. 短时 / 平凡 The output and the adversarial spectrum X based on short-time speech or trivial pronunciation. 对抗 The output is used to train a lightweight student network.

[0107] When training the lightweight student network, a knowledge distillation method is used to guide the learning process of short-term speech or trivial pronunciation spectra using the embedding layer output and posterior probability of the teacher network. Therefore, the trivial pronunciation embedding layer of the lightweight student network approximates the embedding layer of the teacher network, and the separability of the embedding layer is also similar, ensuring speaker recognition performance based on trivial pronunciations. Furthermore, the use of adversarial spectra in training, guided by the teacher model, further improves the robustness of the student network's speaker embedding layer to noise. As seen in the generation process of adversarial spectra, adversarial examples have a certain probability (e.g., 0.5) of directly adding noise, and another 0.5 probability of transforming into a more difficult-to-recognize spectrum along the gradient increase direction. These two involve different noisy conditions during training. Using the embedding layer and posterior probability of the teacher model to guide the student model's training based on adversarial examples ultimately improves the overall robustness of the student model.

[0108] Furthermore, the spectrum used in this embodiment can be either the spectrum of short-duration speech or the spectrum of ordinary pronunciations. Both are used together as training samples to train a lightweight student network, expand the dataset, and improve the overall student network, including the second speaker embedding layer S. e Second speaker classification layer S clf The ability to separate speech signals is due to the fact that when the number of time frames in a short-time speech spectrum is short enough, the corresponding speech information is phoneme information, and its physical characteristics are similar to those of ordinary pronunciations. Therefore, it can be used to expand the dataset and improve the speaker recognition ability based on ordinary pronunciations.

[0109] Compared to existing speaker recognition systems that focus only on lightweight design or robustness of ordinary pronunciations, the method proposed in this embodiment satisfies both requirements.

[0110] Step 104: Obtain the second speaker embedding layer network S of the trained lightweight student network. e Embedded layer network as a common pronunciation speaker.

[0111] The above describes the creation process of the ordinary pronunciation speaker embedding layer network. The number of parameters and computational cost of the ordinary pronunciation speaker embedding layer network are far less than those of the teacher network's first speaker embedding layer network T. e Therefore, it is suitable for deployment on resource-constrained edge devices. Furthermore, because the lightweight student network's output based on short-time speech or trivial pronunciation spectra is guided by the embedding layer output and posterior probabilities of the teacher network during training, the trivial pronunciation embedding layer of the lightweight student network will closely resemble the teacher network's embedding layer, and its separability will also be similar. In addition, the use of adversarial spectrum in training, also guided by the teacher model, can further improve the robustness of the trivial pronunciation speaker embedding layer network to noise.

[0112] Step 2: Collect the speaker's registered audio and extract its spectral features. Input the extracted registered audio spectrum into the ordinary pronunciation speaker embedding layer network to obtain the output of the registered audio spectrum in the ordinary pronunciation speaker embedding layer network.

[0113] The registered audio is a common pronunciation. However, during spectral feature extraction, it needs to be processed according to a set short frame count requirement to obtain a qualified registered audio spectrum.

[0114] Step 3: Obtain real-time audio data and detect whether the real-time audio data is a common pronunciation.

[0115] This embodiment utilizes a trivial pronunciation detector within a neural network model for judgment. To further illustrate and explain this embodiment, the specific detection process is described below: First, real-time audio data is obtained and spectral features are extracted. Second, the extracted spectral features are input into the trivial pronunciation detector to detect whether the real-time audio data contains trivial pronunciations. The trivial pronunciation detector is a neural network model trained using trivial pronunciations from the audio training set as positive samples and other audio samples as negative samples. The neural network model mainly consists of several convolutional layers and one binary fully connected layer. The binary fully connected layer has two output nodes: one representing a trivial pronunciation and the other representing a negative sample. When the real-time audio data contains trivial pronunciations, the value of the output node representing a trivial pronunciation in the fully connected layer is higher than the value of a negative sample.

[0116] When the real-time audio data is not a mundane sound, continue to acquire and process other real-time audio data.

[0117] If the real-time audio data is a mundane pronunciation, proceed to step 4.

[0118] Step 4: Extract features from the real-time audio data and input the extracted real-time audio spectrum into the ordinary speech embedding layer network to obtain the output of the real-time audio spectrum in the ordinary speech embedding layer network.

[0119] Step 5: Perform cosine similarity calculation on the output of the registered audio spectrum in the ordinary pronunciation speaker embedding layer network and the output of the real-time audio spectrum in the ordinary pronunciation speaker embedding layer network to determine whether they are from the same speaker.

[0120] Specifically, if the cosine similarity exceeds a set threshold, the speaker in the real-time audio data is considered to be the same as the registered speaker; otherwise, they are not. After the judgment is completed, a neural network model can be used to detect the next real-time audio data.

[0121] Based on the same inventive concept as in the foregoing embodiments, this invention also discloses a knowledge distillation-based mundane speech speaker recognition system, see below. Figure 6The system includes:

[0122] A module 601 is used to obtain a speaker embedding layer network for ordinary pronunciation; wherein the speaker embedding layer network is obtained by knowledge distillation of a pre-constructed teacher network; the knowledge distillation includes: performing lightweight processing based on the teacher network to construct the network structure of a lightweight student network to which the speaker embedding layer network belongs, and using the embedding layer output and posterior probability of the teacher network to guide the output of the lightweight student network to train the lightweight student network, and extracting the speaker embedding layer network from the lightweight student network after training is completed;

[0123] The first processing module 602 is used to collect the registered audio of the speaker and extract its spectral features, input the extracted registered audio spectrum into the ordinary pronunciation speaker embedding layer network, and obtain the output of the registered audio spectrum in the ordinary pronunciation speaker embedding layer network; wherein, the registered audio is an ordinary pronunciation;

[0124] The judgment module 603 is used to obtain real-time audio data and detect whether the real-time audio data is a common pronunciation;

[0125] The second processing module 604 is used to extract features from the real-time audio data if the condition is met, and input the extracted real-time audio spectrum into the ordinary pronunciation speaker embedding layer network to obtain the output of the real-time audio spectrum in the ordinary pronunciation speaker embedding layer network.

[0126] The calculation module 605 is used to perform cosine similarity calculation on the output of the registered audio spectrum in the ordinary pronunciation speaker embedding layer network and the output of the real-time audio spectrum in the ordinary pronunciation speaker embedding layer network to determine whether they are the same speaker.

[0127] Based on the same inventive concept as in the foregoing embodiments, this embodiment of the invention also discloses a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described above.

[0128] Based on the same inventive concept as in the foregoing embodiments, this embodiment of the invention also discloses a voice recognition device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the methods described above.

[0129] Through one or more embodiments of the present invention, the present invention has the following beneficial effects or advantages:

[0130] This invention discloses a method and system for speaker recognition based on knowledge distillation, which lightweights the original large teacher network structure using knowledge distillation to construct a lightweight student network structure to which the speaker embedding layer network belongs. The embedded layer output and posterior probability of the teacher network guide the output of the lightweight student network for training. After training, the speaker embedding layer network is extracted from the lightweight student network. This makes the speaker embedding layer network structure more lightweight. Because the lightweight student network is trained using the embedded layer output and posterior probability of the teacher network, its separability is comparable to that of the teacher network. Therefore, the speaker embedding layer network can simultaneously achieve both lightweight design and robustness in speaker recognition of ordinary pronunciations, making it well-suited for deployment on edge devices. It is evident that, compared to existing speaker recognition schemes that only focus on lightweight design or robustness of ordinary pronunciations, the method proposed in this invention can simultaneously meet both requirements.

[0131] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0132] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0133] Similarly, it should be understood that, in order to simplify the invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, this disclosure should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into this detailed description, wherein each claim itself is a separate embodiment of the invention.

[0134] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0135] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0136] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components of the gateway, proxy server, or system according to embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., a computer program and computer program product) for performing some or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0137] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

Claims

1. A method for identifying speakers with ordinary pronunciation based on knowledge distillation, characterized in that, The method includes: Obtain a network for the speaker embedding layer of speakers with ordinary pronunciation, specifically including: obtaining an audio training set, where each audio in the audio training set has a corresponding speaker label; constructing and training a teacher network; constructing and training a lightweight student network; obtaining the second speaker embedding layer network of the lightweight student network that has completed training As the network for the speaker embedding layer of speakers with ordinary pronunciation; wherein, the network for the speaker embedding layer is obtained by knowledge distillation of the pre-constructed teacher network; the knowledge distillation includes: performing lightweight processing based on the teacher network to construct the network structure of the lightweight student network to which the network for the speaker embedding layer belongs, and using the embedding layer output and posterior probability of the teacher network to guide the output of the lightweight student network to train the lightweight student network, and extracting the network for the speaker embedding layer from the lightweight student network after the training is completed; Collecting the registered audio of the speaker and performing spectral feature extraction, and inputting the extracted registered audio spectrum into the ordinary pronunciation speaker embedding layer network to obtain the output of the registered audio spectrum in the ordinary pronunciation speaker embedding layer network; wherein, the registered audio is ordinary pronunciation; Obtaining real-time audio data and detecting whether the real-time audio data is ordinary pronunciation; If so, performing feature extraction on the real-time audio data, and inputting the extracted real-time audio spectrum into the ordinary pronunciation speaker embedding layer network to obtain the output of the real-time audio spectrum in the ordinary pronunciation speaker embedding layer network; Calculating the cosine similarity between the output of the registered audio spectrum in the ordinary pronunciation speaker embedding layer network and the output of the real-time audio spectrum in the ordinary pronunciation speaker embedding layer network to determine whether it is the same speaker.

2. The method according to claim 1, wherein The network structure of the teacher network includes: a first speaker embedding layer network and a first speaker classification layer implemented by a fully connected layer ; where The first speaker embedding layer network is mainly composed of a stack of several sequentially connected convolutional layers and a fully connected layer, and each convolutional layer is equipped with a batch normalization (BN) layer and a rectified linear unit (ReLU) layer.

3. The method according to claim 2, wherein The training of the teacher network specifically includes: Dividing the audio training set into several rounds to train the teacher network until convergence; wherein, each round contains several batches, each batch has several different speakers, and each speaker corresponds to a long-term speech spectrum; Input the long-term speech spectrum into the first speaker embedding layer network in sequence and the first speaker classification layer The output of the first speaker classification layer After passing through the softmax function, the posterior probability of each speaker is obtained Using the speaker label and the posterior probability Calculate the cross entropy to obtain the loss of the teacher network Using the loss Backpropagate to update the parameters of the teacher network 4. The method according to claim 3, wherein The long-term speech spectrum is obtained through the following steps: Comparing the audio spectrum corresponding to the speaker with the set long-term number of frames; If it exceeds the set long-term number of frames, arbitrarily select the spectrum corresponding to the continuous set long-term number of frames from the audio spectrum as the long-term speech spectrum; If it is less than the set long-term number of frames, repeat the audio spectrum content until the set long-term number of frames is satisfied.

5. The method according to claim 1, wherein The network structure of the lightweight student includes: a second speaker embedding layer network and a second speaker classification layer implemented by a fully connected layer ; where The second speaker embedding layer network It is mainly composed of a number of sequentially connected convolutional layers and a fully connected layer stacked together. Each convolutional layer is equipped with a batch normalization (BN) layer and a rectified linear unit (ReLU) layer; among the number of convolutional layers, except for the first layer which is a convolutional layer, the remaining layers are separable convolutional layers.

6. The method according to claim 5, wherein The training of the lightweight student network specifically includes: Divide the audio training set into several rounds for training the lightweight student network until convergence; where each round contains several batches, each batch has several different speakers, and each speaker corresponds to a spectrogram of a short speech or a normal pronunciation , and a spectrogram of an adversarial speech based on the short speech or normal pronunciation ; Loss function for training lightweight student network , and the expression is as follows: Among them, represents relative entropy; , is the long-term speech spectrum The posterior probability in the teacher network, and the input is the spectrum of the short-term speech or trivial pronunciation The long-term speech spectrum corresponding to the sentence , obtained after passing through the teacher network and using a normalization function, is the spectrum of the short-term speech or trivial pronunciation The posterior probability in the lightweight student network, and the input is the spectrum of the short-term speech or trivial pronunciation , obtained by passing through the lightweight student network and using a normalization function; is the adversarial spectrum based on the short-term speech or trivial pronunciation The posterior probability in the lightweight student network, and the input is the adversarial spectrum based on the short-term speech or trivial pronunciation , obtained by passing through the lightweight student network and using a normalization function; represents the cosine distance loss; , is the long-term speech spectrum The output in the first speaker embedding layer network , and the input is the spectrum of the short-term speech or trivial pronunciation The long-term speech spectrum corresponding to the sentence , obtained through the first speaker embedding layer network in the teacher network ; is the spectrum of the short-term speech or trivial pronunciation The output in the second speaker embedding layer network , and the input is the spectrum of the short-term speech or trivial pronunciation , obtained through the second speaker embedding layer network of the lightweight student network ; is the adversarial spectrum based on the short-term speech or trivial pronunciation The output in the second speaker embedding layer network , and the input is the adversarial spectrum based on the short-term speech or trivial pronunciation , obtained through the second speaker embedding layer network of the lightweight student network ; Input each batch of the audio training set into the lightweight student network, and combine the output of the first speaker embedding layer network of the teacher network and the posterior probability with the loss function to guide the lightweight student network to output the spectrum of the short-time speech or ordinary pronunciation and the output of the adversarial spectrum based on the short-time speech or ordinary pronunciation for training the lightweight student network. ​ 7. The method according to claim 6, wherein The spectrum of the short-time speech or ordinary pronunciation is obtained through the following steps: Comparing the audio spectrum corresponding to the speaker with the set short-term number of frames; If the set short time frame number is exceeded, arbitrarily select the spectrum corresponding to the consecutive set short time frame numbers from the audio spectrum as the spectrum of the short-time speech or ordinary pronunciation ; If it is less than the set short-term number of frames, repeat the audio spectrum content until the set short-term number of frames is satisfied.

8. The method according to claim 6, wherein The adversarial spectrum based on short-time speech or ordinary pronunciation is obtained through the following steps: The spectrum of the short-time speech or ordinary pronunciation Add background noise according to a random signal-to-noise ratio to obtain an initial adversarial spectrum; Judging whether to continue based on the set probability; If not continuing, output the initial adversarial spectrum as the adversarial spectrum based on short-time speech or ordinary pronunciation ; If continuing, based on the initial adversarial spectrum, use the lightweight student network to respectively obtain the output of the initial adversarial spectrum in the second speaker embedding layer network and the posterior probability of the initial adversarial spectrum in the lightweight student network ; Calculation The long - term speech spectrum corresponding to the sentence of the initial adversarial spectrum At the output of the first - speaker embedding layer network Output Cosine - distance loss And The long - term speech spectrum corresponding to the sentence of the initial adversarial spectrum The posterior probability in the teacher network KL relative entropy ; Obtain and the gradient of the initial adversarial spectrum after addition; After linearly adding the initial adversarial spectrum and the corresponding gradient according to a set coefficient, an adversarial spectrum based on short-time speech or ordinary pronunciation is obtained .

9. The method according to claim 1, wherein The obtaining of the real-time audio data and the detection of whether the real-time audio data is ordinary pronunciation specifically includes: Obtaining the real-time audio data and performing spectral feature extraction; Inputting the extracted spectral features into an ordinary pronunciation detector to detect whether the real-time audio data is ordinary pronunciation; wherein, the ordinary pronunciation detector is a neural network model trained with ordinary pronunciation in the audio training set as positive samples and other audios as negative samples; When the real-time audio data is not ordinary pronunciation, continue to obtain and process other real-time audio data; When the real-time audio data is ordinary pronunciation, execute the step of "performing feature extraction on the real-time audio data, and inputting the extracted real-time audio spectrum into the ordinary pronunciation speaker embedding layer network to obtain the output of the real-time audio spectrum in the ordinary pronunciation speaker embedding layer network".

10. The method according to claim 9, wherein The neural network model is mainly composed of several convolutional layers and a binary classification fully connected layer stacked; wherein, The output nodes of the binary classification fully connected layer have two, one representing ordinary pronunciation and the other representing negative samples; when the real-time audio data contains ordinary pronunciation, the value of the fully connected layer belonging to the ordinary pronunciation output node is higher than the value of the negative sample.

11. A speaker recognition system for ordinary pronunciation based on knowledge distillation, characterized in that, The system includes: An acquisition module for acquiring a speaker embedding layer network for ordinary pronunciation, specifically including: acquiring an audio training set, where each audio sentence in the audio training set has a corresponding speaker label; constructing and training a teacher network; constructing and training a lightweight student network; and obtaining the second speaker embedding layer network of the trained lightweight student network As the speaker embedding layer network for ordinary pronunciation; wherein, the speaker embedding layer network is obtained by knowledge distillation of the pre-constructed teacher network; the knowledge distillation includes: performing lightweight processing based on the teacher network to construct the network structure of the lightweight student network to which the speaker embedding layer network belongs, and using the embedding layer output and posterior probability of the teacher network to guide the output of the lightweight student network to train the lightweight student network, and extracting the speaker embedding layer network from the lightweight student network after the training is completed; The first processing module is used to collect the registration audio of the speaker, extract the spectral features, and input the extracted registration audio spectrum into the ordinary pronunciation speaker embedding layer network to obtain the output of the registration audio spectrum in the ordinary pronunciation speaker embedding layer network; wherein, the registration audio is ordinary pronunciation; The judgment module is used to obtain real-time audio data and detect whether the real-time audio data is ordinary pronunciation; The second processing module is used to, if so, extract the features of the real-time audio data, input the extracted real-time audio spectrum into the ordinary pronunciation speaker embedding layer network, and obtain the output of the real-time audio spectrum in the ordinary pronunciation speaker embedding layer network; The calculation module is used to calculate the cosine similarity between the output of the registration audio spectrum in the ordinary pronunciation speaker embedding layer network and the output of the real-time audio spectrum in the ordinary pronunciation speaker embedding layer network to determine whether it is the same speaker.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method according to any one of claims 1-10.

13. A voice recognition device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1-10.

Images