Cross-channel invariant voiceprint feature extraction method and system based on meta-learning
Generate channel-independent voice embedding through meta-learning, solving the problem of low performance of the automatic speaker verification system in cross-channel scenarios, achieving higher recognition rate and better channel adaptability.
Patent Information
- Application Number
- CN202510765730.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-09
- Publication Date
- 2025-08-26
AI Technical Summary
The existing automatic speaker verification system based on deep neural networks has severely reduced performance in cross-channel scenarios, especially in the absence of channel recognition, and lacks domain adaptability.
The meta-learning method is used to generate channel-independent voice embeddings. By building a meta-speech embedding network, combining meta-optimization strategies and global distribution optimization, a channel-invariant voice embedding space is generated, and robust voiceprint features are extracted for cross-channel verification.
The recognition rate of the automatic speaker verification system is significantly improved, the adverse impact of channel mismatch on the recognition results is reduced, and the model's adaptability in the unseen channel is enhanced.
Smart Images
Figure CN120544582A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of speech recognition technology, and more particularly to a meta-learning-based cross-channel invariant voiceprint feature extraction method and system. Background Art
[0002] Automatic speaker verification (ASV) systems based on deep neural networks (DNNs) have been successfully applied to identity verification. However, in real-world scenarios, channel mismatch caused by recording equipment and environments severely degrades performance, especially in unseen channels. Therefore, techniques to eliminate channel mismatch are essential for cross-channel speaker verification. Classic cross-channel speaker verification methods generally utilize feature projection to improve sample discriminability or employ adversarial training to learn speech representations. However, these methods lack domain adaptation capabilities for unseen channels. This limitation can cause the model to overfit to the prior knowledge of the channel in the training set, resulting in poor performance when processing speech from unseen channels. Meta-learning, as an effective domain adaptation technique, has been applied to fields such as face recognition and small-sample recognition. Meta-learning enhances the learning ability of neural networks rather than their ability to solve specific problems, enabling rapid adaptation to new domains. Therefore, meta-learning can be used to address the problem of cross-channel voiceprint extraction.
[0003] Therefore, in view of the current situation where the performance of AVS is severely reduced in cross-channel scenarios, it is an urgent problem that technicians in this field need to solve to propose a cross-channel invariant voiceprint feature extraction method and system based on meta-learning. Summary of the Invention
[0004] In view of this, the present invention provides a cross-channel invariant voiceprint feature extraction method and system based on meta-learning, which generates channel-independent sentence embeddings based on meta-learning, aiming to reduce the adverse impact of channel mismatch on the recognition results of the automatic speaker verification system and significantly improve its recognition rate.
[0005] To achieve the above objectives, the present invention adopts the following technical solution: a cross-channel invariant voiceprint feature extraction method based on meta-learning, comprising:
[0006] Step 1: Obtain multi-channel voice data;
[0007] Step 2: constructing a meta-speech embedding network, and inputting the multi-channel speech data to train the meta-speech embedding network;
[0008] Step 3: Optimize the trained meta-speech embedding network by simulating cross-channel scenarios to generate training tasks. Combined with meta-optimization strategies and global distribution optimization, the meta-speech embedding network is adjusted to generate a channel-invariant speech embedding space.
[0009] Step 4: For unknown channel speech data, use the trained meta-speech embedding network to extract voiceprint features, generate robust speech voiceprint embedding, and perform cross-channel speech verification.
[0010] Preferably, optimizing the trained meta-speech embedding network includes:
[0011] Step 31: sampling the multi-channel speech data to construct a simulated cross-channel scenario of the meta-task;
[0012] Step 32: Optimize the simulated cross-channel scenario using a meta-optimization strategy to generate a channel-invariant embedding space, and stabilize the embedding space distribution through global distribution optimization;
[0013] Step 33: Optimize the meta-speech embedding network by simulating cross-channel scenarios, meta-optimization strategies, and global distribution optimization to stabilize the embedding distribution.
[0014] Preferably, the meta-task includes a source data set of N channels Randomly extract P samples to construct the support set and Q samples to construct the query set; a pair of support set and query set constitutes a meta-task; through meta-task sampling, the source dataset is converted into a set of meta-tasks to simulate cross-channel scenarios
[0015] in is the source data from different channels, For the different meta-tasks constituted.
[0016] Preferably, the meta-optimization strategy optimization is achieved by reducing the meta-optimization loss, the meta-optimization loss L MO The calculation formula is as follows:
[0017]
[0018] where f θ is the voiceprint extraction model, C k is the average embedding of the selected speakers k=1,…,B in the support set, B is the number of speakers selected in the meta-task, P is the number of corpora for each speaker in the support set, To support the embedding of the p-th corpus of speaker k, d q,k is the cosine similarity between each corpus embedding and the average embedding in the query set, is the embedding of the qth corpus in the query set, L MO is the multi-class cross entropy, C=[C1,C2,…,C k-1 , C k ] is the average embedding set in the support set, Q is the number of corpora for each speaker in the query set, and K is the single-click form of speaker k selected in the meta-task.
[0019] Preferably, the global distribution optimization is achieved by reducing the global distribution loss, the global distribution loss L DO The calculation formula is as follows:
[0020]
[0021] where N s is the number of speakers, is the linear layer for classification, is the embedding of the i-th corpus in the meta-task, and Y is the one-hot encoding form of speaker y.
[0022] Preferably, the cross-channel voice verification method includes:
[0023] During the testing phase, multidimensional log mel-filterbank features are extracted from the input speech to be authenticated from other unknown channels, and M consecutive frames are randomly intercepted as input features. The input features are fed into a meta-speech embedding network based on a deep residual network structure combined with global average pooling to generate an N-dimensional speech embedding. The speaker identity is determined by calculating the cosine similarity between the input speech embedding and the registered speech embedding.
[0024] Preferably, the calculation formula for determining the speaker's identity is as follows:
[0025]
[0026] Among them, d test is the cosine similarity between the speech embedding and the registered speech embedding, and τ is the threshold; if the cosine similarity between the speech embedding and the registered speech embedding is greater than the threshold, it is determined that they are the same person; if the cosine similarity between the speech embedding and the registered speech embedding is less than the threshold, it is determined that they are not the same person.
[0027] Preferably, a meta-learning-based cross-channel invariant voiceprint feature extraction system includes:
[0028] Data acquisition module: used to acquire multi-channel voice data;
[0029] Training module: used for constructing a meta-speech embedding network and inputting the multi-channel speech data to train the meta-speech embedding network;
[0030] Optimization module: This module optimizes the trained meta-speech embedding network by simulating cross-channel scenarios to generate training tasks. It then adjusts the meta-speech embedding network using a meta-optimization strategy and global distribution optimization to generate a channel-invariant speech embedding space.
[0031] Verification module: It is used to extract voiceprint features from unknown channel voice data using the trained meta-speech embedding network, generate robust speech embeddings, and perform cross-channel voice verification.
[0032] It can be seen from the above technical solution that compared with the existing technology, the present invention discloses a cross-channel invariant voiceprint feature extraction method and system based on meta-learning. In response to the current situation where the performance of AVS is severely reduced in cross-channel scenarios, a cross-channel invariant voiceprint feature extraction method based on meta-learning is provided. This method can generate channel-independent sentence embeddings, reduce the adverse effects of channel mismatch on the recognition results of the automatic speaker verification system, and improve its recognition rate. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0034] Figure 1 Flowchart of the cross-channel invariant voiceprint feature extraction method based on meta-learning provided by the present invention;
[0035] Figure 2 This is a flow chart for determining the speaker's identity provided by the present invention. DETAILED DESCRIPTION
[0036] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.
[0037] like Figure 1 As shown, the embodiment of the present invention discloses a cross-channel invariant voiceprint feature extraction method based on meta-learning, comprising:
[0038] Step 1: Obtain multi-channel voice data;
[0039] Step 2: constructing a meta-speech embedding network, and inputting the multi-channel speech data to train the meta-speech embedding network;
[0040] Step 3: Optimize the trained meta-speech embedding network by simulating cross-channel scenarios to generate training tasks. Combined with meta-optimization strategies and global distribution optimization, the meta-speech embedding network is adjusted to generate a channel-invariant speech embedding space.
[0041] Step 4: For unknown channel speech data, use the trained meta-speech embedding network to extract voiceprint features, generate robust speech voiceprint embedding, and perform cross-channel speech verification.
[0042] Specifically, the optimization of the trained meta-speech embedding network includes:
[0043] Step 31: sampling the multi-channel speech data to construct a simulated cross-channel scenario of the meta-task;
[0044] Step 32: Optimize the simulated cross-channel scenario using a meta-optimization strategy to generate a channel-invariant embedding space, and stabilize the embedding space distribution through global distribution optimization;
[0045] Step 33: Optimize the meta-speech embedding network by simulating cross-channel scenarios, meta-optimization strategies, and global distribution optimization to stabilize the embedding distribution.
[0046] Specifically, the meta-task includes a source data set of N channels Randomly extract P samples to construct the support set and Q samples to construct the query set; a pair of support set and query set constitutes a meta-task; through meta-task sampling, the source dataset is converted into a set of meta-tasks to simulate cross-channel scenarios
[0047] in is the source data from different channels, For the different meta-tasks constituted.
[0048] Specifically, the meta-optimization strategy optimization is achieved by reducing the meta-optimization loss, the meta-optimization loss L MO The calculation formula is as follows:
[0049]
[0050] where f θ is the voiceprint extraction model, C k is the average embedding of the selected speakers k=1,…,B in the support set, B is the number of speakers selected in the meta-task, P is the number of corpora for each speaker in the support set, To support the embedding of the p-th corpus of speaker k, d q,k is the cosine similarity between each corpus embedding and the average embedding in the query set, is the embedding of the qth corpus in the query set, L MO is the multi-class cross entropy, C=[C1,C2,…,C k-1 , C k ] is the average embedding set in the support set, Q is the number of corpora for each speaker in the query set, and K is the single-click form of speaker k selected in the meta-task.
[0051] Specifically, the global distribution optimization is achieved by reducing the global distribution loss, the global distribution loss L DO The calculation formula is as follows:
[0052]
[0053] where N s is the number of speakers, is the linear layer for classification, is the embedding of the i-th corpus in the meta-task, and Y is the one-hot encoding form of speaker y.
[0054] Specifically, the cross-channel voice verification method includes:
[0055] Multi-dimensional log mel-filterbank features are extracted from the input speech to be authenticated in other unknown channels, and M consecutive frames are randomly intercepted as input features. The input features (extracted voiceprint features) are fed into a meta-speech embedding network based on a deep residual network structure combined with global average pooling to generate an N-dimensional speech embedding. The speaker identity is determined by calculating the cosine similarity between the input speech embedding and the registered speech embedding.
[0056] In a specific embodiment of the present invention, for speech input from an unknown channel, a trained meta-speech embedding network is used to extract voiceprint features and generate a robust speech embedding for cross-channel speech verification. 40-dimensional logmel-filterbank features are extracted from the input speech, and 100 consecutive frames are randomly intercepted as input features. The input features are fed into a meta-speech embedding network based on a deep residual network structure combined with global average pooling to generate a 256-dimensional speech embedding. The speaker identity is determined by calculating the cosine similarity between the input speech embedding and the registered speech embedding.
[0057] like Figure 2 As shown, specifically, the calculation formula for determining the speaker's identity is as follows:
[0058]
[0059] Among them, d test is the cosine similarity between the speech embedding and the registered speech embedding, and τ is the threshold; if the cosine similarity between the speech embedding and the registered speech embedding is greater than the threshold, it is determined that they are the same person; if the cosine similarity between the speech embedding and the registered speech embedding is less than the threshold, it is determined that they are not the same person.
[0060] Specifically, a meta-learning-based cross-channel invariant voiceprint feature extraction system includes:
[0061] Data acquisition module: used to acquire multi-channel voice data;
[0062] Training module: used for constructing a meta-speech embedding network and inputting the multi-channel speech data to train the meta-speech embedding network;
[0063] Optimization module: This module optimizes the trained meta-speech embedding network by simulating cross-channel scenarios to generate training tasks. It then adjusts the meta-speech embedding network using a meta-optimization strategy and global distribution optimization to generate a channel-invariant speech embedding space.
[0064] Verification module: It is used to extract voiceprint features from unknown channel voice data using the trained meta-speech embedding network, generate robust speech embeddings, and perform cross-channel voice verification.
[0065] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.
[0066] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A cross-channel invariant voiceprint feature extraction method based on meta-learning, characterized by: include: Step 1: Obtain multi-channel voice data; Step 2: constructing a meta-speech embedding network, and inputting the multi-channel speech data to train the meta-speech embedding network; Step 3: Optimize the trained meta-speech embedding network by simulating cross-channel scenarios to generate training tasks. Combined with meta-optimization strategies and global distribution optimization, the meta-speech embedding network is adjusted to generate a channel-invariant speech embedding space. Step 4: For unknown channel speech data, use the trained meta-speech embedding network to extract voiceprint features, generate robust speech voiceprint embedding, and perform cross-channel speech verification.
2. The meta-learning-based cross-channel invariant voiceprint feature extraction method according to claim 1, characterized in that: Optimizing the trained meta-speech embedding network includes: Step 31: sampling the multi-channel speech data to construct a simulated cross-channel scenario of the meta-task; Step 32: Optimize the simulated cross-channel scenario using a meta-optimization strategy to generate a channel-invariant embedding space, and stabilize the embedding space distribution through global distribution optimization; Step 33: Optimize the meta-speech embedding network by simulating cross-channel scenarios, meta-optimization strategies, and global distribution optimization to stabilize the embedding distribution.
3. The cross-channel invariant voiceprint feature extraction method based on meta-learning according to claim 2 is characterized in that: The meta-task includes a source dataset of N channels Randomly select P samples to construct the support set and Q samples to construct the query set; a pair of support set and query set constitutes a meta-task; Through meta-task sampling, the source dataset is converted into a set of meta-tasks to simulate cross-channel scenarios in is the source data from different channels, For the different meta-tasks constituted.
4. The meta-learning-based cross-channel invariant voiceprint feature extraction method according to claim 2, characterized in that: The meta-optimization strategy optimization is achieved by reducing the meta-optimization loss, the meta-optimization loss L MO The calculation formula is as follows: where f θ is the voiceprint extraction model, C k is the average embedding of the selected speakers k=1,…,B in the support set, B is the number of speakers selected in the meta-task, P is the number of corpora for each speaker in the support set, To support the embedding of the p-th corpus of speaker k, d q,k is the cosine similarity between each corpus embedding and the average embedding in the query set, is the embedding of the qth corpus in the query set, L MO is the multi-class cross entropy, C=[C1,C2,…,C k-1 , C k ] is the average embedding set in the support set, Q is the number of corpora for each speaker in the query set, and K is the single-click form of speaker k selected in the meta-task.
5. The meta-learning-based cross-channel invariant voiceprint feature extraction method according to claim 2, characterized in that: The global distribution optimization is achieved by reducing the global distribution loss, the global distribution loss L DO The calculation formula is as follows: where N s is the number of speakers, is the linear layer for classification, is the embedding of the i-th corpus in the meta-task, and Y is the one-hot encoding form of speaker y.
6. The meta-learning-based cross-channel invariant voiceprint feature extraction method according to claim 1, characterized in that: The cross-channel voice verification method includes: During the testing phase, multidimensional log mel-filterbank features are extracted from the input speech to be authenticated from other unknown channels, and M consecutive frames are randomly intercepted as input features. The input features are fed into a meta-speech embedding network based on a deep residual network structure combined with global average pooling to generate an N-dimensional speech embedding. The speaker identity is determined by calculating the cosine similarity between the input speech embedding and the registered speech embedding.
7. The meta-learning-based cross-channel invariant voiceprint feature extraction method according to claim 6, characterized in that: The calculation formula for determining the speaker's identity is as follows: Among them, d test is the cosine similarity between the speech embedding and the registered speech embedding, and τ is the threshold; if the cosine similarity between the speech embedding and the registered speech embedding is greater than the threshold, it is determined that they are the same person; if the cosine similarity between the speech embedding and the registered speech embedding is less than the threshold, it is determined that they are not the same person.
8. A cross-channel invariant voiceprint feature extraction system based on meta-learning, characterized by: include: Data acquisition module: used to acquire multi-channel voice data; Training module: used for constructing a meta-speech embedding network and inputting the multi-channel speech data to train the meta-speech embedding network; Optimization module: This module optimizes the trained meta-speech embedding network by simulating cross-channel scenarios to generate training tasks. It then adjusts the meta-speech embedding network using a meta-optimization strategy and global distribution optimization to generate a channel-invariant speech embedding space. Verification module: It is used to extract voiceprint features from unknown channel voice data using the trained meta-speech embedding network, generate robust speech embeddings, and perform cross-channel voice verification.