Voiceprint recognition method based on double feature branch structure
By employing a dual-feature branch structure for voiceprint recognition, combining Mel cepstral and wavelet transform features, and utilizing self-attention and convolutional TDNN networks for feature modeling and fusion, the problem of poor voiceprint feature extraction under low-quality speech signals is solved, thereby improving the robustness and accuracy of voiceprint recognition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HARBIN INST OF TECH AT WEIHAI
- Filing Date
- 2025-04-03
- Publication Date
- 2026-05-12
AI Technical Summary
Existing voiceprint recognition methods are ineffective in feature extraction under low-quality speech signals, and voiceprint feature information is easily lost. They also neglect multi-scale information of speech signals and fail to combine global and local features adequately.
A voiceprint recognition method based on a dual-feature branch structure is adopted, which extracts Mel cepstral features and wavelet transform features respectively. Multi-scale time-frequency feature modeling and time-delay convolution processing are performed through a self-attention module and a convolutional TDNN network. Feature fusion is performed by combining an SE module and an average statistical pooling layer. Finally, AAM-Softmax loss is used for speaker classification.
It better preserves key acoustic features in low-quality or noisy environments, improves the discriminative power and robustness of deep models for different speakers' speech, reduces the risk of losing voiceprint information, and achieves accurate voiceprint feature recognition.
Smart Images
Figure CN120412591B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of deep learning voiceprint recognition technology, and in particular to a voiceprint recognition method based on a dual-feature branch structure. Background Technology
[0002] Voiceprint recognition has undergone rapid development, evolving from statistical methods based on Gaussian mixture models and i-vectors to end-to-end solutions such as x-vectors that combine deep neural networks. Current research mainly focuses on better feature extraction and modeling strategies, such as combining attention mechanisms and fusing multi-scale features, to achieve more robust voice identity recognition in complex environments.
[0003] Current methods often extract Mel-frequency cepstrum or other single features using a single path, neglecting the multi-scale information of speech signals. Furthermore, deep models often fall short in combining global and local feature information. In summary, existing speaker recognition methods suffer from poor feature extraction performance on low-quality speech signals and are prone to losing speaker feature information. Summary of the Invention
[0004] The purpose of this invention is to provide a voiceprint recognition method based on a dual-feature branch structure, which solves the problems of poor feature extraction of low-quality speech signals and easy loss of voiceprint feature information in existing voiceprint recognition methods.
[0005] To achieve the above objectives, the present invention provides a voiceprint recognition method based on a dual-feature branch structure, the voiceprint recognition method based on a dual-feature branch structure comprising the following steps:
[0006] The original speech signal is preprocessed, and Mel cepstral features and wavelet transform features are extracted to form two feature branches.
[0007] Mel cepstral features are input into the self-attention module to perform sequence modeling of multi-scale time-frequency features;
[0008] Input wavelet transform features into a convolutional TDNN network and perform time-delayed convolution on the segmented or framed wavelet coefficients.
[0009] The outputs of the two branches are fused to obtain the fused voiceprint feature representation;
[0010] The fused voiceprint feature representation is input into the SE module and connected to the average statistical pooling layer and a linear layer. Finally, AAM-Softmax loss is used for speaker classification or similarity measurement.
[0011] In the step of "preprocessing the original speech signal and extracting Mel cepstral features and wavelet transform features to form two feature branches", the extraction of Mel cepstral features satisfies the following formula:
[0012]
[0013] Where X[k] is the Fourier transform of the signal, B m [k] is the mask for the cepstral frequency at the Mel frequency, which can be calculated using the following formula:
[0014]
[0015] Where K is the number of frequency domain sampling points and M is the number of Mel filters.
[0016] In the step of "preprocessing the original speech signal and extracting Mel-frequency cepstral features and wavelet transform features to form two feature branches", the extraction of wavelet transform features satisfies the following formula:
[0017]
[0018] Where x(t) is the input signal, ψ is the mother wavelet function, a is the scaling factor, and b is the translation factor.
[0019] The specific content of the step "inputting Mel cepstral features into the self-attention module to perform sequence modeling of multi-scale time-frequency features" includes:
[0020] The linear mapping yields Q, K, and V. The Mel-frequency cepstral feature matrix X is then multiplied by the learnable parameter matrix W. h (Q) W h (K) and W h (V) :
[0021]
[0022] Q h With K h Perform the dot product and divide by Perform normalization, then use softmax to obtain the attention weight matrix, and finally combine it with V. h Multiplication:
[0023]
[0024] The outputs of all attention heads (h = 1, ..., H) are concatenated and multiplied by the learnable output mapping W. O :
[0025] MultiHead(X)=[head1|head2|…|head H ]·W O .
[0026] In the step of “inputting wavelet transform features into a convolutional TDNN network and performing time-delayed convolution processing on the segmented or framed wavelet coefficients”, the convolutional TDNN network performs time-delayed convolution processing on the wavelet transform features after segmentation or frame-by-frame, and captures local features through layer normalization and convolutional spatial gating units before projecting them back to the original dimension.
[0027] In the step of "feature fusion of the outputs of the two branches", a covariance transformation or channel normalization mechanism can be optionally introduced to align the feature distribution, and the fused feature map is restored through linear mapping, as follows:
[0028] Y = ω1·Y mfcc +ω2·Y wav .
[0029] In the step "inputting the fused voiceprint feature representation into the SE module and the average statistical pooling layer and connecting it to a linear layer", the SE module performs channel weighting on the fused feature map, the average statistical pooling layer aggregates global temporal features, and the linear layer outputs the voiceprint embedding. The operation process of the average statistical pooling layer is expressed as follows:
[0030] The computational channel-dependent soft attention mechanism, with self-attention weights α t,c The importance of each frame for a given channel is represented by the activation output h of each frame:
[0031]
[0032] For a given speech, the weighted mean Each channel component for:
[0033]
[0034] For a given speech, the weighted mean Each channel component for:
[0035]
[0036] In the step "finally using AAM-Softmax loss for speaker classification or similarity measurement",...
[0037] The AAM-Softmax loss function can be calculated using the following formula:
[0038]
[0039] For all categories j≠y i , z j =s·cos(θ) j For the correct category, add an additional angle m on top of the original cosine θ.
[0040] The voiceprint recognition method based on a dual-feature branch structure further includes a voiceprint database update step. This step specifically refers to updating the corresponding matching voiceprint data in the database using the model output to ensure its timeliness, as expressed by the following formula:
[0041] speaker i =α·speaker i +(1-α)·Y
[0042] Where i represents the recognition result and α represents the hyperparameter.
[0043] This invention presents a voiceprint recognition method based on a dual-feature branch structure. This method utilizes the dual-feature branch structure to independently extract and model Mel-frequency cepstral hypothesis and wavelet multi-scale time-frequency information at the front end, enabling better preservation of key acoustic features in low-quality or noisy environments. Combining the advantages of self-attention and convolutional TDNN, it simultaneously performs refined fusion of local and global features. Through parallel or cascaded operations on multi-level feature flows, it effectively reduces the risk of voiceprint information loss in single feature channels or at fixed resolutions, helping to improve the discriminative power and robustness of deep models for different speakers' speech. Furthermore, by combining the SE module and average statistical pooling layer, global temporal information can be aggregated, and AAM-Softmax loss is used for more stringent similarity constraints, achieving accurate recognition of different voiceprint features. Attached Figure Description
[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0045] Figure 1 This is a flowchart of the steps of the voiceprint recognition method based on a dual-feature branch structure provided by the present invention.
[0046] Figure 2 This is a schematic diagram of the dual-feature branch model structure provided by the present invention.
[0047] Figure 3This is a schematic diagram of the voiceprint feature SE module provided by the present invention.
[0048] Figure 4 This is a schematic diagram of the voiceprint database update operation provided by the present invention. Detailed Implementation
[0049] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.
[0050] Please see Figures 1 to 4 This invention provides a voiceprint recognition method based on a dual-feature branch structure, which specifically includes the following steps:
[0051] Step S1 involves preprocessing the original signal, including framing, windowing, and noise reduction.
[0052] Step S2 involves extracting two types of features from the preprocessed speech signal to form a multi-level cascaded input; one branch is the Mel cepstral feature, and the extraction process of the Mel cepstral feature is expressed by the following formula:
[0053]
[0054] Where X[k] is the Fourier transform of the signal, B m [k] is the mask for the Mel frequency cepstral spectrum, which can be calculated using the following formula, where K is the number of frequency domain sampling points and M is the number of Mel filters. This process yields the Mel cepstral features for each frame.
[0055]
[0056] Another branch is wavelet decomposition features, and the extraction process of the wavelet transform features is expressed by the following formula:
[0057]
[0058] Where x(t) is the input signal, ψ is the mother wavelet function, a is the scaling factor, and b is the translation factor. By transforming under different values of a and b, a multi-scale representation of the signal in the time-frequency domain can be obtained.
[0059] Optionally, concatenated coding networks capable of supporting increasing channel numbers can be used to model different features using convolution or attention, and pooling position indices can be saved during downsampling to preserve key acoustic structural information in subsequent operations.
[0060] Specifically, the provided voiceprint recognition method based on a dual-feature branch structure may also include multi-scale feature extraction and downsampling. The input layer of each stage of the encoder is filled by boundary reflection and features are activated by sliding window convolution with a stride of 1. The output layer combines group separation convolution to achieve incremental feature channels.
[0061] Step S3: After processing each scale of the two feature streams through the corresponding network module, the features are fused to generate a joint voiceprint feature representation; for example... Figure 2 As shown, one branch is a self-attention network that models the sequence context of Mel cepstral features. The specific calculation process of the multi-head attention module can be divided into the following steps (taking the h-th attention head as an example):
[0062] The linear mapping yields Q, K, and V. The Mel-frequency cepstral feature matrix X is then multiplied by the learnable parameter matrix W. h (Q) W h (K) and W h (V) :
[0063]
[0064] Q h With K h Perform the dot product and divide by Perform normalization, then use softmax to obtain the attention weight matrix, and finally combine it with V. h Multiplication:
[0065]
[0066] The outputs of all attention heads (h = 1, ..., H) are concatenated and multiplied by the learnable output mapping W. O :
[0067] MultiHead(X)=[head1|head2|…|head H ]·W O .
[0068] By using the parallel and concatenated multi-head attention methods described above, we can comprehensively model the dependencies of Mel cepstral features at different angles or subspaces.
[0069] Another branch uses a convolutional TDNN network to perform time-delayed convolution on wavelet features. After layer normalization, the input feature channels are projected to a higher dimension and activated. Subsequently, a convolutional spatial gating unit is used to capture local information. This module first divides the features into two segments, then applies layer normalization and depthwise convolution to one segment, and finally performs element-wise multiplication between the two branches. Finally, the output is projected back to the original dimension.
[0070] Finally, the output Y of the multi-head self-attention module is... mfcc The output Y of the convolutional TDNN network wav Feature fusion is performed, and the fusion process can be formally expressed as follows:
[0071] Y = ω1·Y mfcc +ω2·Y wav .
[0072] Optionally, a fusion module based on covariance or channel normalization can be introduced to first differentiate and align the feature distributions of different branches, and then restore them to a fused feature map.
[0073] Step S4: The fused feature map is further input into the SE module and the average statistical pooling layer, and then connected to a linear layer; as shown... Figure 3 As shown, the fused feature map is first fed into the SE module to highlight key channels and suppress redundant information. Then, the global features are aggregated through the average statistical pooling layer, and finally, the linear layer is input for embedding mapping or classification output.
[0074] The computation process of the average statistical pooling layer can be represented as follows:
[0075] The computational channel-dependent soft attention mechanism, with self-attention weights α t,c The importance of each frame in a given channel is represented by the activation output h of each frame. t The calculation shows that:
[0076]
[0077] For a given speech, the weighted mean Each channel component The estimate is:
[0078]
[0079] For a given speech, the weighted mean Each channel component The estimate is:
[0080]
[0081] Specifically, the provided voiceprint recognition method based on a dual-feature branch structure can also include multi-level use of the covariance matrix: during the fusion process, the difference features between the wavelet branch and the Mel cepstral branch are first adjusted by normalization, and then the features are mixed through the covariance matrix or attention matrix to remove redundant information and highlight complementary features.
[0082] Step S5: Calculate multi-level recognition or classification loss for the fused feature map to ensure that the fused feature map retains the recognizability of the speaker's voiceprint; AAM-Softmax or other loss functions based on metric learning can be used to discriminate the embedded features at multiple scales, enhance the model's ability to distinguish features of different speakers, and reduce the impact of noise and distortion.
[0083] The AAM-Softmax loss function can be calculated using the following formula:
[0084]
[0085] For all categories j≠y i , z j =s·cos(θ) j For the correct category, add an additional angle m on top of the original cosine θ.
[0086] Step S6: Match the classification results with the voiceprint data in the voiceprint database. If the match is successful, such as... Figure 4 The following formula is used to update the corresponding data to ensure the timeliness of the voiceprint recognition system:
[0087] speaker i =α·speaker i +(1-α)·Y
[0088] Where i represents the recognition result, and α is a hyperparameter that can be set manually or calculated using methods such as cosine similarity. Generally, this value should exceed 0.9 to ensure the accuracy and security of the voiceprint database.
[0089] In one embodiment, a voiceprint recognition method based on a dual-feature branch structure may further include the calculation process of a covariance transform network: first, the Mel cepstrum and wavelet features are normalized so that their distributions meet similar statistical requirements; then, the covariance matrix is introduced to achieve feature mixing and restoration; and by re-adding with the original mean information, a more robust voiceprint feature representation after fusion is obtained.
[0090] In summary, the voiceprint recognition method based on a dual-feature branch structure provided in this technical solution utilizes the dual-feature branch structure to independently extract and model Mel-frequency cepstral hypothesis and wavelet multi-scale time-frequency information at the front end, which can better preserve key acoustic features in low-quality or noisy environments. Combining the advantages of self-attention and convolutional TDNN, it simultaneously performs refined fusion of local and global features. Through parallel or cascaded operations on multi-level feature streams, it effectively reduces the risk of loss of voiceprint information under single feature channels or fixed resolution, which helps to improve the discriminability and robustness of deep models for different speakers' speech. Furthermore, by combining the SE module and average statistical pooling layer, global temporal information can be aggregated, and AAM-Softmax loss is used for more stringent similarity constraints, achieving accurate recognition of different voiceprint features.
[0091] The above description discloses only one preferred embodiment of the present invention, and should not be construed as limiting the scope of the present invention. Those skilled in the art will understand that all or part of the processes of the above embodiments can be implemented, and equivalent changes made in accordance with the claims of the present invention are still within the scope of the invention.
Claims
1. A voiceprint recognition method based on a dual-feature branch structure, characterized in that, Includes the following steps: The original speech signal is preprocessed, and Mel cepstral features and wavelet transform features are extracted to form two feature branches. Mel cepstral features are input into the self-attention module to perform sequence modeling of multi-scale time-frequency features; Input wavelet transform features into a convolutional TDNN network and perform time-delayed convolution on the segmented or framed wavelet coefficients. The results of sequence modeling of Mel cepstral features by the self-attention module and the results of time-delayed convolution of wavelet transform features by the convolutional TDNN network are fused to obtain the fused voiceprint feature representation. The fused voiceprint feature representation is input into the SE module and connected to the average statistical pooling layer and a linear layer. Finally, AAM-Softmax loss is used for speaker classification or similarity measurement. It also includes: extracting and downsampling multi-scale features; filling the input layer of each stage of the encoder with boundary reflection and using sliding window convolution with stride of 1 for feature activation; and combining group separation convolution to achieve incremental feature channels in the output layer.
2. The voiceprint recognition method based on a dual-feature branch structure as described in claim 1, characterized in that, In the step "preprocessing the original speech signal and extracting Mel-frequency cepstral features and wavelet transform features to form two feature branches", the extraction of the Mel-frequency cepstral features satisfies the following formula: ； ； in, For the Fourier transform of the signal, The mask for the cepstral frequency of the Mel frequency can be calculated using the following formula: ； in, This represents the number of sampling points in the frequency domain. This represents the number of Mel filters.
3. The voiceprint recognition method based on a dual-feature branch structure as described in claim 2, characterized in that, In the step "preprocessing the original speech signal and extracting Mel-frequency cepstral features and wavelet transform features to form two feature branches", the extraction of wavelet transform features satisfies the following formula: ； in, For input signal, For the mother wavelet function, It is a scale factor. It is the translation factor.
4. The voiceprint recognition method based on a dual-feature branch structure as described in claim 3, characterized in that, The specific content of the step "inputting Mel cepstral features into the self-attention module for multi-scale time-frequency feature sequence modeling" includes: Linear mapping yields Mel cepstral characteristic matrix Multiply by the learnable parameter matrix respectively and : ; Will and Perform the dot product and divide by Perform normalization, then use softmax to obtain the attention weight matrix, and finally combine it with... Multiplication: ; All attention heads The outputs are concatenated and multiplied by a learnable output mapping. : 。 5. The voiceprint recognition method based on a dual-feature branch structure as described in claim 4, characterized in that, In the step "inputting wavelet transform features into a convolutional TDNN network and performing time-delayed convolution processing on the segmented or framed wavelet coefficients", the convolutional TDNN network performs time-delayed convolution processing on the wavelet transform features after segmentation or frame-by-frame, and captures local features through layer normalization and convolutional spatial gating units before projecting them back to the original dimension.
6. The voiceprint recognition method based on a dual-feature branch structure as described in claim 5, characterized in that, In the step "Fusing features from the two branches mentioned above", a covariance transformation or channel normalization mechanism is introduced to align the feature distribution, and a fused feature map is restored through linear mapping, as follows: 。 7. The voiceprint recognition method based on a dual-feature branch structure as described in claim 6, characterized in that, In the step "inputting the fused voiceprint feature representation into the SE module and the average statistical pooling layer and connecting it to a linear layer", the SE module performs channel weighting on the fused feature map, the average statistical pooling layer aggregates the global temporal features, and the linear layer outputs the voiceprint embedding. The operation process of the average statistical pooling layer is expressed as follows: Computational channel-dependent soft attention mechanisms, self-attention weights The importance of each frame in a given channel is indicated by the activation output of each frame. The calculation shows that: ； ； For a given speech, the weighted mean Each channel component for: ； For a given speech, the weighted mean Each channel component for: 。 8. The voiceprint recognition method based on a dual-feature branch structure as described in claim 7, characterized in that, In the step "Finally, AAM-Softmax loss is used for speaker classification or similarity measurement", The AAM-Softmax loss function can be calculated using the following formula: ； Among them, for all categories , For the correct category in the original cosine Add an extra angle above , .
9. The voiceprint recognition method based on a dual-feature branch structure as described in claim 8, characterized in that, The voiceprint recognition method based on a dual-feature branch structure further includes a voiceprint database update step. Specifically, this step involves updating the corresponding matched voiceprint data in the database using the model output to ensure its timeliness, as expressed by the following formula: ； in, For the recognition results, This is a hyperparameter.