Adaptive voiceprint recognition system based on Mama model

Abstract

The invention discloses a self-adaptive voiceprint recognition system based on a Mama model. The self-adaptive voiceprint recognition system comprises a voice acquisition module, a voice enhancement module, a voice feature extraction module, a spectrogram enhancement module, a model training module and a model decision module, according to the invention, through well-designed network architecture and algorithm optimization, high identification accuracy is ensured, and good robustness is shown; the system can keep a low equal error rate under various noise conditions and in different speaking styles, which indicates high robustness and accuracy of the system in a voiceprint recognition task; a bidirectional state space model of the Mama module can effectively capture local details and global context of voiceprints, and the recognition capability of the system on complex voice features is enhanced; the TDNN module strengthens the sensitivity of the system to time sequence features, the DenseNet module improves the transmission and fusion of the features through the dense connection of the DenseNet module, and the combination of the TDNN module and the DenseNet module effectively improves the feature extraction fineness of the Mamba module and improves the robustness of the model.

Description

Technical Field

[0001] The present invention belongs to the technical field of voiceprint recognition, and in particular relates to an adaptive voiceprint recognition system based on a Mamba model. Background Art

[0002] With the rapid development of information technology, the demand for personal identity authentication technology in terms of security and convenience is growing. Voiceprint recognition, as a biometric identification method, has attracted widespread attention due to its non-invasive, easy to collect and user-friendly characteristics. Voiceprint recognition systems automatically verify the identity of speakers by analyzing the acoustic characteristics of individual voices, and are widely used in security monitoring, mobile device unlocking, financial services and other fields. However, existing voiceprint recognition systems still have problems with insufficient recognition accuracy and robustness when facing practical application challenges such as environmental noise, differences in recording quality, and changes in speaking styles. In addition, with the development of deep learning technology, how to effectively combine traditional voiceprint recognition algorithms with deep learning models to improve the system's generalization and anti-interference capabilities has become a hot topic in current research.

[0003] A search of the prior art literature revealed that in the field of speaker recognition, deep neural networks (DNNs) have become the main driving force for building efficient recognition systems. Recent research progress shows that there is a clear trend to move from traditional time delay neural network (TDNN) architectures to ResNet-based models. TDNN models effectively aggregate temporal information through statistical pooling layers, while ResNet models process speech signals using two-dimensional convolutional layers through residual blocks (ResBlocks). Although ResNet models are able to preserve temporal information when processing three-dimensional tensors, their reduction in spatial dimensions, especially in the time axis, may lead to the loss of temporal information. To address this issue, Choi et al. proposed a novel Temporal-Bottleneck ResNet (TB-ResNet) model that captures and preserves frame-level context through a temporal bottleneck configuration within its building blocks, thereby enhancing the performance of statistical pooling layers. The TB-ResNet model outperformed the traditional ResNet model on the VoxCeleb1 test set, significantly reducing the equal error rate (EER) and minimum detection cost function (minDCF), proving its effectiveness in automatic speaker verification tasks. In addition, by introducing attention mechanisms and data augmentation techniques, the TB-ResNet model further improved the quality of speaker embedding and enhanced the generalization ability of the system. This research achievement not only promotes the development of ASV technology, but also provides new directions for future research. Summary of the invention

[0004] In order to make up for the shortcomings of the prior art, the purpose of the present invention is to provide an adaptive voiceprint recognition system based on the Mamba model, which integrates the advantages of local and global feature extraction through innovative network architecture design, and significantly improves the accuracy and robustness of voiceprint recognition; the modular design not only enhances the flexibility of upgrading and adjustment, but also effectively improves the expressive ability of voiceprint features and the generalization performance of the model through a series of advanced data processing and network training technologies.

[0005] The adaptive voiceprint recognition system based on the Mamba model comprises a voice collection module, a voice enhancement module, a voice feature extraction module, a spectrogram enhancement module, a model training module and a model decision module.

[0006] The voice collection module is used to collect voice data;

[0007] The speech enhancement module is used to enhance the collected speech data, including adding reverberation, adding noise, adding speed disturbance and frequency domain masking;

[0008] The speech feature extraction module is used to extract the features of the speech data after enhancement processing, including acoustic feature extraction, feature regularization processing, FBank feature extraction and network input of feature vector;

[0009] The speech spectrogram enhancement module is used to perform spectrogram enhancement processing on the FBank feature data of the speech, including frequency domain mask processing and time domain mask processing; the FBank feature is an 80-dimensional feature vector generated by performing frequency domain analysis on the speech signal. In the enhancement processing, the module will apply frequency domain masking and time domain masking to the FBank feature, thereby improving the expressiveness of the spectrogram;

[0010] The model training module is used to establish a voiceprint recognition model and train the model, and adopts a Mamba-based adaptive voiceprint recognition method to establish a voiceprint recognition model and a training model;

[0011] The model decision module is used to evaluate and provide feedback on the effect of model training in real time, including model evaluation and model iteration.

[0012] Furthermore, for the speech enhancement module:

[0013] The adding of reverberation is used to enhance the authenticity and diversity of voice data by simulating sound reflection and delay superposition in different environments. The adding of reverberation can simulate the sound effects of different spaces from small rooms to large auditoriums;

[0014] The added noise is used to introduce different types of background noise into the speech data to improve the robustness of the model to noise in the actual environment;

[0015] The added speed disturbance is used to fine-tune the speech playback speed by performing time expansion and contraction processing on the speech signal, thereby generating a slightly slower or faster speech signal;

[0016] The frequency domain masking is used to add noise or modulated signals at specific frequencies to simulate interference in a real environment and improve the robustness and adaptability of the model to complex sound environments.

[0017] Furthermore, for the speech feature extraction module:

[0018] The acoustic feature extraction is used to extract acoustic features from the collected speech data;

[0019] The feature regularization process is used to regularize the extracted acoustic features, which can eliminate noise and irregularity in the data;

[0020] The FBank feature extraction is used to further perform FBank feature extraction on the regularized speech data to generate an 80-dimensional FBank feature vector;

[0021] The network input of the feature vector is used to send the obtained 80-dimensional FBank feature vector to the spectrogram enhancement module, and then it will be sent to the Mamba-based voiceprint recognition network.

[0022] Further, for the spectrogram enhancement module:

[0023] The specific operation of the frequency domain masking process is: within the frequency domain range of the FBank feature, a frequency band with a width of 0 to 8 is randomly selected for masking; the frequency domain masking process is to set the spectrum value in the selected frequency band to 0 to simulate the spectrum information loss or interference that may be encountered in actual applications;

[0024] The specific operation of the time domain masking process is: within the time domain range of the FBank feature, randomly select a time period with a width of 0 to 10 for masking; the time domain masking process is to set the spectrum value in the selected time period to 0 to simulate the loss or interference of the speech signal in different time periods.

[0025] Furthermore, for the model training module, the specific steps of the adaptive voiceprint recognition method based on Mamba are as follows:

[0026] S1. Use the Mamba-based network to learn the feature data and form a voiceprint recognition model;

[0027] S2. In the voiceprint recognition model, the speech recognition process begins by dividing the input spectrogram into smaller segments, which are then mapped into a series of tokens. The encoder in the ViM model receives these tokens and performs two independent linear transformations on each token to extract deep features. The ViM model uses a bidirectional processing mechanism to simulate the function of a bidirectional neural network layer, that is, it processes token sequences in both the forward and reverse directions to capture information in the speech signal.

[0028] S3. In each processing direction, a one-dimensional convolution operation is first applied to extract local features, and then a Sigmoid linear unit activation function is used to enhance the nonlinear expression capability; and additional learnable classification tags are added to the token sequence;

[0029] S4. After the convolution and activation operations, an additional linear transformation is performed and a softplus function is applied to ensure that all output values ​​are positive, in preparation for the labeling of the state-space model sequence modeling function; then, a gating mechanism is introduced to regulate the information flow, which controls the contribution of each direction processing to the final result by multiplying the output of the state-space model with the elements of the forward and reverse sequences after SiLU activation;

[0030] S5. Use residual connection to add the original input sequence to the gated output and output a new labeled sequence;

[0031] S6. While using Mamba for voiceprint recognition, the MFCCs features are extracted from the spectrogram through the fast Fourier transform (FFT), and the MFCCs are sequentially passed through the time-delay neural network module (TDNN) and the DenseNet module to extract the local features of the target; S6. While using Mamba for voiceprint recognition, the Mel-frequency cepstral coefficient features are extracted from the spectrogram through the fast Fourier transform, and the extracted Mel-frequency cepstral coefficient features are sequentially passed through the time-delay neural network module and the DenseNet module to extract the local features of the target;

[0032] S7. The target local features extracted by the above-mentioned DenseNet module and TDNN module will become the input information of the attention strategy pool; the attention strategy pool can extract key information from the voice data, assign different weights to the voice features of different frames, and obtain a more discriminative voiceprint vector;

[0033] S8. After the attention strategy pool, the local features will be integrated in the fully connected module to form global features.

[0034] Further, in step S6, the time-delay neural network module is composed of three standard time-delay neural network layers and N time-delay neural network layers with special structures, where N is an integer greater than 1;

[0035] The DenseNet module consists of two convolutional layers and two residual layers. The first convolutional layer performs convolution processing on the received speech acoustic features to extract local features of the speech data. The jump connection between the two residual layers enables the network to learn more effective feature representations and alleviate the gradient vanishing problem in deep network training. The second convolutional layer performs convolution processing on the feature map output by the second residual layer, and merges the frequency domain features of the feature map with the original frequency domain and channel features to assist the Mamba module in voiceprint recognition.

[0036] Furthermore, in step S7, the attention strategy pool module uses the target local features as input information and outputs a speech weighted voiceprint vector; for a speech X=x containing T frames 1 ,x 2 ,…,x T , a speech vector H=h with T frames can be calculated through the network 1 ,h 2 ,…,h T , where the scale of H is d h ×T, the calculation method of speech feature weight A is as follows:

[0037] A=σ(g(HW 1 )W 2 ),

[0038] Where W 1 is a h ×d r The matrix, W 2 is a r ×d h , g(·) represents the activation function, σ represents the Sigmoid function, A=a 1 ,a 2 ,…,a T , the scale of A is d h ×T;

[0039] The mean μ of the speech vector calculated based on the attention strategy pool is:

[0040]

[0041] Among them, a t is the weight of the speech vector of the tth frame, h t is the speech vector of the tth frame, T represents the total number of speech frames;

[0042] The standard deviation σ of the speech vector calculated based on the attention strategy pool is:

[0043]

[0044] Among them, a t is the weight of the speech vector of the tth frame, h t is the speech vector of the tth frame, μ is the mean of the speech vector, and T represents the total number of speech frames;

[0045] The voiceprint vector V obtained by the attention strategy pool is expressed as:

[0046]

[0047] Where μ is the mean of the speech vector and σ is the standard deviation of the speech vector.

[0048] Furthermore, in step S8, the fully connected module includes two fully connected layers: the first fully connected layer is responsible for receiving the voiceprint vector from the attention strategy pool module and calculating the voiceprint embedding vector; the second fully connected layer is responsible for receiving the voiceprint embedding vector output by the first fully connected layer, and on this basis predicting and outputting the classification information corresponding to the speech information.

[0049] Furthermore, for the model decision module:

[0050] The evaluation model is used to test and determine the effect of using the voiceprint recognition model trained in the current stage to evaluate the training of the real-time feedback model, including but not limited to the false acceptance rate, false rejection rate, equal error rate, and minimum detection cost function;

[0051] The model iteration is used to update the training model through multiple iterations, and the adaptive voiceprint recognition method based on DenseNet and TDNN will obtain a mature voiceprint recognition network model.

[0052] Compared with the prior art, the present invention has the following advantages:

[0053] 1) This system achieves a balance between robustness and accuracy in the voiceprint recognition system by comprehensively utilizing advanced network architecture and algorithms, providing a new perspective and solution for the development of voiceprint recognition technology. Specifically, the present invention achieves good robustness while ensuring high recognition accuracy through a carefully designed network architecture and algorithm optimization. The system can maintain a low equal error rate (EER) under a variety of noise conditions and in different speaking styles, proving its high robustness and accuracy in voiceprint recognition tasks; the bidirectional state-space model of the Mamba module effectively captures the local details and global context of the voiceprint, enhancing the system's ability to recognize complex speech features. Figure 2 The TDNN module enhances the system's sensitivity to time series features, while the DenseNet module improves feature transmission and fusion through its dense connections. The combination of the two significantly improves the robustness of the model.

[0054] 2) The test results on the open source speaker recognition corpus VoxCeleb show that the present invention has achieved significant improvement in voiceprint recognition performance, verifying the effectiveness of the system design. The VoxCeleb dataset contains more than 1 million voice clips of more than 7,000 celebrities, which are extracted from YouTube videos using speech recognition and facial verification technology and are used to research and develop speaker recognition and verification systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 is a system module block diagram of the present invention;

[0056] Figure 2 It is a flowchart of the steps of the adaptive voiceprint recognition method based on Mamba of the present invention;

[0057] Figure 3 This is a network structure diagram of the voiceprint recognition model of the present invention. DETAILED DESCRIPTION

[0058] The following is a further description of an adaptive voiceprint recognition system based on the Mamba model of the present invention in conjunction with the accompanying drawings.

[0059] like Figure 1 As shown, an adaptive voiceprint recognition system based on the Mamba model includes a voice collection module, a voice enhancement module, a voice feature extraction module, a spectrogram enhancement module, a model training module and a model decision module. The details are as follows:

[0060] (1) Voice Collection Module

[0061] The voice collection module is used to collect voice data.

[0062] (2) Speech Enhancement Module

[0063] The speech enhancement module is used to enhance the collected speech data, which can strengthen the voiceprint data and improve the performance and adaptability of the voiceprint recognition model in practical applications, including adding reverberation, adding noise, adding speed disturbance and frequency domain masking.

[0064] Add reverberation: Enhance the authenticity and diversity of voice data by simulating sound reflection and delayed superposition in different environments.

[0065] Adding noise: Introducing different types of background noise into the speech data, such as music, environmental noise, noisy human voices, and TV noise, to improve the model's robustness to noise in the actual environment.

[0066] Add speed perturbation: By performing time stretching on the speech signal, the speech playback speed can be fine-tuned to produce slightly slower or faster speech signals, helping the model adapt to the speed changes of different speakers and thus enhancing the generalization ability of the model.

[0067] Frequency domain masking: Add noise or modulated signals at specific frequencies to simulate interference in real environments and improve the robustness and adaptability of the model to complex sound environments.

[0068] (3) Speech feature extraction module

[0069] The speech feature extraction module is used to extract the features of the enhanced speech data, including acoustic feature extraction, feature regularization processing, FBank feature extraction and network input of feature vectors.

[0070] Acoustic feature extraction: Extract acoustic features from the collected voice data. These features can represent the key attributes of the voice signal and provide the necessary information for voiceprint recognition.

[0071] Feature regularization processing: Regularization is performed on the extracted acoustic features to eliminate noise and irregularities in the data, thereby improving the stability and reliability of the features.

[0072] FBank feature extraction: The regularized speech data is further subjected to FBank (Filter Bank) feature extraction to generate an 80-dimensional feature vector. FBank features are widely used in speech processing because they can simulate the sensitivity of the human ear to sounds of different frequencies. They can effectively capture the key acoustic properties of speech signals, such as rhythm and timbre.

[0073] Network input of feature vector: The obtained 80-dimensional FBank feature vector is fed into the spectrogram enhancement module, and then fed into the Mamba-based voiceprint recognition network. These feature vectors are the key input for training the voiceprint recognition model, enabling the model to learn and distinguish the voiceprint features of different speakers.

[0074] (4) Spectrogram Enhancement Module

[0075] The speech spectrogram enhancement module is used to perform spectrogram enhancement processing on the extracted features, increase the diversity and complexity of the training data, thereby improving the generalization ability and robustness of the voiceprint recognition model, and helping the model to accurately identify and verify voiceprints when facing various noises and interferences in the actual environment, including frequency domain mask processing and time domain coding processing.

[0076] Frequency domain masking: In the frequency domain range of the FBank feature, frequency bands with a width of 0 to 8 are randomly selected for masking. Frequency domain masking means setting the spectrum value in the selected frequency band to 0, which can simulate the spectrum information loss or interference that may be encountered in actual applications.

[0077] Time domain masking: In the time domain range of the FBank feature, randomly select time periods with a width of 0 to 10 for masking. Similar to frequency domain masking, time domain masking also sets the spectrum value in the selected time period to 0 to simulate the loss or interference of the speech signal in different time periods.

[0078] (5) Model training module

[0079] The model training module is used to establish a voiceprint recognition model and train the model, and adopts the Mamba-based adaptive voiceprint recognition method to establish the voiceprint recognition model and the training model. Figure 2 As shown, the specific steps of the adaptive voiceprint recognition method based on Mamba include:

[0080] S1. Use the Mamba-based network to learn the feature data and form a voiceprint recognition model. The network structure is shown in Figure 3 .

[0081] S2. In the voiceprint recognition model, the speech recognition process begins by dividing the input spectrogram into smaller segments, which are then mapped into a series of tokens; the encoder in the ViM model receives these tokens and performs two independent linear transformations on each token to extract deep features. Unlike unidirectional processing, the voiceprint recognition model uses a bidirectional processing mechanism to simulate the function of a bidirectional neural network layer, that is, processing token sequences in both the forward (left to right) and reverse (right to left) directions at the same time, thereby capturing the information in the speech signal more comprehensively.

[0082] S3. In each processing direction, the voiceprint recognition model first applies a one-dimensional convolution operation to extract local features, and then uses the Sigmoid Linear Unit (SiLU) activation function to enhance the nonlinear expression capability. In order to further improve the classification performance of the model, the voiceprint recognition model adds additional learnable classification tags to the token sequence. This mechanism draws on the special tag usage in the BERT model.

[0083] S4. After the convolution and activation operations, the voiceprint recognition model performs an additional linear transformation and applies the softplus function to ensure that all output values ​​are positive, preparing for the labeling of the state-space model (SSM) sequence modeling function; then, a gating mechanism is introduced to regulate the information flow, which controls the contribution of each direction processing to the final result by multiplying the output of the SSM with the elements of the forward and reverse sequences activated by SiLU.

[0084] The voiceprint recognition model (ViM model) described in the S5.Mamba module adopts a residual connection design to add the original input sequence to the gated output, which not only helps to retain important information in the early layers of the network, but also helps to alleviate the problem of gradient disappearance; after this series of complex transformations, the voiceprint recognition model outputs a new tag sequence that captures the complex dependencies of the sequence in both directions, thereby improving the accuracy of speech-to-text conversion.

[0085] S6. While using Mamba for voiceprint recognition, the MFCCs features are extracted from the spectrogram through fast Fourier transform (FFT), and then the MFCCs are passed through the time delay neural network module (TDNN) and the DenseNet module in turn to extract the target local features.

[0086] Specifically, the time-delay neural network module (TDNN) consists of three standard time-delay neural network layers (TDNN layers) and N time-delay neural network layers of special structures (SK-TDNN layers), where N is an integer greater than 1. The specific implementation method is as follows: the first and second TDNN layers complete the dimension reduction processing of the features and further extract the temporal relationship of the speech features, and then the time-delay neural units of the SK-TDNN layer process the feature map of the second layer in different frames, and use the first time-delay neural unit to consider 1 frame of speech information and process the current feature map; then, use the second and third time-delay neural units to consider the adjacent 3 frames and 5 frames of speech information respectively, process the feature map output by a time-delay neural unit respectively, and sum the processing results; then, the summed features are processed by the multi-scale time-delay neural unit attention module, and the global average pooling operation is performed in the time domain dimension, and then passed through a fully connected layer. The output of the fully connected layer is divided into two branches, each of which contains a fully connected layer and a Sigmoid layer, which are used to adaptively calculate the weights of the features, and the weights are multiplied by the results of the second and third time-delay neural units respectively, and then the sum is calculated; then, the fourth time-delay neural unit is used to process the previous feature map by considering only one frame of speech information, and then the attention module processes the feature map obtained by the fourth time-delay neural unit; finally, the third TDNN layer accepts the outputs of N SK-TDNN layers, aggregates language information at different levels, and obtains the feature map required by the TDNN module. Figure 2It is reflected in the TDNN module module, which is used to assist the Mamba module in voiceprint recognition;

[0087] The DenseNet module consists of two convolutional layers and two residual layers. The first convolutional layer performs convolution processing on the received speech acoustic features to extract the local features of the speech data. Then, the skip connection between the two residual layers enables the network to learn more effective feature representations and alleviate the gradient vanishing problem in deep network training. Finally, the second convolutional layer performs convolution processing on the feature map output by the second residual layer to merge the frequency domain features of the feature map with the original frequency domain and channel features. Figure 2 It is reflected in the DenseNet module and is used to assist the Mamba module in voiceprint recognition.

[0088] S7. The target local features extracted by the above DenseNet module and TDNN module will become the input information of the attention strategy pool. The attention strategy pool can extract key information from the voice data, assign different weights to the voice features of different frames, and obtain a more discriminative voiceprint vector.

[0089] Among them, the attention strategy pool module takes the target local features as input information and outputs the speech weighted voiceprint vector; for a speech containing T frames X = x 1 ,x 2 ,…,x T , a speech vector H=h with T frames can be calculated through the network 1 ,h 2 ,…,h T , where the scale of H is d h ×T; the calculation method of speech feature weight A is as follows:

[0090] A=σ(g(HW 1 )W 2 ),

[0091] Here, W 1 is a h ×d r The matrix, W 2 is a r ×d h , g(·) represents the activation function, σ represents the Sigmoid function, A=a 1 ,a 2 ,…,a T , the scale of A is d h ×T;

[0092] The mean μ of the speech vector calculated based on the attention strategy pool is:

[0093]

[0094] Among them, a t is the weight of the speech vector of the tth frame, h t is the speech vector of the tth frame, T represents the total number of speech frames;

[0095] The standard deviation σ of the speech vector calculated based on the attention strategy pool is:

[0096]

[0097] Among them, a t is the weight of the speech vector of the tth frame, h t is the speech vector of the tth frame, μ is the mean of the speech vector, and T represents the total number of speech frames;

[0098] The voiceprint vector V obtained by the attention strategy pool is expressed as:

[0099]

[0100] Here, μ is the mean of the speech vector and σ is the standard deviation of the speech vector;

[0101] S8. After the attention strategy pool, the local features will be integrated in the fully connected module to form global features.

[0102] Among them, the fully connected module contains two fully connected layers: the first fully connected layer is responsible for receiving the voiceprint vector from the attention strategy pool module and calculating the voiceprint embedding vector; the voiceprint embedding vector is a further abstraction and representation of the voiceprint features, providing a more compact and highly discriminative feature representation for subsequent classification and recognition; the second fully connected layer is responsible for receiving the voiceprint embedding vector output by the first fully connected layer, and on this basis predicting and outputting the classification information corresponding to the speech information.

[0103] (6) Model decision module

[0104] The model decision module is used to evaluate and provide feedback on the effects of model training in real time, including model evaluation and model iteration.

[0105] Evaluation model: The test discrimination module uses the voiceprint recognition model trained in the current stage to evaluate the effect of real-time feedback model training, including but not limited to false acceptance rate, false rejection rate, equal error rate, and minimum detection cost function.

[0106] Model iteration: After multiple iterations of training, the voiceprint recognition method based on Mamba combined with DenseNet and TDNN will obtain a mature voiceprint recognition network model. In the testing phase, the voiceprint recognition model trained in the current phase can be used to evaluate and provide feedback on the effect of model training in real time, so as to make necessary adjustments and optimizations.

[0107] The system has demonstrated excellent recognition performance and good generalization capabilities on datasets such as VoxCeleb. The VoxCeleb dataset is an open source speaker recognition corpus that contains more than 1 million voice clips of more than 7,000 celebrities. These clips are extracted from YouTube videos using speech recognition and facial verification technology and are used to research and develop speaker recognition and verification systems.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An adaptive voiceprint recognition system based on the Mamba model, characterized in that: It includes speech acquisition module, speech enhancement module, speech feature extraction module, spectrogram enhancement module, model training module and model decision module. The voice collection module is used to collect voice data; The speech enhancement module is used to enhance the collected speech data, including adding reverberation, adding noise, adding speed disturbance and frequency domain masking; The speech feature extraction module is used to extract the features of the speech data after enhancement processing, including acoustic feature extraction, feature regularization processing, FBank feature extraction and network input of feature vector; The speech spectrogram enhancement module is used to process the FBank features of the speech, including frequency domain mask processing and time domain mask processing; The model training module is used to establish a voiceprint recognition model and train the model, and adopts a Mamba-based adaptive voiceprint recognition method to establish a voiceprint recognition model and train the model; The model decision module is used to evaluate and provide feedback on the effect of model training in real time, including model evaluation and model iteration.

2. According to claim 1, the adaptive voiceprint recognition system based on the Mamba model is characterized in that: For the speech enhancement module: The adding of reverberation is used to enhance the authenticity and diversity of voice data by simulating sound reflection and delayed superposition in different environments. The adding of reverberation can simulate the sound effects of different spaces. The added noise is used to introduce different types of background noise into the speech data to improve the robustness of the model to noise in the actual environment; The added speed disturbance is used to achieve fine adjustment of the speech playback speed by performing time expansion and contraction processing on the speech signal; The frequency domain masking is used to add noise or modulated signals at specific frequencies to simulate interference in a real environment and improve the robustness and adaptability of the model to complex sound environments.

3. According to claim 1, the adaptive voiceprint recognition system based on the Mamba model is characterized in that: For the speech feature extraction module: The acoustic feature extraction is used to extract acoustic features from the collected speech data; The feature regularization process is used to regularize the extracted acoustic features, thereby eliminating noise and irregularity in the data; The FBank feature extraction is used to further perform FBank feature extraction on the regularized speech data to generate an 80-dimensional FBank feature vector; The network input of the feature vector is used to send the obtained 80-dimensional FBank feature vector to the spectrogram enhancement module, and then to the Mamba-based voiceprint recognition network.

4. The adaptive voiceprint recognition system based on the Mamba model according to claim 1, characterized in that: For the spectrogram enhancement module: The specific operation of the frequency domain masking process is: within the frequency domain range of the FBank feature, a frequency band with a width of 0 to 8 is randomly selected for masking; the frequency domain masking process is to set the spectrum value in the selected frequency band to 0 to simulate the spectrum information loss or interference that may be encountered in actual applications; The specific operation of the time domain masking process is: within the time domain range of the FBank feature, randomly select a time period with a width of 0 to 10 for masking; the time domain masking process is to set the spectrum value in the selected time period to 0 to simulate the loss or interference of the speech signal in different time periods.

5. The adaptive voiceprint recognition system based on the Mamba model according to claim 1, characterized in that: For the model training module, the specific steps of the adaptive voiceprint recognition method based on Mamba are as follows: S1. Use the Mamba-based network to learn the feature data and establish a voiceprint recognition model; S2. In the voiceprint recognition model, the speech recognition process begins by segmenting the input spectrogram into smaller segments, which are then mapped into a series of tokens. The encoder in the ViM model receives these tokens and performs two independent linear transformations on each token to extract deep features. The ViM model uses a bidirectional processing mechanism to simulate the function of a bidirectional neural network layer, that is, it processes token sequences in both the forward and reverse directions to capture information in speech signals. S3. In each processing direction, a one-dimensional convolution operation is first applied to extract local features, followed by a Sigmoid linear unit activation function to enhance the nonlinear expression capability, and an additional learnable classification tag is added to the token sequence; S4. After the convolution and activation operations, an additional linear transformation is performed and a softplus function is applied to ensure that all output values ​​are positive, in preparation for the labeling of the state-space model sequence modeling function; then, a gating mechanism is introduced to regulate the information flow, which controls the contribution of each direction processing to the final result by multiplying the output of the state-space model with the elements of the forward and reverse sequences after SiLU activation; S5. Use residual connection to add the original input sequence to the gated output and output a new labeled sequence; S6. While using Mamba for voiceprint recognition, extract Mel-frequency cepstral coefficient features from the spectrogram through fast Fourier transform, and extract the target local features through the time-delay neural network module and the DenseNet module in turn; S7. The target local features extracted by the above-mentioned DenseNet module and TDNN module will become the input information of the attention strategy pool; the attention strategy pool can extract key information from the voice data, assign different weights to the voice features of different frames, and obtain a more discriminative voiceprint vector; S8. After the attention strategy pool, the local features will be integrated in the fully connected module to form global features.

6. The adaptive voiceprint recognition system based on the Mamba model according to claim 5, characterized in that: In step S6, the time-delay neural network module is composed of three standard time-delay neural network layers and N time-delay neural network layers with special structures, where N is an integer greater than 1; The DenseNet module consists of two convolutional layers and two residual layers. The first convolutional layer performs convolution processing on the received speech acoustic features to extract local features of the speech data. The jump connection between the two residual layers enables the network to learn more effective feature representations and alleviate the gradient vanishing problem in deep network training. The second convolutional layer performs convolution processing on the feature map output by the second residual layer, and merges the frequency domain features of the feature map with the original frequency domain and channel features to assist the Mamba module in voiceprint recognition.

7. The adaptive voiceprint recognition system based on the Mamba model according to claim 5, characterized in that: In step S7, the attention strategy pool module uses the target local features as input information and outputs a speech weighted voiceprint vector; for a speech X containing T frames, X = x1, x2, ..., x T , a speech vector H=h1,h2,…,h with T frames can be calculated through the network T , where the scale of H is d h ×T, the calculation method of speech feature weight A is as follows: A=σ(g(HW1)W2), Where W1 is a d h ×d r matrix, W2 is a d r ×d h , g(·) represents the activation function, σ represents the Sigmoid function, A=a1,a2,…,a T , the scale of A is d h ×T; The mean μ of the speech vector calculated based on the attention strategy pool is: Among them, a t is the weight of the speech vector of the tth frame, h t is the speech vector of the tth frame, T represents the total number of speech frames; The standard deviation σ of the speech vector calculated based on the attention strategy pool is: Among them, a t is the weight of the speech vector of the tth frame, h t is the speech vector of the tth frame, μ is the mean of the speech vector, and T represents the total number of speech frames; The voiceprint vector V obtained by the attention strategy pool is expressed as: Where μ is the mean of the speech vector and σ is the standard deviation of the speech vector.

8. The adaptive voiceprint recognition system based on the Mamba model according to claim 5, characterized in that: In step S8, the fully connected module includes two fully connected layers: the first fully connected layer is responsible for receiving the voiceprint vector from the attention strategy pool module and calculating the voiceprint embedding vector; the second fully connected layer is responsible for receiving the voiceprint embedding vector output by the first fully connected layer, and on this basis predicting and outputting the classification information corresponding to the speech information.

9. The adaptive voiceprint recognition system based on the Mamba model according to any one of claims 1 to 8, characterized in that: For the model decision module: The evaluation model is used to test and determine the effect of using the voiceprint recognition model trained in the current stage to evaluate the training of the real-time feedback model, including but not limited to the false acceptance rate, false rejection rate, equal error rate, and minimum detection cost function; The model iteration is used to update the training model through multiple iterations, and the adaptive voiceprint recognition method based on DenseNet and TDNN will obtain a mature voiceprint recognition network model.

Images