A method and system for speaker recognition based on deep speech embedding

Abstract

This invention provides a voiceprint recognition method and system based on deep speech embedding, comprising: acquiring speech data and obtaining buffered waveform data based on the speech data; resampling based on the buffered waveform data and a preset sampling rate to obtain sampled speech data; segmenting the speech data into segments based on the sampled speech data and a preset window time to obtain segmented speech data; performing data augmentation processing on the segmented speech data to obtain enhanced speech data; extracting features from the enhanced speech data based on a pre-trained model to obtain high-dimensional feature vectors; calculating frame-level statistical information based on the high-dimensional feature vectors and obtaining segment feature vectors based on an attention mechanism and frame-level statistical information; and performing feature vector matching based on the segment feature vectors to perform identity mapping and achieve voiceprint recognition. The voiceprint recognition method and system based on deep speech embedding proposed in this invention ensures the accuracy of voiceprint recognition and greatly improves the efficiency and reliability of voiceprint recognition.

Description

Technical Field

[0001] This invention relates to the field of intelligent processing of voice data, and in particular to a method and system for voiceprint recognition based on deep voice embedding. Background Technology

[0002] Voiceprint recognition technology is a method of identity verification that analyzes and identifies individual voice characteristics. Because everyone's vocal cord structure, oral cavity shape, and other physiological characteristics are different, each person's voice has its own unique voiceprint characteristics. Voiceprint recognition technology uses these unique biometric features to achieve identity verification.

[0003] However, traditional cryptographic and biometric identification methods have gradually revealed limitations in performance and timeliness in practical applications, mainly manifested in the following shortcomings: (1) insufficient accuracy of voiceprint recognition; (2) most of the voice feature processing is for offline audio, with insufficient consideration for real-time audio processing; (3) weak resistance to environmental noise in complex scenarios. Summary of the Invention

[0004] The present invention aims to provide a voiceprint recognition method and system based on deep speech embedding to solve the above-mentioned technical problems and effectively improve the recognition ability and accuracy of the voiceprint recognition system in complex environments.

[0005] To address the aforementioned technical problems, this invention provides a speaker recognition method based on deep speech embedding, comprising the following steps:

[0006] Acquire speech data and obtain buffered waveform data based on the speech data;

[0007] Resampling is performed based on buffered waveform data and a preset sampling rate to obtain sampled speech data;

[0008] Segmentation is performed based on sampled speech data and a preset window time to obtain segmented speech data;

[0009] Data augmentation processing is performed on segmented speech data to obtain augmented speech data;

[0010] Feature extraction is performed on enhanced speech data based on a pre-trained model to obtain high-dimensional feature vectors;

[0011] Frame-level statistical information is calculated based on high-dimensional feature vectors, and paragraph feature vectors are obtained based on attention mechanisms and frame-level statistical information.

[0012] Feature vector matching is performed based on paragraph feature vectors to achieve identity mapping and voiceprint recognition.

[0013] The above scheme obtains high-dimensional feature vectors after noise reduction and data augmentation by resampling, segmenting, data augmentation, and feature extraction of speech data. This effectively suppresses background noise, enhances the quality of speech data, and extracts refined high-dimensional feature vectors from the speech data, thereby improving the accuracy and robustness of subsequent voiceprint recognition and enhancing its recognition capability and adaptability in complex environments. Furthermore, based on an attention mechanism, it aggregates frame-level statistical information to accurately aggregate frame-level features into segment-level representations for final voice identity mapping. This enables rapid and accurate identification of the speaker's identity, ensuring the accuracy of voiceprint recognition and improving its efficiency and reliability.

[0014] Furthermore, acquiring voice data and obtaining buffered waveform data based on the voice data includes: acquiring initial voice information; acquiring voice data based on a preset format processing method and the initial voice information; and acquiring buffered waveform data based on a local buffering method and the voice data.

[0015] In the above scheme, the voice data is processed according to a preset format and buffered locally to complete the preprocessing process of the voice data, so as to be able to receive voice data of different formats and sources and perform subsequent recognition.

[0016] Furthermore, resampling is performed based on the buffered waveform data and a preset sampling rate to obtain sampled speech data. The specific calculation process satisfies the following formula:

[0017]

[0018] In the formula: x represents the buffered waveform data, y(n) represents the resampled speech data, and f out f represents the target sampling rate. in This represents the original sampling rate, and n represents all integers.

[0019] In the above scheme, by setting a certain sampling rate to resample the buffer waveform, it is ensured that enough sound information can be captured in each time interval, thereby reducing the complexity of voice data processing and storage while maintaining sufficient audio quality.

[0020] Furthermore, based on the sampled speech data and a preset window time, segmentation is performed to obtain segmented speech data. The specific calculation process satisfies the following formula:

[0021] L = T × SA

[0022] In the formula: L represents the window length of the segmented speech data, T represents the preset window time, and SA represents the sampling rate after resampling.

[0023] In the above scheme, the input audio segments are cut into fixed-length audio segments by using a fixed time window, ensuring that each segment contains sufficient speech information, while avoiding data redundancy and increased processing complexity caused by excessively long segments.

[0024] Furthermore, data augmentation processing is performed on the segmented speech data to obtain augmented speech data, including:

[0025] Noise estimation is performed based on segmented speech data and noise estimation algorithms to obtain the noise spectrum. The specific calculation process satisfies the following formula:

[0026]

[0027] In the formula, X(t,f) represents the segmented speech data. Represents the estimated noise spectrum, and EstimateNoise represents the minimum mean square error estimation function;

[0028] Based on the noise spectrum and segmented speech data, filtering is performed to obtain filtered speech data. The specific calculation process satisfies the following formula:

[0029]

[0030] In the formula, t represents a certain moment in the time domain, f represents a certain frequency in the frequency domain, Y(t,f) represents the filtered speech data, and S(t,f) represents the spectrum of the speech signal. Represents the estimated noise spectrum;

[0031] Enhanced speech data is obtained by performing data augmentation and smoothing on filtered speech data.

[0032] In the above scheme, noise estimation is performed on the segmented speech data, that is, the background noise characteristics are estimated by using a noise estimation algorithm. By analyzing the silent or low-energy segments of the audio signal, the spectral characteristics of the background noise are identified. Then, the filtering parameters are adaptively adjusted according to the noise spectrum for filtering processing, and the effective components of the signal are preserved to obtain enhanced speech data. This effectively suppresses background noise, enhances the quality of speech data, and thus improves the accuracy and robustness of subsequent voiceprint recognition.

[0033] Furthermore, feature extraction is performed on the enhanced speech data based on the pre-trained model to obtain high-dimensional feature vectors. This includes: introducing skip connections into the deep learning network model and pre-training the deep learning network model using a large-scale speech dataset to obtain a pre-trained model; and performing multi-layer convolution and pooling operations based on the pre-trained model and the enhanced speech data to obtain high-dimensional feature vectors.

[0034] In the above scheme, by introducing skip connections into the deep learning network model, the vanishing gradient problem in deep networks can be alleviated, thereby enabling more effective extraction of deep speech features. At the same time, the deep learning network model is pre-trained using a large-scale speech dataset, allowing the system to learn rich speech features from big data, which helps improve the accuracy of feature extraction. The enhanced speech data is then input into the pre-trained model and subjected to multiple convolution and pooling operations to extract a refined high-dimensional speech feature vector, that is, a high-dimensional feature vector containing key information of the speech signal, which is helpful for subsequent voiceprint recognition.

[0035] Furthermore, skip connections are introduced into the deep learning network model, and the deep learning network model is pre-trained using a large-scale speech dataset to obtain a pre-trained model. This includes: randomly deforming, cropping, or rotating the speech data based on the large-scale speech dataset to obtain a complex speech dataset under complex noise conditions; introducing skip connections into the deep learning network model, and pre-training the deep learning network model using the complex speech dataset to obtain a pre-trained model.

[0036] In the above scheme, by randomly deforming, cropping or rotating a large-scale speech dataset, the simulation of a complex noise environment is achieved on the basis of diverse data. This increases the diversity of data, enabling deep learning network models to learn a wider range of voiceprint features. As a result, speech data recognition and processing can be achieved in complex noise environments, thereby improving the recognition ability and adaptability in complex real-world scenarios.

[0037] Furthermore, frame-level statistical information is calculated based on high-dimensional feature vectors, and paragraph feature vectors are obtained based on the attention mechanism and frame-level statistical information, including:

[0038] Statistical information, including the mean and standard deviation, is obtained based on statistical pooling functions and high-dimensional feature vectors. The specific calculation process satisfies the following formula:

[0039]

[0040] In the formula, x i Let mean(x) represent the high-dimensional feature vector of the i-th frame, where N represents the number of frames. i std(x) represents the mean. i () represents the standard deviation;

[0041] Attention weights for obtaining high-dimensional feature vectors based on deep neural networks are calculated using the following formula:

[0042]

[0043] In the formula: W and b represent preset learning parameters, A(x i) represents the attention weight of the high-dimensional feature vector of the i-th frame, and T represents the period;

[0044] The paragraph feature vector is obtained by calculating a weighted average feature vector based on the attention pooling mechanism and attention weights. The specific calculation process satisfies the following formula:

[0045]

[0046] In the formula: P(x) represents the paragraph feature vector.

[0047] In the above scheme, statistical pooling function is first used to calculate the statistical information of high-dimensional feature vectors at the frame level, and deep neural network is used to learn the attention weights of high-dimensional feature vectors. Finally, attention mechanism is used to learn the weights of different frame-level features and perform weighted averaging of feature vectors to emphasize important features and ignore irrelevant noise. By combining statistical pooling function and attention mechanism, frame-level features are aggregated more accurately to obtain segment feature vectors and capture important information in speech signals.

[0048] Furthermore, feature vector matching is performed based on paragraph feature vectors to achieve identity mapping and voiceprint recognition, including:

[0049] Feature vector matching is performed based on paragraph feature vectors and a feature vector database to obtain matching similarity. The specific calculation process satisfies the following formula:

[0050]

[0051] In the formula, x represents the paragraph feature vector, y represents the feature vector in the feature vector database, and similarity(x,y) represents the matching similarity.

[0052] Based on preset matching principles and matching similarity, a matching mapping is performed to obtain the corresponding identity mapping label, thereby realizing voiceprint recognition.

[0053] In the above scheme, the similarity of feature vectors is calculated and mapped to the corresponding speaker's voiceprint label to achieve voiceprint recognition, which can quickly and accurately identify the speaker's identity and improve the efficiency and reliability of the system.

[0054] The aforementioned scheme obtains high-dimensional feature vectors after noise reduction and data augmentation by resampling, segmenting, data augmentation, and feature extraction of speech data. This effectively suppresses background noise and enhances the quality of speech data, thus extracting refined high-dimensional feature vectors. Furthermore, it uses large-scale speech data for processing to simulate more complex speech data, training a deep learning network model to enhance its noise resistance and adaptability, enabling it to maintain a high recognition rate in complex environments. This improves the accuracy and robustness of subsequent voiceprint recognition, enhancing its recognition ability and adaptability in complex environments. Finally, based on an attention mechanism, it aggregates frame-level statistical information to accurately represent it at the segment level, ultimately mapping the speaker's identity. This allows for rapid and accurate identification of the speaker, ensuring the accuracy of voiceprint recognition and improving its efficiency and reliability.

[0055] The present invention also provides a voiceprint recognition system based on deep speech embedding, comprising:

[0056] The voice acquisition module is used to acquire voice data and obtain buffered waveform data based on the voice data;

[0057] The data sampling module is used to resample the buffered waveform data at a preset sampling rate to obtain sampled speech data;

[0058] The segmentation module is used to segment speech data based on sampled speech data and a preset window time to obtain segmented speech data.

[0059] The data augmentation module is used to perform data augmentation processing on segmented speech data to obtain augmented speech data;

[0060] The feature extraction module is used to extract features from enhanced speech data based on a pre-trained model to obtain high-dimensional feature vectors.

[0061] The feature aggregation module is used to calculate frame-level statistical information based on high-dimensional feature vectors, and to obtain paragraph feature vectors based on attention mechanisms and frame-level statistical information.

[0062] The tag mapping module is used to perform feature vector matching based on paragraph feature vectors, thereby performing identity mapping and realizing voiceprint recognition.

[0063] The aforementioned solution provides a voiceprint recognition system that performs resampling, segmentation, data enhancement, and feature extraction operations on speech data through a speech acquisition module, a data sampling module, a segmentation module, a data enhancement module, and a feature extraction module. This process yields high-dimensional feature vectors after noise reduction and data enhancement, effectively suppressing background noise and improving the quality of speech data. It can extract refined high-dimensional feature vectors from the speech data, thereby improving the accuracy and robustness of subsequent voiceprint recognition and enhancing its recognition capability and adaptability in complex environments. Furthermore, based on a feature aggregation module, an attention mechanism is used to aggregate frame-level statistical information, accurately aggregating frame-level features to segment-level representations. Finally, a label mapping module maps the voice to the speaker's identity, enabling rapid and accurate identification of the speaker's identity, ensuring the accuracy of voiceprint recognition, and improving recognition efficiency and reliability. Attached Figure Description

[0064] Figure 1 A schematic diagram of a voiceprint recognition method based on deep speech embedding provided in an embodiment of the present invention;

[0065] Figure 2 This is a schematic diagram of a voiceprint recognition system based on deep speech embedding, provided as an embodiment of the present invention. Detailed Implementation

[0066] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] Example 1:

[0068] This embodiment provides a speaker recognition method based on deep speech embedding, including the following steps:

[0069] S1: Acquire voice data and obtain buffered waveform data based on the voice data;

[0070] S2: Resample based on buffered waveform data and preset sampling rate to obtain sampled speech data;

[0071] S3: Segment the audio data based on the sampled audio data and the preset window time to obtain segmented audio data;

[0072] S4: Perform data augmentation processing on the segmented speech data to obtain augmented speech data;

[0073] S5: Extract features from enhanced speech data based on a pre-trained model to obtain high-dimensional feature vectors;

[0074] S6: Calculate frame-level statistical information based on high-dimensional feature vectors, and obtain paragraph feature vectors based on attention mechanism and frame-level statistical information;

[0075] S7: Based on paragraph feature vectors, feature vector matching is performed to achieve identity mapping and voiceprint recognition.

[0076] The above scheme obtains high-dimensional feature vectors after noise reduction and data augmentation by resampling, segmenting, data augmentation, and feature extraction of speech data. This effectively suppresses background noise, enhances the quality of speech data, and extracts refined high-dimensional feature vectors from the speech data, thereby improving the accuracy and robustness of subsequent voiceprint recognition and enhancing its recognition capability and adaptability in complex environments. Furthermore, based on an attention mechanism, it aggregates frame-level statistical information to accurately aggregate frame-level features into segment-level representations for final voice identity mapping. This enables rapid and accurate identification of the speaker's identity, ensuring the accuracy of voiceprint recognition and improving its efficiency and reliability.

[0077] Optionally, step S1 includes: acquiring initial speech information; acquiring speech data based on a preset format processing method and the initial speech information; and acquiring buffered waveform data based on a local buffering method and the speech data.

[0078] In the specific implementation process, by processing the voice data according to the preset format and performing local buffering of the voice data, the system can flexibly support data testing and complete the preprocessing of the voice data, thereby enabling the reception of voice data of different formats and sources and subsequent recognition.

[0079] Optionally, the specific calculation process of step S2 satisfies the following formula:

[0080]

[0081] In the formula: x represents the buffered waveform data, y(n) represents the resampled speech data, and f out f represents the target sampling rate. in This represents the original sampling rate, and n represents all integers.

[0082] In the specific implementation process, the buffer waveform is resampled by setting a certain sampling rate to ensure that enough sound information can be captured in each time interval and to ensure that the sampling rate of all input audio is consistent with the target voiceprint sampling rate stored in the database. The resampling sampling rate is set to 16000Hz. This sampling rate can reduce the complexity of voice data processing and storage while maintaining sufficient audio quality.

[0083] Optionally, the specific calculation process of step S3 satisfies the following formula:

[0084] L = T × SA

[0085] In the formula: L represents the window length of the segmented speech data, T represents the preset window time, and SA represents the sampling rate after resampling.

[0086] In the specific implementation process, the input audio segments are cut into fixed-length audio segments by using a fixed time window (such as 25 milliseconds) to ensure that each segment contains sufficient speech information, while avoiding data redundancy and increased processing complexity caused by excessively long segments.

[0087] Optionally, step S4 includes:

[0088] Noise estimation is performed based on segmented speech data and noise estimation algorithms to obtain the noise spectrum. The specific calculation process satisfies the following formula:

[0089]

[0090] In the formula, X(t,f) represents the segmented speech data. Represents the estimated noise spectrum, and EstimateNoise represents the minimum mean square error estimation function;

[0091] Based on the noise spectrum and segmented speech data, filtering is performed to obtain filtered speech data. The specific calculation process satisfies the following formula:

[0092]

[0093] In the formula, t represents a certain moment in the time domain, f represents a certain frequency in the frequency domain, Y(t,f) represents the filtered speech data, and S(t,f) represents the spectrum of the speech signal. Represents the estimated noise spectrum;

[0094] Enhanced speech data is obtained by performing data augmentation and smoothing on filtered speech data.

[0095] In the specific implementation process, the segmented speech data is processed by filters to suppress background noise and enhance the speech signal. First, noise estimation is performed on the segmented speech data, that is, the characteristics of background noise are estimated using noise estimation algorithms, and the spectral characteristics of background noise are identified by analyzing the silent or low-energy segments of the audio signal. Then, a noise suppression filter (such as a Wiener filter or an adaptive filter) is applied to process the audio signal. The filter adaptively adjusts the filtering parameters according to the estimated noise characteristics to suppress background noise while retaining the effective components of the speech signal. Finally, the filtered signal is converted back to the time domain through inverse time and frequency domain transformation and smoothed to enhance the clarity and intelligibility of the speech signal, resulting in enhanced speech data. This effectively suppresses background noise, enhances the quality of speech data, and thus improves the accuracy and robustness of subsequent voiceprint recognition.

[0096] Optionally, step S5 includes: introducing skip connections into the deep learning network model and pre-training the deep learning network model using a large-scale speech dataset to obtain a pre-trained model; and performing multi-layer convolution and pooling operations based on the pre-trained model and enhanced speech data to obtain high-dimensional feature vectors.

[0097] In the specific implementation process, the speech data is used to extract features through a pre-trained deep learning network model to generate high-dimensional feature vectors. Specifically, the following steps can be performed using a residual network (ResNet): (1) Use ResNet to pre-train the model: Introducing skip connections into the residual network can alleviate the gradient vanishing problem in deep networks, thus making it more advantageous in processing complex speech data and better able to capture subtle features in the speech signal; at the same time, a large-scale speech dataset is selected to pre-train the deep learning network model, so that the system can learn rich speech features from big data, which helps to improve the accuracy of feature extraction; (2) Input the enhanced speech data into the pre-trained model and perform multi-layer convolution and pooling operations to extract a fine high-dimensional speech feature vector, that is, a high-dimensional feature vector containing key information of the speech signal, which helps with subsequent voiceprint recognition.

[0098] Optionally, skip connections are introduced into the deep learning network model, and the deep learning network model is pre-trained using a large-scale speech dataset to obtain a pre-trained model. This includes: randomly deforming, cropping, or rotating the speech data based on the large-scale speech dataset to obtain a complex speech dataset under complex noise conditions; introducing skip connections into the deep learning network model, and pre-training the deep learning network model using the complex speech dataset to obtain a pre-trained model.

[0099] In the specific implementation process, by randomly deforming, cropping or rotating large-scale speech datasets, the simulation of complex noise environments is achieved on the basis of diversified data. This increases the diversity of data, enabling deep learning network models to learn a wider range of voiceprint features. As a result, speech data recognition and processing in complex noise environments can be achieved, thereby improving the recognition ability and adaptability in complex real-world scenarios.

[0100] Optionally, step S6 includes:

[0101] Statistical information, including the mean and standard deviation, is obtained based on statistical pooling functions and high-dimensional feature vectors. The specific calculation process satisfies the following formula:

[0102]

[0103]

[0104] In the formula, x i Let mean(x) represent the high-dimensional feature vector of the i-th frame, where N represents the number of frames. i std(x) represents the mean. i () represents the standard deviation;

[0105] Attention weights for obtaining high-dimensional feature vectors based on deep neural networks are calculated using the following formula:

[0106]

[0107] In the formula: W and b represent preset learning parameters, A(x i ) represents the attention weight of the high-dimensional feature vector of the i-th frame, and T represents the period;

[0108] The paragraph feature vector is obtained by calculating a weighted average feature vector based on the attention pooling mechanism and attention weights. The specific calculation process satisfies the following formula:

[0109]

[0110] In the formula: P(x) represents the paragraph feature vector.

[0111] In the specific implementation process, a pooling function based on statistical and attention mechanisms is adopted to aggregate frame-level features into paragraph-level representations. The statistical pooling function calculates the statistical information (such as mean and standard deviation) of each feature, while the attention mechanism emphasizes important features and ignores irrelevant noise by learning weights. The specific steps are as follows: (1) Statistical pooling: Calculate the statistical information of the feature vector at the frame level, including the mean and standard deviation. (2) Attention pooling: Adopt the attention mechanism, learn the weights of different frame-level features, and perform a weighted average of the feature vectors. The attention weights are learned through a deep neural network. For a given series of frame-level features {x1, x2, ..., x...} N} Calculate the attention weight A(x) for each frame. i Based on these weights, a weighted average feature vector is calculated. By combining statistical methods with attention pooling functions, frame-level features can be more accurately aggregated into segment-level representations, capturing important information in the speech signal.

[0112] Optionally, step S7 includes:

[0113] Feature vector matching is performed based on paragraph feature vectors and a feature vector database to obtain matching similarity. The specific calculation process satisfies the following formula:

[0114]

[0115] In the formula, x represents the paragraph feature vector, y represents the feature vector in the feature vector database, and similarity(x,y) represents the matching similarity.

[0116] Based on preset matching principles and matching similarity, a matching mapping is performed to obtain the corresponding identity mapping label, thereby realizing voiceprint recognition.

[0117] In the specific implementation process, feature vector matching is performed by calculating the similarity of feature vectors. The paragraph-level feature vectors are compared with the speaker tag feature vectors in the database. By calculating the similarity between the vectors, the speaker's identity is identified. Then, the feature vector with the highest similarity is mapped to the corresponding speaker tag to realize voiceprint recognition. This enables the rapid and accurate identification of the speaker's identity, improving the efficiency and reliability of the system.

[0118] In the specific implementation process, compared with the conventional preset tag library for mapping and matching, this embodiment achieves more accurate feature vector matching: (1) Frame-level feature aggregation: Before performing feature vector matching, the frame-level features are aggregated by a pooling function based on statistics and attention mechanisms to obtain a paragraph-level feature vector that more accurately reflects the important information of the speech signal. Compared with the conventional method of matching using only a single feature vector, the aggregated feature vector contains richer information, making the matching more accurate. (2) Paragraph-level audio feature vector similarity calculation: A specific similarity calculation formula is used to calculate the similarity of paragraph-level audio feature vectors. The inner product and magnitude of the vectors are considered, which can more scientifically measure the similarity between two paragraph-level audio feature vectors. Compared with the conventional simple distance calculation or other similarity calculation methods, this formula can better adapt to the characteristics of voiceprint recognition and improve the accuracy of recognition. And it achieves more efficient identity mapping. Based on the principle of the highest similarity, this embodiment performs identity mapping based on the feature vector with the highest similarity, which can quickly and accurately determine the speaker's identity. When dealing with a large amount of data and complex scenarios, this method can efficiently filter out the most likely speaker tags and improve the system's response speed and efficiency.

[0119] The aforementioned scheme obtains high-dimensional feature vectors after noise reduction and data augmentation by resampling, segmenting, data augmentation, and feature extraction of speech data. This effectively suppresses background noise and enhances the quality of speech data, thus allowing for the extraction of refined high-dimensional feature vectors. Furthermore, it employs large-scale speech data processing to simulate more complex speech data, training a deep learning network model to enhance its noise resistance and adaptability. This enables the model to maintain a high recognition rate in complex environments, thereby improving the accuracy and robustness of subsequent voiceprint recognition and enhancing its recognition capability and adaptability in complex environments. Finally, based on an attention mechanism, it aggregates frame-level statistical information to accurately represent segment-level features for final voice identity mapping. This allows for rapid and accurate speaker identification, ensuring the accuracy of voiceprint recognition and significantly improving the system's recognition capability and adaptability in complex environments.

[0120] This embodiment also provides a voiceprint recognition system based on deep speech embedding, including:

[0121] The voice acquisition module is used to acquire voice data and obtain buffered waveform data based on the voice data;

[0122] The data sampling module is used to resample the buffered waveform data at a preset sampling rate to obtain sampled speech data;

[0123] The segmentation module is used to segment speech data based on sampled speech data and a preset window time to obtain segmented speech data.

[0124] The data augmentation module is used to perform data augmentation processing on segmented speech data to obtain augmented speech data;

[0125] The feature extraction module is used to extract features from enhanced speech data based on a pre-trained model to obtain high-dimensional feature vectors.

[0126] The feature aggregation module is used to calculate frame-level statistical information based on high-dimensional feature vectors, and to obtain paragraph feature vectors based on attention mechanisms and frame-level statistical information.

[0127] The tag mapping module is used to perform feature vector matching based on paragraph feature vectors, thereby performing identity mapping and realizing voiceprint recognition.

[0128] The aforementioned solution provides a voiceprint recognition system that performs resampling, segmentation, data enhancement, and feature extraction operations on speech data through a speech acquisition module, a data sampling module, a segmentation module, a data enhancement module, and a feature extraction module. This process yields high-dimensional feature vectors after noise reduction and data enhancement, effectively suppressing background noise and improving the quality of speech data. It can extract refined high-dimensional feature vectors from the speech data, thereby improving the accuracy and robustness of subsequent voiceprint recognition and enhancing its recognition capability and adaptability in complex environments. Furthermore, based on a feature aggregation module, an attention mechanism is used to aggregate frame-level statistical information, accurately aggregating frame-level features to segment-level representations. Finally, a label mapping module maps the voice to the speaker's identity, enabling rapid and accurate identification of the speaker's identity, ensuring the accuracy of voiceprint recognition, and improving recognition efficiency and reliability.

[0129] In the specific implementation process, a universal data input interface can be built in the voice acquisition module to receive voice data of different formats and sources, including real-time recording input, pre-recorded audio file upload, and streaming audio input. Whether it is pre-recorded audio read from local storage devices or audio collected in real time through a microphone, it can be conveniently input into the voiceprint recognition system of this embodiment for testing and recognition, and real-time audio processing can be realized. A buffer area is established locally to load the original waveform data of the received voice data into the buffer for buffering, thereby obtaining buffered waveform data.

[0130] In the specific implementation process, it also includes continuous monitoring and optimization of system performance, regular updates and retraining of the model to cope with the ever-changing speech environment and new sound input, to ensure the long-term stable and efficient operation of the system, and to continuously improve the system's recognition ability and adaptability by introducing new data and improving the model structure. Specifically, the system will be optimized in the following four aspects: (1) Diversified data acquisition, actively collecting speech data in different scenarios, including various noise environments (such as factories, streets, restaurants, etc.), speech with different accents and language habits, and speech of speakers of different ages and genders. This will enable the model to learn a wider range of voiceprint features and improve its adaptability in complex real-world scenarios. (2) Expansion of data augmentation techniques, in addition to existing data augmentation methods such as filtering, time and frequency domain transformation, exploring new data augmentation techniques. For example, adding different types of synthetic noise to the original speech data to simulate more complex noise environments; performing random deformation of speech signals, such as stretching, compression, and distortion, to increase the diversity of data. Combining data augmentation methods from other fields, such as the ideas of flipping, rotating, and cropping operations in the image field, and applying them to speech data. For example, random cropping or rotation of the speech spectrum can be achieved by randomly adjusting the phase of the spectrum. (3) Multi-model fusion: Combine different types of deep learning models, such as convolutional neural networks (CNN), recurrent neural networks (RNN), long short-term memory networks (LSTM), etc., and fuse them with existing residual networks (ResNet). Combining the different advantages of different types of models, CNN is good at extracting local features, while RNN and LSTM are good at processing sequence data. By fusion, the advantages of various models can be fully utilized to improve the accuracy of voiceprint recognition. Design a hybrid model structure to fuse the intermediate layer outputs of different models, or adopt a layered fusion method to gradually fuse the feature representations of different models. For example, CNN can be used to extract the local spectral features of speech first, then input into LSTM to process time series information, and finally fuse with the deep features extracted by ResNet. (4) Attention mechanism optimization: Further improve the existing attention mechanism to make it pay more attention to the key information in speech. For example, introduce a multi-scale attention mechanism to pay attention to the important features of speech at different time scales and frequency scales at the same time; or design an adaptive attention mechanism to automatically adjust the distribution of attention weights according to the noise level and complexity of the input speech. The attention mechanism can be more closely integrated with other modules, such as joint optimization with feature extraction and pooling modules. For example, during feature extraction, the attention mechanism can guide the extraction of important speech features more precisely; during pooling, the attention mechanism can be used to dynamically adjust the size and position of the pooling region.

[0131] This embodiment proposes a voiceprint recognition system based on deep speech embedding. This system extracts high-dimensional features through deep speech embedding technology and combines statistical pooling and attention pooling techniques to achieve high accuracy in voiceprint recognition. This method not only improves recognition performance but also supports online feature extraction, enabling real-time audio data processing, enhancing user experience, and is applicable to various application scenarios such as smart assistants and real-time monitoring. Furthermore, data augmentation technology enhances noise resistance and adaptability, maintaining a high recognition rate in complex environments. The system design incorporates adaptive learning capabilities, continuously optimizing its performance, and supports various forms of voiceprint data input. It supports online feature preparation, performing speech segmentation, data augmentation, and feature extraction online. It supports statistical and attention-based pooling functions, aggregating frame-level features into segment-level representations and ultimately mapping them to speaker tags, achieving accurate recognition of individual voiceprints and increasing flexibility. Finally, the system continuously monitors and optimizes its performance, ensuring long-term stable and efficient operation. Therefore, this invention has broad application prospects in fields such as financial security, smart homes, mobile device unlocking, and online identity authentication, providing users with a convenient and secure identity verification solution.

[0132] In practical applications, deep learning-based voiceprint recognition technology extracts high-dimensional features of speech, greatly improving the system's recognition ability and adaptability in complex environments: deep neural networks can capture more speech details and complex features, significantly improving the accuracy of voiceprint recognition; deep learning models can maintain high recognition rates under complex conditions such as noisy environments and speech variations; with the help of efficient deep learning algorithms, voiceprint recognition systems can process audio data in real time; real-time audio processing can improve user experience and can be widely used in scenarios such as intelligent assistants and real-time monitoring, ensuring that the system can respond and recognize quickly; deep speech embedding technology has strong anti-noise and anti-interference capabilities; by adding various environmental noises and interference factors during training, the model can work stably in various complex scenarios, ensuring the reliability of recognition results; deep learning models have adaptive learning capabilities, which can continuously optimize and improve their recognition performance; through continuous learning and training, the system can adapt to different users, environments, and devices, further improving the recognition effect; deep learning-based voiceprint recognition systems can be integrated with other biometric recognition technologies (such as facial recognition and fingerprint recognition) to achieve multimodal identity verification, providing higher security and reliability.

[0133] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A voiceprint recognition method based on deep speech embedding, characterized in that, Includes the following steps: Acquire speech data and obtain buffered waveform data based on the speech data; Resampling is performed based on buffered waveform data and a preset sampling rate to obtain sampled speech data; Segmentation is performed based on sampled speech data and a preset window time to obtain segmented speech data; Data augmentation processing is performed on segmented speech data to obtain augmented speech data; Feature extraction of enhanced speech data based on pre-trained models to obtain high-dimensional feature vectors includes: introducing skip connections into deep learning network models and pre-training the deep learning network models using large-scale speech datasets to obtain pre-trained models; and performing multi-layer convolution and pooling operations based on the pre-trained models and enhanced speech data to obtain high-dimensional feature vectors. Frame-level statistical information is calculated based on high-dimensional feature vectors, and segment feature vectors are obtained based on attention mechanisms and frame-level statistical information. Specifically, this includes: obtaining statistical information based on statistical pooling functions and high-dimensional feature vectors, wherein the statistical information includes the mean and standard deviation, and the specific calculation process satisfies the following formula: In the formula, Let N represent the high-dimensional feature vector of the i-th frame, and N represent the number of frames, with mean( std() represents the mean. () represents the standard deviation; The attention weights of the high-dimensional feature vector are obtained based on a deep neural network, and the specific calculation process satisfies the following formula: In the formula: W and b represent preset learning parameters, A( ) represents the attention weights of the high-dimensional feature vector of the i-th frame, and T represents the transpose. This represents the high-dimensional feature vector of the j-th frame; The paragraph feature vector is obtained by calculating a weighted average feature vector based on the attention pooling mechanism and attention weights. The specific calculation process satisfies the following formula: P( )= In the formula: P(x) represents the paragraph feature vector; Feature vector matching is performed based on paragraph feature vectors to achieve identity mapping and voiceprint recognition.

2. The speaker recognition method based on deep speech embedding according to claim 1, characterized in that, The acquisition of voice data and the generation of buffered waveform data based on the voice data include: Acquire initial voice information, and acquire voice data based on a preset format processing method and the initial voice information; Buffered waveform data is obtained based on local buffering methods and voice data.

3. The speaker recognition method based on deep speech embedding according to claim 1, characterized in that, The resampling based on buffered waveform data and a preset sampling rate to obtain sampled speech data is specifically calculated according to the following formula: y(n)=x(n× ) In the formula: x represents the buffered waveform data, y(n) represents the resampled speech data, and f out f represents the target sampling rate. in This represents the original sampling rate, and n represents all integers.

4. The speaker recognition method based on deep speech embedding according to claim 1, characterized in that, The segmentation based on sampled speech data and a preset window time yields segmented speech data, and the specific calculation process satisfies the following formula: In the formula: L represents the window length of the segmented speech data. Indicates the preset window time. This represents the sampling rate after resampling.

5. The speaker recognition method based on deep speech embedding according to claim 1, characterized in that, The data augmentation processing based on segmented speech data to obtain augmented speech data includes: Noise estimation is performed based on segmented speech data and noise estimation algorithms to obtain the noise spectrum. The specific calculation process satisfies the following formula: (t,f)=EstimateNoise(X(t,f)) In the formula, X(t,f) represents the segmented speech data. (t,f) represents the estimated noise spectrum, and EstimateNoise represents the minimum mean square error estimation function; Based on the noise spectrum and segmented speech data, filtering is performed to obtain filtered speech data. The specific calculation process satisfies the following formula: In the formula, t represents a certain moment in the time domain, f represents a certain frequency in the frequency domain, Y(t,f) represents the filtered speech data, and S(t,f) represents the spectrum of the speech signal. Represents the estimated noise spectrum; Enhanced speech data is obtained by performing data augmentation and smoothing on filtered speech data.

6. The speaker recognition method based on deep speech embedding according to claim 1, characterized in that, The method of introducing skip connections into a deep learning network model and pre-training the deep learning network model using a large-scale speech dataset to obtain a pre-trained model includes: Based on large-scale speech datasets, speech data is randomly deformed, cropped, or rotated to obtain complex speech datasets under complex noise environments. Skip connections are introduced into the deep learning network model, and the deep learning network model is pre-trained using a complex speech dataset to obtain a pre-trained model.

7. The speaker recognition method based on deep speech embedding according to claim 1, characterized in that, The process of matching feature vectors based on paragraph feature vectors to perform identity mapping and achieve voiceprint recognition includes: Feature vector matching is performed based on paragraph feature vectors and a feature vector database to obtain matching similarity. The specific calculation process satisfies the following formula: In the formula, x represents the paragraph feature vector, y represents the feature vector in the feature vector database, and similarity(x,y) represents the matching similarity. Based on preset matching principles and matching similarity, a matching mapping is performed to obtain the corresponding identity mapping label, thereby realizing voiceprint recognition.

8. A voiceprint recognition system based on deep speech embedding, characterized in that, include: The voice acquisition module is used to acquire voice data and obtain buffered waveform data based on the voice data; The data sampling module is used to resample the buffered waveform data at a preset sampling rate to obtain sampled speech data; The segmentation module is used to segment speech data based on sampled speech data and a preset window time to obtain segmented speech data. The data augmentation module is used to perform data augmentation processing on segmented speech data to obtain augmented speech data; The feature extraction module is used to extract features from the enhanced speech data based on the pre-trained model to obtain high-dimensional feature vectors. Specifically, it includes: introducing skip connections into the deep learning network model and pre-training the deep learning network model using a large-scale speech dataset to obtain a pre-trained model; and performing multi-layer convolution and pooling operations based on the pre-trained model and the enhanced speech data to obtain high-dimensional feature vectors. The feature aggregation module is used to calculate frame-level statistical information based on high-dimensional feature vectors, and to obtain segment feature vectors based on attention mechanisms and frame-level statistical information. Specifically, it includes: obtaining statistical information based on statistical pooling functions and high-dimensional feature vectors, wherein the statistical information includes the mean and standard deviation, and the specific calculation process satisfies the following formula: In the formula, Let N represent the high-dimensional feature vector of the i-th frame, and N represent the number of frames, with mean( std() represents the mean. () represents the standard deviation; The attention weights of the high-dimensional feature vector are obtained based on a deep neural network, and the specific calculation process satisfies the following formula: In the formula: W and b represent preset learning parameters, A( ) represents the attention weights of the high-dimensional feature vector of the i-th frame, and T represents the transpose. This represents the high-dimensional feature vector of the j-th frame; The paragraph feature vector is obtained by calculating a weighted average feature vector based on the attention pooling mechanism and attention weights. The specific calculation process satisfies the following formula: P( )= In the formula: P(x) represents the paragraph feature vector; The tag mapping module is used to perform feature vector matching based on paragraph feature vectors, thereby performing identity mapping and realizing voiceprint recognition.

Images