Voiceprint registration method and device based on deep voice embedding

Abstract

The invention discloses a voiceprint registration method and device based on deep voice embedding, and belongs to the technical field of voice biological recognition. The method comprises the following steps: acquiring voice data in a natural scene, storing the voice data in a local annular buffer area, resampling the voice data at a unified sampling rate, segmenting the resampled voice data by adopting a sliding window technology, performing spectrum feature extraction on the segmented voice data based on a filter bank, and extracting the spectrum feature of the segmented voice data. The method comprises the following steps: extracting a frequency spectrum feature of a user, performing cepstrum mean value and variance normalization processing on the extracted frequency spectrum feature, storing the processed frequency spectrum feature in a server, extracting the frequency spectrum feature based on a preset trained voiceprint model on the server to obtain a final feature vector, and storing the final feature vector in a MySQL database. According to the method, the robustness, the calculation efficiency and the safety of voiceprint registration in a noise environment are remarkably improved through a deep voice embedding and embedded optimization technology.

Description

Technical Field

[0001] This invention relates to the field of voice biometrics, specifically to a voiceprint registration method and apparatus based on deep voice embedding. Background Technology

[0002] Voiceprint recognition, as an important branch of biometric identification, has received widespread attention and application in recent years. Traditional voiceprint registration methods are mainly based on signal processing techniques, employing hand-designed feature extraction algorithms such as Mel-frequency cepstral coefficients (MFCC) and perceptual linear prediction (PLP). These methods extract spectral features of speech signals to build voiceprint models for identity verification. With the development of deep learning technology, voiceprint registration methods based on deep neural networks (DNN) and convolutional neural networks (CNN) have gradually become mainstream, such as Google's SpeakerNet and VoxCeleb projects, which extract more discriminative voiceprint features through deep speech embedding technology.

[0003] Despite the progress made in existing technologies, the following technical problems still exist in practical applications: 1. Insufficient environmental adaptability Traditional feature extraction methods (such as MFCC) are sensitive to environmental noise and exhibit poor feature stability in noisy environments. While deep learning methods have made some improvements, they still struggle to effectively handle complex acoustic environments in real-world applications due to the discrepancy between training data and the actual environment.

[0004] 2. High computational resource requirements and deployment difficulty Existing deep voiceprint models typically require significant computational resources and storage space, making it difficult to achieve efficient operation on resource-constrained embedded devices. Especially in edge computing scenarios such as mobile terminals and IoT devices, the computational complexity and power consumption of the models become major limiting factors.

[0005] 3. Insufficient data security and privacy protection As a type of biometric data, the security and privacy protection of voiceprints are of paramount importance. Current technologies primarily focus on recognition accuracy, neglecting the encrypted storage and secure transmission of voiceprint data, thus posing a risk of data leakage.

[0006] 4. The registration process is complex and the user experience is poor. Traditional voiceprint registration often requires repeated recordings and complex parameter settings, lacking a unified interface and automated processing flow, resulting in a poor user experience and hindering the promotion of the technology.

[0007] To address the aforementioned issues, there is an urgent need for a voiceprint registration method and apparatus based on deep speech embedding to resolve the problems associated with traditional methods. Summary of the Invention

[0008] The purpose of this invention is to provide a voiceprint registration method and apparatus based on deep speech embedding. Through deep speech embedding and embedded optimization technology, the robustness, computational efficiency and security of voiceprint registration in noisy environments are significantly improved.

[0009] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A speaker registration method based on deep speech embedding includes: Step 1: Collect speech data in natural scenes; Step 2: Store the voice data in a local circular buffer; specifically: Initialize the buffer array, read pointer, and write pointer; By determining the positional relationship between the write pointer and the read pointer, the voice data can be written and read in a loop. Step 3: Resample the speech data at a uniform sampling rate; Step 4: Use the sliding window technique to segment the resampled speech data; Step 5: Extract spectral features from the segmented speech data based on the filter bank, and perform cepstral mean and variance normalization on the extracted spectral features. Step 6: Store the processed spectral features on the server, extract the spectral features based on the pre-trained voiceprint model on the server to obtain the final feature vector, and store it in the MySQL database; the voiceprint model is a model converted from ONNX format to RKNN format and deployed on an embedded platform.

[0010] Furthermore, in step 1, voice data in natural scenes is collected, specifically as follows: The system uses a high-sensitivity microphone to collect users' voice data in natural settings, converting the collected analog voice signals into digital signals.

[0011] Furthermore, in step 2, the voice data is stored in a local circular buffer, specifically as follows: Initialize the buffer array, read pointer, and write pointer; By determining the positional relationship between the write pointer and the read pointer, the cyclic writing and reading of voice data can be achieved.

[0012] Furthermore, in step 3, the speech data is resampled at a uniform sampling rate, specifically as follows: Calculate the sampling rate conversion factor; When the target sampling rate is higher than the original sampling rate, linear interpolation is used for upsampling. When the target sampling rate is lower than the original sampling rate, low-pass filtering is performed first, followed by decimation and downsampling.

[0013] Furthermore, in step 4, the resampled speech data is segmented using a sliding window technique, specifically as follows: Set the window size W=25ms and the jump step H=10ms; Starting from the beginning of the speech data, audio data within each window is extracted sequentially to form an audio segment; Move the window forward by H samples and continue extracting the next audio segment; For windows with a length less than W at the end of the signal, zero padding is performed.

[0014] Furthermore, step 4 also includes: Data augmentation processing is performed on the segmented speech data, including noise injection, time-domain distortion, random pruning with zeroing, and room simulation.

[0015] The present invention also provides a speaker registration device based on deep speech embedding, applied to the above-mentioned speaker registration method based on deep speech embedding, comprising: The sound acquisition module is used to collect voice data; The buffer management module is used to implement circular buffer storage of voice data; The resampling module is used to convert the sampling rate of the speech data; The segmentation module is used to segment speech using a sliding window. The feature extraction module is used to extract deep speech embedding features; The storage module is used to store feature vectors in the database.

[0016] Furthermore, it also includes: The model deployment module is used to convert the voiceprint recognition model into a format supported by the embedded platform and deploy it. The communication interface module provides a unified web interface for users to register their voiceprints.

[0017] In summary, the present invention has at least one of the following beneficial technical effects: 1. In terms of environmental adaptability, by adopting deep speech embedding technology and data enhancement processing, the system can effectively suppress environmental noise interference and maintain high-precision voiceprint registration capability in noisy real-world scenarios.

[0018] 2. When faced with changes in the speaker's own voice, such as fluctuations in emotion and speech rate, thanks to the powerful representation capabilities of deep networks and the sliding window multi-time feature extraction mechanism, the system has better tolerance and stability for fluctuations in voiceprint features.

[0019] 3. In terms of computational efficiency and deployment, by optimizing the model to be lightweight, converting it to the dedicated RKNN format, and using a circular buffer to efficiently manage the data flow, real-time and low-power operation on embedded devices such as RK3576 has been successfully achieved.

[0020] 4. In terms of data security and privacy protection, the system stores irreversible deep feature vectors instead of the original speech, and combines encrypted transmission and storage technologies to reduce the risk of biometric data leakage from the source.

[0021] 5. In terms of user experience, by providing a unified web interface and a fully automated backend processing flow, the registration process is greatly simplified. Users only need to provide voice to complete the registration, realizing a convenient and efficient "one-click" experience. Attached Figure Description

[0022] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0024] like Figure 1 As shown, this invention provides a speaker registration method based on deep speech embedding, comprising: Step 1: Collect speech data in natural scenes; Step 2: Store the voice data in a local circular buffer; specifically: Initialize the buffer array, read pointer, and write pointer; By determining the positional relationship between the write pointer and the read pointer, the voice data can be written and read in a loop. Step 3: Resample the speech data at a uniform sampling rate; Step 4: Use the sliding window technique to segment the resampled speech data; Step 5: Extract spectral features from the segmented speech data based on the filter bank, and perform cepstral mean and variance normalization on the extracted spectral features. Step 6: Store the processed spectral features on the server, extract the spectral features based on the pre-trained voiceprint model on the server to obtain the final feature vector, and store it in the MySQL database; the voiceprint model is a model converted from ONNX format to RKNN format and deployed on an embedded platform.

[0025] In step 1, voice data is collected in natural scenes, specifically as follows: The system uses a high-sensitivity microphone to collect users' voice data in natural settings, converting the collected analog voice signals into digital signals.

[0026] In step 2, the voice data is stored in a local circular buffer, specifically as follows: Initialize an array of size N as a buffer, and initialize the read pointer read_ptr and write pointer write_ptr to 0; When writing data, the following are included: 1. Check if the buffer is full. The condition is: (write_ptr + 1) mod N == read_ptr; 2. If the buffer is not full, write the new data to buffer[write_ptr]; 3. Update the write pointer: write_ptr = (write_ptr + 1) mod N; When reading data, the following are included: 1. Check if the buffer is empty. The condition is: write_ptr == read_ptr; 2. If not empty, read data from buffer[read_ptr]; 3. Update the read pointer: read_ptr = (read_ptr + 1) mod N.

[0027] In step 3, the speech data is resampled at a uniform sampling rate, specifically as follows: Calculate the sampling rate conversion factor R ,for: ,in, f s The original sampling rate, f d The target sampling rate; Explanation based on different situations: 1. When the target sampling rate is higher than the original sampling rate, i.e. during upsampling: Linear interpolation is used to increase the number of sampling points. For adjacent sampling points x[n] and x[n+1], the interpolation point x[n+a] is calculated using the formula: x[n+a] = ⋅x[n]+a⋅x[n+1], where a is the interpolation factor, 0≤a<1.

[0028] 2. When the target sampling rate is lower than the original sampling rate, i.e., during downsampling: (1) First, use a cutoff frequency of min(f s / 2, f d The low-pass filter ( / 2) filters the signal to prevent aliasing, and the filtered output... y [ n ]for: ,in, It is the impulse response of the low-pass filter. x [ k ] is the original signal; (2) Extract the filtered signal to generate a resampled signal. z [ m ]: .

[0029] In step 4, the resampled speech data is segmented using a sliding window technique, specifically as follows: Set the window size W=25ms and the jump step H=10ms; Starting from the beginning of the speech data, audio data within each window is extracted sequentially to form an audio segment; Move the window forward by H samples and continue extracting the next audio segment; For windows with a length less than W at the end of the signal, zero padding is performed.

[0030] Step 4 also includes: To improve the robustness of the voiceprint model in complex environments, data augmentation is performed on the segmented speech data during the training phase or in scenarios with high requirements for registered speech quality. This simulates real speech variations and noisy environments. The augmentation operations are performed in the time or frequency domain and include noise injection, time-domain warping, random pruning with zeroing, and room simulation. These are described in detail below: 1. Noise injection: Background noise (such as white noise, conference room noise, street noise) is superimposed onto the clean audio segment with a random signal-to-noise ratio (SNR).

[0031] 2. Time-domain distortion: Randomly fine-tuning the speed or pitch of an audio signal to simulate the natural fluctuations in a speaker's speech rate or pitch.

[0032] 3. Random cropping and zeroing: Randomly zeroing a very short portion of an audio segment (simulating a short interruption), or randomly discarding the beginning and end of an audio segment (simulating an incomplete pronunciation).

[0033] 4. Room Simulation: By using convolution reverb, different types of room impulse responses (RIR) are added to the audio segments to simulate different recording environments.

[0034] In step 5, spectral features are extracted from the segmented speech data based on the filter bank, and the extracted spectral features are then normalized using cepstral mean and variance. Specifically: 1. Use the filter bank method to extract the spectral features of the audio. m Energy of each filter E m The calculation formula is: ,in, X k It is the frequency domain representation of the speech signal; and f m It is the first m The lower and upper limits of the triangular filter (the boundary frequency is set to 8kHz); m For filter indexing, the total number of filters is set to 26 in this embodiment.

[0035] 2. The extracted spectral features are normalized using cepstral mean and variance normalization to eliminate bias caused by differences in recording conditions. The normalization formula is: ,in, x These are the original eigenvalues; μ It is the mean of this feature across all time frames; σ It is the standard deviation of this feature across all time frames; C ( x ) are the normalized eigenvalues.

[0036] In step 6, the processed spectral features are stored on the server. Based on the pre-trained voiceprint model on the server, the spectral features are extracted to obtain the final feature vector, which is then stored in the MySQL database. Specifically: (1) The voiceprint recognition model trained on the high-performance server is converted from the ONNX (Open Neural Network Exchange) format to the RKNN (Rockchip Neural Network) format specifically for Rockchip chips; (2) Upload the converted RKNN model to the RK3576 board; (3) Configure the runtime environment on the board and load the RKNN model using the RKNN API; (4) Use the loaded model to perform forward inference on the audio segments obtained in step 4 and extract the deep speech embedding feature vector; (5) The final feature vector is persisted in the MySQL database in the form of a matrix, where each row of the matrix represents the feature vector of a time frame.

[0037] The present invention also provides a speaker registration device based on deep speech embedding, applied to the above-mentioned speaker registration method based on deep speech embedding, comprising: The sound acquisition module is used to collect voice data; The buffer management module is used to implement circular buffer storage of voice data; The resampling module is used to convert the sampling rate of the speech data; The segmentation module is used to segment speech using a sliding window. The feature extraction module is used to extract deep speech embedding features; The storage module is used to store feature vectors in the database; The model deployment module is used to convert the voiceprint recognition model into a format supported by the embedded platform and deploy it. The communication interface module provides a unified web interface for users to register their voiceprints.

[0038] Furthermore, the present invention also provides an embodiment, specifically as follows: 1. A double buffering mechanism is adopted to replace the circular buffer, specifically: (1) Set up two buffers A and B of the same size; (2) While buffer A is being written to, buffer B is used for reading. (3) The two buffers are used alternately by using flag bits; (4) Buffer switching condition: Automatic switching when the write buffer reaches the capacity threshold.

[0039] 2. Multiphase filtering resampling is used instead of linear interpolation, specifically as follows: (1) Design a polyphase filter bank to achieve sampling rate conversion; (2) Reduce computational complexity through multiphase decomposition; (3) Polynomial interpolation is used to improve resampling accuracy.

[0040] 3. Adaptive segmentation based on speech activity detection is used instead of voice segmentation, specifically as follows: (1) An energy-based endpoint detection algorithm is adopted; (2) Speech / non-speech discrimination is performed by combining short-time energy and zero-crossing rate; (3) Adjust the segmentation position and length dynamically based on the test results.

[0041] 4. Wavelet transform feature extraction is used instead of feature extraction, specifically: (1) Use wavelet packet decomposition to extract time-frequency features; (2) Select appropriate wavelet basis functions (such as db4 wavelet); (3) Extract the statistical features of wavelet coefficients as voiceprint features.

[0042] 5. Adopt TensorFlow Lite format deployment instead of model deployment, specifically: (1) Convert the trained model to TFLite format; (2) Implement hardware acceleration using TFLite Delegates; (3) Model reasoning is performed using TFLite Interpreter.

[0043] 6. Features are stored in a vector repository, specifically: (1) Use a dedicated vector database (such as Milvus, Faiss); (2) Achieve efficient similarity search and feature matching; (3) Supports large-scale voiceprint feature library management.

[0044] 7. Homomorphic encryption technology is used for security protection, specifically: (1) Use homomorphic encryption in the feature extraction stage; (2) Implement feature comparison under encrypted conditions; (3) Protect user privacy through secure multi-party computation.

[0045] Embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0046] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1A device that provides the functions specified in one or more boxes.

[0047] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0048] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0049] Contents not described in detail in this specification are prior art known to those skilled in the art. It is hereby indicated that the above description is intended to help those skilled in the art understand this invention, but does not limit the scope of protection of this invention. Any equivalent substitutions, modifications, improvements, or simplifications of the above descriptions that do not depart from the essential content of this invention fall within the scope of protection of this invention.

Claims

1. A speaker registration method based on deep speech embedding, characterized in that, include: Step 1: Collect speech data in natural scenes; Step 2: Store the voice data in a local circular buffer; specifically: Initialize the buffer array, read pointer, and write pointer; By determining the positional relationship between the write pointer and the read pointer, the voice data can be written and read in a loop. Step 3: Resample the speech data at a uniform sampling rate; Step 4: Use the sliding window technique to segment the resampled speech data; Step 5: Extract spectral features from the segmented speech data based on the filter bank, and perform cepstral mean and variance normalization on the extracted spectral features. Step 6: Store the processed spectral features on the server, extract the spectral features based on the pre-trained voiceprint model on the server, obtain the final feature vector, and store it in the MySQL database; The voiceprint model is a model converted from ONNX format to RKNN format and deployed on an embedded platform.

2. The speaker registration method based on deep speech embedding according to claim 1, characterized in that, In step 1, voice data is collected in natural scenes, specifically as follows: The system uses a high-sensitivity microphone to collect users' voice data in natural settings, converting the collected analog voice signals into digital signals.

3. The speaker registration method based on deep speech embedding according to claim 1, characterized in that, In step 3, the speech data is resampled at a uniform sampling rate, specifically as follows: Calculate the sampling rate conversion factor; When the target sampling rate is higher than the original sampling rate, linear interpolation is used for upsampling. When the target sampling rate is lower than the original sampling rate, low-pass filtering is performed first, followed by decimation and downsampling.

4. The speaker registration method based on deep speech embedding according to claim 3, characterized in that, In step 4, the resampled speech data is segmented using a sliding window technique, specifically as follows: Set the window size W=25ms and the jump step H=10ms; Starting from the beginning of the speech data, audio data within each window is extracted sequentially to form an audio segment; Move the window forward by H samples and continue extracting the next audio segment; For windows with a length less than W at the end of the signal, zero padding is performed.

5. The speaker registration method based on deep speech embedding according to claim 4, characterized in that, Step 4 also includes: Data augmentation processing is performed on the segmented speech data, including noise injection, time-domain distortion, random pruning with zeroing, and room simulation.

6. A voiceprint registration device based on deep speech embedding, applied to the voiceprint registration method based on deep speech embedding as described in any one of claims 1-5, characterized in that, include: The sound acquisition module is used to collect voice data; The buffer management module is used to implement circular buffer storage of voice data; The resampling module is used to convert the sampling rate of the speech data; The segmentation module is used to segment speech using a sliding window. The feature extraction module is used to extract deep speech embedding features; The storage module is used to store feature vectors in the database.

7. The apparatus according to claim 6, characterized in that, Also includes: The model deployment module is used to convert the voiceprint recognition model into a format supported by the embedded platform and deploy it. The communication interface module provides a unified web interface for users to register their voiceprints.

Images