Voiceprint matching method

Abstract

The invention discloses a voiceprint matching method, relates to the technical field of voiceprint recognition, and solves the technical problems that it is difficult to construct a voiceprint noise reduction model to carry out noise reduction processing on collected voiceprints under the environment condition with noise, and it is also difficult to construct a voiceprint feature fusion model library by integrating time domain and frequency domain features and carry out matching with voiceprints to be verified. Comprising the following steps: constructing a voiceprint noise reduction model, and carrying out preprocessing operation on a collected voiceprint sample; extracting short-time energy, short-time average amplitude and short-time zero-crossing rate to construct a fusion time domain feature value, and extracting an energy mean value, a variance and a proportion value to perform fusion weighting operation to obtain a fusion frequency domain feature value; inputting the obtained fusion time domain feature value and the fusion frequency domain feature value into a model to form a voiceprint feature fusion model library; and calculating the similarity between the to-be-verified voiceprint feature vector and each registered voiceprint feature vector in the voiceprint feature fusion model library.

Description

Technical Field

[0001] The present invention relates to the technical field of voiceprint recognition, and in particular to a voiceprint matching method. Background Art

[0002] With the rapid development of information technology, voiceprint recognition has been widely used in many fields as a convenient and efficient way of identity authentication, such as security monitoring, financial transactions, smart homes, etc. However, the accuracy and robustness of existing voiceprint matching methods in complex environments still need to be improved. For example, when there is background noise interference and differences in recording equipment, the accuracy of voiceprint matching may drop significantly, and it cannot meet the needs of high-precision identity authentication in practical applications. It is difficult to build a voiceprint denoising model in a noisy environment to denoise the collected voiceprints to generate clean voiceprint samples, and it is also difficult to extract multiple features in the time domain and frequency domain, and add them with the corresponding weight coefficients to obtain fused time domain eigenvalues ​​and fused frequency domain eigenvalues. There is a lack of using the fused time domain eigenvalues ​​and fused frequency domain eigenvalues ​​as feature vectors to construct a voiceprint feature fusion model library, and using cosine similarity to calculate the similarity between the registered voiceprint and the voiceprint to be verified to determine whether the voiceprint matches successfully. Summary of the invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a voiceprint matching method for solving the following technical problems: It is difficult to build a voiceprint denoising model in a noisy environment to denoise the collected voiceprints and generate clean voiceprint samples. It is also difficult to extract multiple features in the time domain and frequency domain, and add them with the corresponding weight coefficients to obtain fused time domain eigenvalues ​​and fused frequency domain eigenvalues. There is a lack of using the fused time domain eigenvalues ​​and fused frequency domain eigenvalues ​​as feature vectors to construct a voiceprint feature fusion model library, and use cosine similarity to calculate the similarity between the registered voiceprint and the voiceprint to be verified to determine whether the voiceprint matches successfully.

[0004] In order to achieve the above object, the present invention proposes a voiceprint matching method, which comprises the following steps: S1: Collect the voiceprint samples to be matched through the audio acquisition device, including the registered voiceprint samples and the voiceprint samples to be verified, pre-process and annotate the collected original noisy voiceprint samples, build the generator and discriminator, form the original noisy voiceprint samples and the corresponding generated voiceprint samples into training data pairs, build the voiceprint denoising model, and pre-process the collected voiceprint samples; S2: Extract short-time energy, short-time average amplitude and short-time zero-crossing rate from the preprocessed voiceprint samples, construct a weight vector to fuse with the time domain features to obtain the fused time domain feature value, extract the frequency domain features, and fuse the energy mean, variance and ratio value to obtain the fused frequency domain feature value; S3: Input the obtained fused time domain feature value and fused frequency domain feature value into the recurrent neural network model to form a voiceprint feature fusion model library; S4: Calculate the similarity between the voiceprint feature vector to be verified and each registered voiceprint feature vector in the voiceprint feature fusion model library, and determine whether the similarity is less than a preset similarity threshold.

[0005] Furthermore, the step S1 comprises the following steps: Under the conditions of different noise environments, the voiceprint samples of the user during registration are collected through an audio acquisition device, the collected voiceprint samples are converted into a unified audio format, and the registered voiceprint samples are input into the voiceprint noise reduction model for preprocessing; using the same audio acquisition device and under conditions similar to the registration voiceprint sample collection environment, the voiceprint samples to be verified are collected and converted into a unified audio format, and at the same time, the voiceprint samples to be verified are preprocessed accordingly.

[0006] Furthermore, the voiceprint noise reduction model includes the following steps: Collect voiceprint samples from different users in different noise environments, perform preprocessing operations on the voiceprint data, including sampling rate conversion, silence removal and pre-emphasis, convert it into a form suitable for model input, annotate the collected original noisy voiceprint data, annotate the noise type, noise level, speech content and user identity it contains, and annotate the source and corresponding original noisy voiceprint identifier for the voiceprint generated by the generator; A shared input layer is designed in the generator and the discriminator. The same preprocessing operations are performed on the input voiceprint, including Fourier transform and Mel-frequency cepstral coefficient extraction. After the voiceprint signal is converted into a unified feature representation, the features extracted from the noisy voiceprint signal are used as input to construct the generator. The discriminator adopts a dual-input structure, one input is the feature representation of the original noisy voiceprint, and the other input is the feature representation of the generated voiceprint. A comparison module is set inside the discriminator to compare and analyze the original noisy voiceprint features and the generated voiceprint features. The original noisy voiceprint samples and the corresponding generated voiceprint samples are combined into a training data pair and input into the discriminator at the same time. The model is trained using the training set and the parameters of the model are updated through multiple iterations so that the model can learn the mapping relationship from noisy voiceprints to clean voiceprints and calculate the root mean square error between the original voiceprint signal and the denoised voiceprint signal.

[0007] Furthermore, the calculating of the root mean square error between the original voiceprint signal and the voiceprint signal after noise reduction comprises the following steps: The original voiceprint signal is represented as , the voiceprint signal after noise reduction is expressed as , for each sampling point, calculate the square of the difference between the original voiceprint signal and the denoised voiceprint signal, add up all the square differences, divide the error sum by the total number of all sampling points, and take the square root of the mean square error to get the root mean square error; Root mean square error calculation formula:

[0008] in, represents the root mean square error, represents the total number of samples, Indicates The original voiceprint signal, Indicates A noise-reduced voiceprint signal; Based on the experience of historical voiceprint noise reduction projects, the RMSE empirical threshold is set: like If it is less than the RMSE empirical threshold, it means that the difference between the original voiceprint signal and the denoised voiceprint signal is small, and the denoising effect is good. The trained model is deployed in the system to denoise the collected voiceprint samples. like If it is greater than or equal to the RMSE empirical threshold, it means that the difference between the original voiceprint signal and the denoised voiceprint signal is large, the denoising effect is poor, and the model should be adjusted.

[0009] Furthermore, the extraction of short-time energy, short-time average amplitude and short-time zero-crossing rate, construction of a weight vector and fusion with the time domain feature to obtain a fused time domain feature value includes the following steps: Extract features from the preprocessed voiceprint samples. The time domain features include short-time energy, short-time average amplitude and short-time zero-crossing rate. Determine the frame length and frame shift, divide the voiceprint signal according to the determined frame length and frame shift, and calculate the short-time energy of the frame to obtain the corresponding short-time energy. For each frame signal, calculate the absolute value of each sampling point, add the absolute values ​​of all sampling points in the frame to obtain the sum of the amplitudes of the frame signal, and divide it by the frame length to obtain the short-time average amplitude of the frame signal. Set the zero-crossing threshold, initialize two timers, record the number of times the signal value changes from positive to negative and the difference is greater than the set threshold, and the number of times the signal value changes from negative to positive and the difference is greater than the preset threshold, traverse the samples in the frame, and after the traversal, add the values ​​of the two counters to obtain the short-time zero-crossing rate of the frame. Determine the weight coefficients of short-time energy, short-time average amplitude and short-time zero-crossing rate according to application requirements and data characteristics, and construct a weight vector ,For each frame, the short-time energy, short-time average amplitude and short-time obsolescence rate are weightedly fused according to the weight vector to obtain the fused time domain feature value; The calculation formula of fusion time domain eigenvalue is:

[0010] in, Indicates The fused time domain feature value of the frame, Indicates The short-term energy of the frame, Indicates The short-term average amplitude of the frame, Indicates The short-term zero-crossing rate of the frame, , and represents the corresponding weight coefficient; After weighted fusion operation on all frames, a fused time domain feature value sequence is obtained ,in, Indicates the number of frames after framing.

[0011] Furthermore, the extraction of frequency domain features, fusing and weighting the energy mean, variance and ratio value to obtain a fused frequency domain feature value, includes the following steps: After windowing each frame of signal, perform fast Fourier transform to calculate the amplitude and phase information of the signal at different frequencies. According to the result of fast Fourier transform, the power spectrum of each frame of signal is obtained by squaring the amplitude spectrum of the spectrum and dividing it by the duration. The frequency resolution is determined according to the set sampling frequency and signal length. The frequency resolution is multiplied by the frequency index to obtain the corresponding actual frequency. The power spectrum is plotted with the actual frequency as the horizontal axis and the corresponding power spectrum as the vertical axis. The energy mean and variance of each frequency component in the power spectrum are calculated and displayed in the power spectrum. Find the maximum power spectrum value and the minimum power spectrum value, subtract them from the mean value to get the corresponding difference, divide them to get the ratio value, and take the absolute value; The energy mean, variance and ratio values ​​are fused and weighted to obtain the fused frequency domain eigenvalue; The calculation formula of fusion frequency domain eigenvalue is:

[0012]

[0013] in, represents the power spectrum mean, represents the number of frequency components, Indicates The power spectrum value of the frequency component, represents the fused frequency domain eigenvalue, represents the maximum power spectrum value, represents the minimum power spectrum value, , and Represents the corresponding weight coefficient.

[0014] Furthermore, the step S3 comprises the following steps: Collect user's voiceprint sample data, annotate each sample, including user's name and speech content, input the collected sample data into the noise reduction voiceprint model for noise reduction, then perform frame division and window function processing, unify the number of sampling points and normalize the amplitude, extract features from the preprocessed data according to the selected fused time domain eigenvalues ​​and fused frequency domain eigenvalues, concatenate the extracted fused time domain eigenvalues ​​and fused frequency domain eigenvalues ​​into a feature vector, create a user table to store user's related information, and create a voiceprint sample table to store the sample's feature vector and corresponding annotation information, associate it with the user table through a foreign key, select a recurrent neural network as the framework for building the model, divide the annotated data into training set data and input it into the recurrent neural network model, train the model, and the model learns and recognizes the voiceprint features of different users, predict the data used for the test set with the trained model, and obtain the voiceprint feature prediction results of different users, store the feature vectors and corresponding prediction results of all users, and form a voiceprint feature fusion model library.

[0015] The extracted feature vector is stored together with the unique identifier of the registered user in the voiceprint feature fusion model library. If the voiceprint sample needs to be verified, the voiceprint sample to be verified is collected in the same way as in the registration stage, and preprocessed. The feature vector of the voiceprint to be verified is obtained by the same feature extraction and calculation steps as in the registration stage for the preprocessed voiceprint sample to be verified. The cosine similarity method is used to calculate the similarity between the feature vector of the voiceprint to be verified and the feature vector of each registered voiceprint in the voiceprint feature fusion model library; Determine a preset similarity threshold based on experience, and compare the calculated similarity with the preset similarity threshold: If the similarity is less than the preset similarity threshold, the voiceprint matching is determined to have failed; If the similarity is greater than or equal to the preset similarity threshold, the voiceprint match is determined to be successful.

[0016] Beneficial effects of the present invention: The present invention constructs a voiceprint denoising model to effectively identify the noisy voiceprint data and weaken these background noise components, so that the main voice of the voiceprint sample is more prominent and the sound quality is significantly improved. In addition, the characteristics of the voiceprint are often hidden in the voiceprint signal, and the presence of noise may cover or distort these characteristics. The denoising model reduces the interference of noise to make the characteristics of the voiceprint more obvious and prominent. It can automatically adjust the denoising parameters according to different environmental conditions, and effectively denoise the voiceprint samples in various environments, so that the model can show good performance in different actual environments. The present invention extracts corresponding features from the time domain and frequency domain respectively, and obtains fused time domain eigenvalues ​​and fused frequency domain eigenvalues ​​through processing and calculation, so as to make full use of the information of voiceprint signals in different domains, reflect the strength, rhythm and difference of sound, splice the two eigenvalues ​​into a feature vector, realize the effective fusion of multi-dimensional features, make up for the limitation of single domain feature description, select recurrent neural network as the framework for building the model, train the model, and the model can learn the voiceprint feature patterns of different users, store the feature vectors of all users and the corresponding prediction results, and form a voiceprint feature fusion model library. This matching process is more efficient and can give accurate recognition results in a short time. As time goes by and new data is added, the data in the model library can be used to continuously train and optimize the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of the method flow of the present invention. DETAILED DESCRIPTION

[0018] In order to better understand the technical content of the present invention, specific embodiments are given and described as follows in conjunction with the accompanying drawings.

[0019] Please refer to Figure 1 , Figure 1 The figure is a schematic flow chart of the method of the present invention. The present invention proposes a voiceprint matching method, comprising the following steps: S1: Collect the voiceprint samples to be matched through the audio acquisition device, including the registered voiceprint samples and the voiceprint samples to be verified, pre-process and annotate the collected original noisy voiceprint samples, build the generator and discriminator, form the original noisy voiceprint samples and the corresponding generated voiceprint samples into training data pairs, build the voiceprint denoising model, and pre-process the collected voiceprint samples; S2: Extract short-time energy, short-time average amplitude and short-time zero-crossing rate from the preprocessed voiceprint samples, construct a weight vector to fuse with the time domain features to obtain the fused time domain feature value, extract the frequency domain features, and fuse the energy mean, variance and ratio value to obtain the fused frequency domain feature value; S3: Input the obtained fused time domain feature value and fused frequency domain feature value into the recurrent neural network model to form a voiceprint feature fusion model library; S4: Calculate the similarity between the voiceprint feature vector to be verified and each registered voiceprint feature vector in the voiceprint feature fusion model library, and determine whether the similarity is less than a preset similarity threshold.

[0020] Specifically, measure the noise decibel of the registration environment, record the environmental noise level, and let the user convert the collected voiceprint sample audio files into a standard audio format in the selected registration environment. Use the same audio acquisition device as that used during registration to collect the voiceprint samples to be verified. The original noisy voiceprint samples and the corresponding generated voiceprint samples form a training data pair, input the training data pair into the discriminator, and use the training set to train the model. For each test sample, calculate the root mean square error between the original voiceprint signal and the denoised voiceprint signal; construct a weight vector, and for each frame, perform weighted fusion of the short-time energy, short-time average amplitude, and short-time zero-crossing rate according to the weight vector to obtain the fused time domain eigenvalues, and perform weighted fusion operations on the energy mean, variance, and ratio values ​​to obtain the fused frequency domain eigenvalues; the fused time of the same frame The domain eigenvalues ​​and the fused frequency domain eigenvalues ​​are spliced ​​into a feature vector in a certain order, and the feature vectors of all frames are combined to form a feature matrix of the entire voiceprint sample for model input. A part of the labeled data is divided as a training set, and the data of the training set is input into the recurrent neural network model. The trained model is used to predict the data used for the test set to obtain the voiceprint feature prediction results of different users; each calculated similarity value is compared with the preset similarity threshold. If all similarity values ​​are less than the preset similarity threshold, the voiceprint match is determined to have failed, that is, the voiceprint to be verified does not match any user in the registered voiceprint library. If at least one similarity value is greater than or equal to the preset similarity threshold, the voiceprint match is determined to be successful, and the registered user with the maximum similarity value is output as the matching result.

[0021] In one embodiment of the present invention, the step S1 comprises the following steps: Under the conditions of different noise environments, the voiceprint samples of the user during registration are collected through an audio acquisition device, the collected voiceprint samples are converted into a unified audio format, and the registered voiceprint samples are input into the voiceprint noise reduction model for preprocessing; using the same audio acquisition device and under conditions similar to the registration voiceprint sample collection environment, the voiceprint samples to be verified are collected and converted into a unified audio format, and at the same time, the voiceprint samples to be verified are preprocessed accordingly.

[0022] Specifically, measure the noise decibels of the registration environment, record the environmental noise level, let the user read the pre-prepared text content through the audio acquisition device in the selected registration environment, convert the collected voiceprint sample audio files into a standard audio format, and input the registered voiceprint samples in the same unified format into the pre-trained voiceprint noise reduction model, which can remove the interference of the registration environment noise to a certain extent; try to make the collection environment of the voiceprint samples to be verified similar to the noise conditions during registration, use the same audio acquisition device as during registration to collect the voiceprint samples to be verified, require the user to speak in the same way as during registration in the verification environment, read the same text content, collect the voiceprint samples to be verified multiple times, and the time and method of each collection are consistent with those during registration, and the collected audio files of the voiceprint samples to be verified are also uniformly converted into the previously determined audio format, and the voiceprint samples to be verified are preprocessed similarly to the registered voiceprint samples.

[0023] In one embodiment of the present invention, the voiceprint noise reduction model includes the following steps: Collect voiceprint samples from different users in different noise environments, perform preprocessing operations on the voiceprint data, including sampling rate conversion, silence removal and pre-emphasis, convert it into a form suitable for model input, annotate the collected original noisy voiceprint data, annotate the noise type, noise level, speech content and user identity it contains, and annotate the source and corresponding original noisy voiceprint identifier for the voiceprint generated by the generator; A shared input layer is designed in the generator and the discriminator. The same preprocessing operations are performed on the input voiceprint, including Fourier transform and Mel-frequency cepstral coefficient extraction. After the voiceprint signal is converted into a unified feature representation, the features extracted from the noisy voiceprint signal are used as input to construct the generator. The discriminator adopts a dual-input structure, one input is the feature representation of the original noisy voiceprint, and the other input is the feature representation of the generated voiceprint. A comparison module is set inside the discriminator to compare and analyze the original noisy voiceprint features and the generated voiceprint features. The original noisy voiceprint samples and the corresponding generated voiceprint samples are combined into a training data pair and input into the discriminator at the same time. The model is trained using the training set and the parameters of the model are updated through multiple iterations so that the model can learn the mapping relationship from noisy voiceprints to clean voiceprints and calculate the root mean square error between the original voiceprint signal and the denoised voiceprint signal.

[0024] Specifically, a large number of users are recruited and asked to use audio collection equipment to record their speech in different noise environments. For example, each user reads a fixed text content, and ensures that each user records multiple times in each noise environment to increase the diversity and quantity of data. The noise type is labeled according to the collection environment. For example, street noise includes vehicle driving sound, pedestrian footsteps, etc., and office noise includes computer fan sound, conversation sound, etc. Professional tools are used to measure and record the noise level of each sample in decibels, accurate to one decimal place, transcribe the user's speech in each sample, and label each sample with the unique identifier of the user who collected the sample; the voiceprint generated by the generator is processed The generated voiceprint is marked in the row, indicating which original noisy voiceprint samples the generator is based on, such as "generated by user A's sample in an office noise environment" and the unique identifier of the original noisy voiceprint sample corresponding to the generated voiceprint, so that it can be accurately matched and analyzed in subsequent processing; check the sampling rate of all voiceprint samples. If the sampling rate is inconsistent, use professional audio processing software to convert all samples to a unified sampling rate, set an energy threshold to detect the silent part in the audio, use the short-time energy analysis method to calculate the energy of each frame of audio, and compare it with the threshold, delete the detected silent part from the audio, and pre-emphasize the audio signal to improve High-frequency components; create a shared input layer in the model architecture, make sure that the data format of the input layer matches the preprocessed voiceprint data format, perform Fourier transform and Mel-frequency cepstral coefficient extraction on the data; the input of the generator is the noisy voiceprint features after preprocessing and feature extraction, learn the mapping relationship from noisy voiceprint features to clean voiceprint features, the output of the generator is the generated clean voiceprint representation, the discriminator adopts a dual-input structure, one input is the feature representation of the original noisy voiceprint, and the other input is the feature representation of the generated voiceprint. A comparison module is set inside the discriminator to compare and analyze the original noisy voiceprint features and the generated voiceprint features. The output of the discriminator is a scalar The value indicates whether a pair of input voiceprint features are from the same category, that is, whether the generated voiceprint is realistic enough and difficult to distinguish from the original noisy voiceprint. The original noisy voiceprint sample and the corresponding generated voiceprint sample form a training data pair, which is input into the discriminator. The model is trained using the training set, and the training process is repeated for multiple iterations. In the testing phase, a part of the preprocessed original noisy voiceprint samples is selected as the test set, and these samples are input into the trained generator to obtain the generated denoised voiceprint signal. At the same time, the corresponding original noisy voiceprint signal is prepared. For each test sample, the root mean square error between the original voiceprint signal and the denoised voiceprint signal is calculated.

[0025] In one embodiment of the present invention, the step of calculating the root mean square error between the original voiceprint signal and the denoised voiceprint signal comprises the following steps: The original voiceprint signal is represented as , the voiceprint signal after noise reduction is expressed as , for each sampling point, calculate the square of the difference between the original voiceprint signal and the denoised voiceprint signal, add up all the square differences, divide the error sum by the total number of all sampling points, and take the square root of the mean square error to get the root mean square error; Root mean square error calculation formula:

[0026] in, represents the root mean square error, represents the total number of samples, Indicates The original voiceprint signal, Indicates A noise-reduced voiceprint signal; Based on the experience of historical voiceprint noise reduction projects, the RMSE empirical threshold is set: like If it is less than the RMSE empirical threshold, it means that the difference between the original voiceprint signal and the denoised voiceprint signal is small, and the denoising effect is good. The trained model is deployed in the system to denoise the collected voiceprint samples. like If it is greater than or equal to the RMSE empirical threshold, it means that the difference between the original voiceprint signal and the denoised voiceprint signal is large, the denoising effect is poor, and the model should be adjusted.

[0027] Specifically, obtain the original voiceprint signal data set and the corresponding voiceprint signal data set after noise reduction processing, and calculate the difference between each pair of corresponding sampling points in the original voiceprint signal and the voiceprint signal after noise reduction. For example, for the i-th sampling point, the difference is the original voiceprint signal sampling point value - the voiceprint signal sampling point value after noise reduction. Square each difference to obtain the square of the difference of each sampling point, add the square of the difference of all sampling points to obtain the sum of squared errors, divide the sum of squared errors by the total number of sampling points to obtain the mean square error, take the square root of the mean square error to obtain the root mean square error; determine an RMSE empirical threshold based on the experience of historical voiceprint noise reduction projects. This threshold is obtained by analyzing the RMSE values ​​between the original voiceprint signal and the voiceprint signal after noise reduction in a large number of successful noise reduction cases in the past. It represents the acceptable range of signal differences in actual application scenarios; calculate the The RMSE value is compared with the set empirical threshold. If the RMSE is less than the RMSE empirical threshold, it means that the difference between the original voiceprint signal and the denoised voiceprint signal is small, and the denoising effect is good. The trained model is deployed in the system for denoising the newly collected voiceprint samples. Otherwise, it means that the difference between the original voiceprint signal and the denoised voiceprint signal is large, and the denoising effect is not good. The model is adjusted. After adjusting the model, repeat the above steps of calculating RMSE until the RMSE value is less than the empirical threshold to ensure that the model achieves a good denoising effect before deployment.

[0028] In one embodiment of the present invention, the extraction of short-time energy, short-time average amplitude and short-time zero-crossing rate, construction of a weight vector and fusion with the time domain feature to obtain a fused time domain feature value includes the following steps: Extract features from the preprocessed voiceprint samples. The time domain features include short-time energy, short-time average amplitude and short-time zero-crossing rate. Determine the frame length and frame shift, divide the voiceprint signal according to the determined frame length and frame shift, and calculate the short-time energy of the frame to obtain the corresponding short-time energy. For each frame signal, calculate the absolute value of each sampling point, add the absolute values ​​of all sampling points in the frame to obtain the sum of the amplitudes of the frame signal, and divide it by the frame length to obtain the short-time average amplitude of the frame signal. Set the zero-crossing threshold, initialize two timers, record the number of times the signal value changes from positive to negative and the difference is greater than the set threshold, and the number of times the signal value changes from negative to positive and the difference is greater than the preset threshold, traverse the samples in the frame, and after the traversal, add the values ​​of the two counters to obtain the short-time zero-crossing rate of the frame. Determine the weight coefficients of short-time energy, short-time average amplitude and short-time zero-crossing rate according to application requirements and data characteristics, and construct a weight vector ,For each frame, the short-time energy, short-time average amplitude and short-time obsolescence rate are weightedly fused according to the weight vector to obtain the fused time domain feature value; The calculation formula of fusion time domain eigenvalue is:

[0029] in, Indicates The fused time domain feature value of the frame, Indicates The short-term energy of the frame, Indicates The short-term average amplitude of the frame, Indicates The short-term zero-crossing rate of the frame, , and represents the corresponding weight coefficient; After weighted fusion operation on all frames, a fused time domain feature value sequence is obtained ,in, Indicates the number of frames after framing.

[0030] Specifically, the frame length is determined according to the characteristics of the voiceprint signal and application requirements. For example, if the voiceprint signal changes slowly, a longer frame length, such as 30ms - 50ms, may be selected; if the voiceprint signal changes quickly, a shorter frame length, such as 10ms - 20ms, may need to be selected. The frame length is usually expressed as L sampling points, and the calculation formula is L = frame length × sampling rate. Common frame shifts can be half or one-quarter of the frame length, etc. According to the determined frame length and frame shift, the preprocessed voiceprint sample signal s(n) is divided according to the frame length and the short-time energy is calculated; for the i-th frame signal s_i(n), the absolute value of each sampling point |s_i(n)| is calculated, and then the absolute values ​​of all sampling points in the frame are added to obtain the sum of the amplitudes of the frame signal, and the sum of the amplitudes is divided by the frame length to obtain the short-time average amplitude of the frame signal; according to the characteristics of the voiceprint signal and application requirements, a suitable zero-crossing threshold T is set. This threshold can be a small value close to zero, such as 0.01 - 0.1, initialize two counters C1 and C2, which are used to record the number of times the signal value changes from positive to negative and the difference is greater than the threshold and the number of times the signal value changes from negative to positive and the difference is greater than the threshold, traverse the samples in the frame, and determine whether to add 1 to counter C1 or C2 according to the set conditions. After traversing all sampling points of the i-th frame, add the values ​​of the two counters to obtain the short-time zero-crossing rate of the frame, construct a weight vector, and for each frame, perform weighted fusion of the short-time energy, short-time average amplitude, and short-time zero-crossing rate according to the weight vector to obtain the fused time domain feature value; in, The value of is 0.4, and The value of is 0.3.

[0031] In one embodiment of the present invention, the step of extracting frequency domain features and fusing and weighting the energy mean, variance and ratio value to obtain a fused frequency domain feature value includes the following steps: After windowing each frame of signal, perform fast Fourier transform to calculate the amplitude and phase information of the signal at different frequencies. According to the result of fast Fourier transform, the power spectrum of each frame of signal is obtained by squaring the amplitude spectrum of the spectrum and dividing it by the duration. The frequency resolution is determined according to the set sampling frequency and signal length. The frequency resolution is multiplied by the frequency index to obtain the corresponding actual frequency. The power spectrum is plotted with the actual frequency as the horizontal axis and the corresponding power spectrum as the vertical axis. The energy mean and variance of each frequency component in the power spectrum are calculated and displayed in the power spectrum. Find the maximum power spectrum value and the minimum power spectrum value, subtract them from the mean value to get the corresponding difference, divide them to get the ratio value, and take the absolute value; The energy mean, variance and ratio values ​​are fused and weighted to obtain the fused frequency domain eigenvalue; The calculation formula of fusion frequency domain eigenvalue is:

[0032]

[0033] in, represents the power spectrum mean, represents the number of frequency components, Indicates The power spectrum value of the frequency component, represents the fused frequency domain eigenvalue, represents the maximum power spectrum value, represents the minimum power spectrum value, , and Represents the corresponding weight coefficient.

[0034] Specifically, the signal is divided into several frames, and a window function is applied to each frame to reduce spectrum leakage. FFT is performed on each frame of the windowed signal to obtain its frequency domain representation, including amplitude and phase information. The amplitude spectrum of the signal is squared and then divided by the duration to estimate the power spectrum of the signal. The sampling frequency and frame length are determined. The frequency resolution is the sampling frequency divided by the frame length. The actual frequency of the kth frequency component is k multiplied by the corresponding frequency resolution. A power spectrum is drawn with the actual frequency as the horizontal coordinate and the power spectrum value as the vertical coordinate. The energy mean and variance of each frequency component in the power spectrum are calculated. The maximum and minimum power spectrum values ​​are found in the power spectrum, and the corresponding differences are subtracted from the mean to obtain the proportional value. The absolute value of the proportional value is taken as a whole, and the corresponding weights are set to perform a fusion weighted operation with the energy mean, variance and proportional value to obtain the fused frequency domain eigenvalue. in, The value of is 0.5, The value of is 0.3, The value of is 0.2.

[0035] In one embodiment of the present invention, step S3 comprises the following steps: Collect user's voiceprint sample data, annotate each sample, including user's name and speech content, input the collected sample data into the noise reduction voiceprint model for noise reduction, then perform frame division and window function processing, unify the number of sampling points and normalize the amplitude, extract features from the preprocessed data according to the selected fused time domain eigenvalues ​​and fused frequency domain eigenvalues, concatenate the extracted fused time domain eigenvalues ​​and fused frequency domain eigenvalues ​​into a feature vector, create a user table to store user's related information, and create a voiceprint sample table to store the sample's feature vector and corresponding annotation information, associate it with the user table through a foreign key, select a recurrent neural network as the framework for building the model, divide the annotated data into training set data and input it into the recurrent neural network model, train the model, and the model learns and recognizes the voiceprint features of different users, predict the data used for the test set with the trained model, and obtain the voiceprint feature prediction results of different users, store the feature vectors and corresponding prediction results of all users, and form a voiceprint feature fusion model library.

[0036] Specifically, the user's name and speech content are annotated for each voiceprint sample. The annotated information should be accurate and the annotated format should be unified. The collected voiceprint sample data is input into the noise reduction voiceprint model. Its purpose is to remove the noise components in the audio and improve the quality of the voiceprint data. The denoised voiceprint data is framed. The continuous audio signal is divided into multiple short frames according to a certain frame length and frame shift. Each frame of data is processed by a window function so that the signal gradually decays to zero at both ends to avoid spectrum leakage. The number of sampling points of all frames is unified to a fixed value, which can be achieved by interpolation or truncation. The amplitude of the voiceprint data is normalized, and the amplitude value of the signal is mapped to [-1, 1] or [0, 1] to avoid the influence of voiceprint data of different volumes on model training. According to the needs of voiceprint recognition and data characteristics, appropriate time domain feature values ​​are selected, including short-time energy, short-time average amplitude difference and zero-crossing rate. These features can reflect the energy change, frequency change and time span information of the voiceprint signal in the time domain. At the same time, the selected frequency domain feature values ​​include energy mean, variance and maximum and minimum ratio values, which can better capture the frequency characteristics of the voiceprint signal. For each preprocessed voiceprint frame, the selected time domain feature value and frequency domain feature value are calculated to obtain the feature vector of the frame. The fused time domain feature value and fused frequency domain feature value of the same frame are spliced ​​into a feature vector in a certain order as the final feature representation of the frame. The feature vectors of all frames are combined to form a feature matrix of the entire voiceprint sample for subsequent model input. The extracted features and annotation information are stored in the designed storage format. A part of the annotated data is divided as a training set, and the data of the training set is input into the recurrent neural network model. Appropriate training parameters are set, and the trained model is used to predict the data used for the test set to obtain the voiceprint feature prediction results of different users. During the prediction process, the input data needs to be subjected to the same preprocessing and feature extraction operations, and then the feature vector is input into the model to obtain the prediction result of the model output. The feature vectors and corresponding prediction results of all users are stored to form a voiceprint feature fusion model library.

[0037] In one embodiment of the present invention, step S4 comprises the following steps: The extracted feature vector is stored together with the unique identifier of the registered user in the voiceprint feature fusion model library. If the voiceprint sample needs to be verified, the voiceprint sample to be verified is collected in the same way as in the registration stage, and preprocessed. The feature vector of the voiceprint to be verified is obtained by the same feature extraction and calculation steps as in the registration stage for the preprocessed voiceprint sample to be verified. The cosine similarity method is used to calculate the similarity between the feature vector of the voiceprint to be verified and the feature vector of each registered voiceprint in the voiceprint feature fusion model library; Determine a preset similarity threshold based on experience, and compare the calculated similarity with the preset similarity threshold: If the similarity is less than the preset similarity threshold, the voiceprint matching is determined to have failed; If the similarity is greater than or equal to the preset similarity threshold, the voiceprint match is determined to be successful.

[0038] Specifically, according to the feature extraction steps described above, the voiceprint samples of each registered user are processed to finally obtain the feature vector of each registered user, and a unique identifier, such as a user ID, is generated for each registered user. The feature vector is stored together with the unique identifier in the voiceprint feature fusion model library. At the same time, other relevant information of the registered user, such as name, contact information, etc., is recorded in the model library. When it is necessary to verify the voiceprint sample, the same recording equipment and environmental conditions as those in the registration stage are used to collect the voiceprint sample to be verified, and the collected voiceprint sample to be verified is preprocessed, including noise reduction by the noise reduction voiceprint model, framing, window function processing, unification of sampling points and amplitude normalization. Feature extraction is performed on the preprocessed voiceprint sample to be verified, and the voiceprint to be verified is obtained by calculation according to the fused time domain eigenvalues ​​and fused frequency domain eigenvalues ​​selected in the registration stage. The feature vector of the voiceprint to be verified shall have the same dimension and type as the feature vector in the registration phase, so as to perform similarity calculation. The cosine similarity method is used to calculate the similarity between the feature vector of the voiceprint to be verified and each registered voiceprint feature vector in the voiceprint feature fusion model library. The preset similarity threshold is determined based on experience. The selection of the threshold needs to consider factors such as the security requirements of the system, the false recognition rate and the rejection rate. Usually, a suitable value is determined through experiments and data analysis. Each calculated similarity value is compared with the preset similarity threshold. If all similarity values ​​are less than the preset similarity threshold, the voiceprint matching is determined to have failed, that is, the voiceprint to be verified does not match any user in the registered voiceprint library. If at least one similarity value is greater than or equal to the preset similarity threshold, the voiceprint matching is determined to be successful, and the registered user with the maximum similarity value is output as the matching result.

[0039] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A voiceprint matching method, characterized in that: The following steps are involved: S1: Collect the voiceprint samples to be matched through the audio acquisition device, including the registered voiceprint samples and the voiceprint samples to be verified, pre-process and annotate the collected original noisy voiceprint samples, build the generator and discriminator, form the original noisy voiceprint samples and the corresponding generated voiceprint samples into training data pairs, build the voiceprint denoising model, and pre-process the collected voiceprint samples; S2: Extract short-time energy, short-time average amplitude and short-time zero-crossing rate from the preprocessed voiceprint samples, construct a weight vector to fuse with the time domain features to obtain the fused time domain feature value, extract the frequency domain features, and fuse the energy mean, variance and ratio value to obtain the fused frequency domain feature value; S3: Input the obtained fused time domain feature value and fused frequency domain feature value into the recurrent neural network model to form a voiceprint feature fusion model library; S4: Calculate the similarity between the voiceprint feature vector to be verified and each registered voiceprint feature vector in the voiceprint feature fusion model library, and determine whether the similarity is less than a preset similarity threshold.

2. A voiceprint matching method according to claim 1, characterized in that: The step S1 comprises the following steps: Under the conditions of different noise environments, the voiceprint samples of the user during registration are collected through an audio acquisition device, the collected voiceprint samples are converted into a unified audio format, and the registered voiceprint samples are input into the voiceprint noise reduction model for preprocessing; using the same audio acquisition device and under conditions similar to the registration voiceprint sample collection environment, the voiceprint samples to be verified are collected and converted into a unified audio format, and at the same time, the voiceprint samples to be verified are preprocessed accordingly.

3. A voiceprint matching method according to claim 2, characterized in that: The voiceprint noise reduction model includes the following steps: Collect voiceprint samples from different users in different noise environments, perform preprocessing operations on the voiceprint data, including sampling rate conversion, silence removal and pre-emphasis, convert it into a form suitable for model input, annotate the collected original noisy voiceprint data, annotate the noise type, noise level, speech content and user identity it contains, and annotate the source and corresponding original noisy voiceprint identification for the voiceprint generated by the generator; A shared input layer is designed in the generator and the discriminator. The same preprocessing operations are performed on the input voiceprint, including Fourier transform and Mel-frequency cepstral coefficient extraction. After the voiceprint signal is converted into a unified feature representation, the features extracted from the noisy voiceprint signal are used as input to construct the generator. The discriminator adopts a dual-input structure, one input is the feature representation of the original noisy voiceprint, and the other input is the feature representation of the generated voiceprint. A comparison module is set inside the discriminator to compare and analyze the original noisy voiceprint features and the generated voiceprint features. The original noisy voiceprint samples and the corresponding generated voiceprint samples are combined into a training data pair and input into the discriminator at the same time. The model is trained using the training set and the parameters of the model are updated through multiple iterations so that the model can learn the mapping relationship from noisy voiceprints to clean voiceprints and calculate the root mean square error between the original voiceprint signal and the denoised voiceprint signal.

4. A voiceprint matching method according to claim 3, characterized in that: The step of calculating the root mean square error between the original voiceprint signal and the noise-reduced voiceprint signal comprises the following steps: The original voiceprint signal is represented as , the voiceprint signal after noise reduction is expressed as , for each sampling point, calculate the square of the difference between the original voiceprint signal and the denoised voiceprint signal, add up all the square differences, divide the error sum by the total number of all sampling points, and take the square root of the mean square error to get the root mean square error; Root mean square error calculation formula: ； in, represents the root mean square error, represents the total number of samples, Indicates The original voiceprint signal, Indicates A noise-reduced voiceprint signal; Based on the experience of historical voiceprint noise reduction projects, the RMSE empirical threshold is set: like If it is less than the RMSE empirical threshold, it means that the difference between the original voiceprint signal and the denoised voiceprint signal is small, and the denoising effect is good. The trained model is deployed in the system to denoise the collected voiceprint samples. like If it is greater than or equal to the RMSE empirical threshold, it means that the difference between the original voiceprint signal and the denoised voiceprint signal is large, the denoising effect is poor, and the model should be adjusted.

5. A voiceprint matching method according to claim 1, characterized in that: The method of extracting short-time energy, short-time average amplitude and short-time zero-crossing rate, constructing a weight vector and fusing it with the time domain feature to obtain a fused time domain feature value includes the following steps: Extract features from the preprocessed voiceprint samples. The time domain features include short-time energy, short-time average amplitude and short-time zero-crossing rate. Determine the frame length and frame shift, divide the voiceprint signal according to the determined frame length and frame shift, and calculate the short-time energy of the frame to obtain the corresponding short-time energy. For each frame signal, calculate the absolute value of each sampling point, add the absolute values ​​of all sampling points in the frame to obtain the sum of the amplitudes of the frame signal, and divide it by the frame length to obtain the short-time average amplitude of the frame signal. Set the zero-crossing threshold, initialize two timers, record the number of times the signal value changes from positive to negative and the difference is greater than the set threshold, and the number of times the signal value changes from negative to positive and the difference is greater than the preset threshold, traverse the samples in the frame, and after the traversal, add the values ​​of the two counters to obtain the short-time zero-crossing rate of the frame. Determine the weight coefficients of short-time energy, short-time average amplitude and short-time zero-crossing rate according to application requirements and data characteristics, and construct a weight vector ,For each frame, the short-time energy, short-time average amplitude and short-time obsolescence rate are weightedly fused according to the weight vector to obtain the fused time domain feature value; The calculation formula of fusion time domain eigenvalue is: ； in, Indicates The fused time domain feature value of the frame, Indicates The short-term energy of the frame, Indicates The short-term average amplitude of the frame, Indicates The short-term zero-crossing rate of the frame, , and represents the corresponding weight coefficient; After weighted fusion operation on all frames, a fused time domain feature value sequence is obtained ,in, Indicates the number of frames after framing.

6. A voiceprint matching method according to claim 1, characterized in that: The method of extracting frequency domain features and fusing and weighting the energy mean, variance and ratio value to obtain a fused frequency domain feature value includes the following steps: After windowing each frame of signal, perform fast Fourier transform to calculate the amplitude and phase information of the signal at different frequencies. According to the result of fast Fourier transform, the power spectrum of each frame of signal is obtained by squaring the amplitude spectrum of the spectrum and dividing it by the duration. The frequency resolution is determined according to the set sampling frequency and signal length. The frequency resolution is multiplied by the frequency index to obtain the corresponding actual frequency. The power spectrum is plotted with the actual frequency as the horizontal axis and the corresponding power spectrum as the vertical axis. The energy mean and variance of each frequency component in the power spectrum are calculated and displayed in the power spectrum. Find the maximum power spectrum value and the minimum power spectrum value, subtract them from the mean value to get the corresponding difference, divide them to get the ratio value, and take the absolute value; The energy mean, variance and ratio values ​​are fused and weighted to obtain the fused frequency domain eigenvalue; The calculation formula of fusion frequency domain eigenvalue is: ； ； in, represents the power spectrum mean, represents the number of frequency components, Indicates The power spectrum value of the frequency component, represents the fused frequency domain eigenvalue, represents the maximum power spectrum value, represents the minimum power spectrum value, , and Represents the corresponding weight coefficient.

7. A voiceprint matching method according to claim 1, characterized in that: The step S3 comprises the following steps: Collect user's voiceprint sample data, annotate each sample, including user's name and speech content, input the collected sample data into the noise reduction voiceprint model for noise reduction, then perform frame division and window function processing, unify the number of sampling points and normalize the amplitude, extract features from the preprocessed data according to the selected fused time domain eigenvalues ​​and fused frequency domain eigenvalues, concatenate the extracted fused time domain eigenvalues ​​and fused frequency domain eigenvalues ​​into a feature vector, create a user table to store user's related information, and create a voiceprint sample table to store the sample's feature vector and corresponding annotation information, associate it with the user table through a foreign key, select a recurrent neural network as the framework for building the model, divide the annotated data into training set data and input it into the recurrent neural network model, train the model, and the model learns and recognizes the voiceprint features of different users, predict the data used for the test set with the trained model, and obtain the voiceprint feature prediction results of different users, store the feature vectors and corresponding prediction results of all users, and form a voiceprint feature fusion model library.

8. A voiceprint matching method according to claim 1, characterized in that: The step S4 comprises the following steps: The extracted feature vector is stored together with the unique identifier of the registered user in the voiceprint feature fusion model library. If the voiceprint sample needs to be verified, the voiceprint sample to be verified is collected in the same way as in the registration stage, and preprocessed. The feature vector of the voiceprint to be verified is obtained by the same feature extraction and calculation steps as in the registration stage for the preprocessed voiceprint sample to be verified. The cosine similarity method is used to calculate the similarity between the feature vector of the voiceprint to be verified and the feature vector of each registered voiceprint in the voiceprint feature fusion model library; Determine a preset similarity threshold based on experience, and compare the calculated similarity with the preset similarity threshold: If the similarity is less than the preset similarity threshold, the voiceprint matching is determined to have failed; If the similarity is greater than or equal to the preset similarity threshold, the voiceprint match is determined to be successful.

Images