Biological feature recognition method, system and equipment and medium

Abstract

The invention relates to a biological feature recognition method, system and device and a medium. The method comprises the following steps: acquiring voice signal data and ear canal echo signal data; extracting a voiceprint feature vector of the voice signal data and an earprint feature vector of the ear canal echo signal data; performing noise interference intensity evaluation based on the voice signal data to generate a voiceprint quality score; dynamically calculating a voiceprint fusion weight and an earprint fusion weight based on the voiceprint quality score; performing weighted fusion on the voiceprint feature vector and the earprint feature vector by using the voiceprint fusion weight and the earprint fusion weight to generate an environment robust joint feature vector; and performing identity recognition based on the environment robust joint feature vector to obtain an identity recognition result. By adopting the method, the feature fusion weight can be dynamically adjusted according to the environmental noise, and the accuracy and robustness of an identity recognition system in a noise change environment are improved.

Description

Technical Field

[0001] This invention belongs to the field of identity recognition technology, and in particular relates to a biometric recognition method, system, device and medium. Background Technology

[0002] With the rapid development of wearable smart devices and mobile security authentication technologies, identity recognition methods based on multimodal biometric fusion have attracted widespread attention due to their higher security and robustness. Among them, the recognition technology that combines voiceprint (voice features) and earprint (ear canal echo features) is particularly prominent. Voiceprint acquisition is convenient but easily affected by environmental noise, while earprint acquisition is carried out in the relatively enclosed ear canal, which has a natural shielding effect against external noise. The two have good complementarity at the theoretical level. Therefore, how to effectively fuse the features of these two modalities to build a stable and reliable identity recognition system has become a research hotspot in this field.

[0003] In existing technical solutions, a common approach is to extract voiceprint and earprint features, then employ a fixed fusion strategy, such as direct concatenation or weighted combination with fixed weights. The fused features are then input into subsequent classification or recognition models for identity determination. This fixed fusion framework achieves good recognition results in laboratory environments or noise-stable scenarios.

[0004] However, environmental noise in real-world applications is dynamic. Fixed-weight fusion strategies cannot adapt to this change: in quiet environments, voiceprint features are of high quality and have strong discriminative power, and should be the dominant factor in recognition; while in noisy environments, voiceprint features are severely contaminated, and their reliability decreases significantly. If the fusion weights set in quiet environments are still used, that is, if voiceprint features are still given high weights, the contaminated feature information will occupy an inappropriate proportion after fusion, thereby interfering with the overall recognition decision and ultimately causing the system's recognition accuracy to fluctuate significantly with the changes in environmental noise. Summary of the Invention

[0005] Therefore, it is necessary to provide a biometric identification method, system, device, and medium that can adapt to dynamic changes in environmental noise in order to address the above-mentioned technical problems.

[0006] In a first aspect, this application provides a biometric identification method, including:

[0007] S1. Acquire speech signal data and ear canal echo signal data;

[0008] S2. Extract the voiceprint feature vector from the speech signal data and the earprint feature vector from the ear canal echo signal data to obtain the voiceprint feature vector and the earprint feature vector.

[0009] S3. Based on the speech signal data, evaluate the noise interference intensity, obtain the evaluation results, and generate a voiceprint quality score based on the evaluation results;

[0010] S4. Dynamically calculate and generate voiceprint fusion weights and earprint fusion weights based on voiceprint quality scores;

[0011] S5. By using voiceprint fusion weights and earprint fusion weights, the voiceprint feature vector and earprint feature vector are weighted and fused to generate an environment-robust joint feature vector.

[0012] S6. Based on the environmental robust joint feature vector, perform identity recognition to obtain the identity recognition result.

[0013] In one embodiment, S3 includes:

[0014] S31. Perform frame segmentation on the speech signal data to obtain multiple speech frame data;

[0015] S32. Calculate the short-time energy of each speech frame data to obtain multiple frame energy values;

[0016] S33. Based on a preset energy threshold, identify the energy value of the noisy frame from multiple frame energy values;

[0017] S34. Calculate the average value of the noise frame energy to obtain the noise energy estimate;

[0018] S35. Calculate the signal-to-noise ratio estimate based on the noise energy estimate and the overall energy of the speech signal data to obtain the signal-to-noise ratio estimate;

[0019] S36. Map the signal-to-noise ratio estimate to a predefined scoring interval to obtain the voiceprint quality score.

[0020] In one embodiment, the energy value of each frame is compared with a preset energy threshold, and the energy values ​​of frames that are less than the preset energy threshold are determined to be noise frame energy values.

[0021] In one embodiment, S4 includes:

[0022] S41. Based on the voiceprint quality score, the voiceprint fusion weight is calculated using a preset weight mapping function to obtain the voiceprint fusion weight; the formula for calculating the voiceprint fusion weight is:

[0023]

[0024] in For voiceprint fusion weights, Score the quality of your voiceprint. and These are predefined function parameters;

[0025] S42. Calculate the difference between value 1 and voiceprint fusion weight to obtain the earprint fusion weight.

[0026] In one embodiment, function parameters and The value of is configured as follows: in response to the voiceprint quality score representing high-quality speech, the voiceprint fusion weight output approaches the first preset weight value; in response to the voiceprint quality score representing low-quality speech, the voiceprint fusion weight output approaches the second preset weight value, and the first preset weight is greater than the second preset weight.

[0027] In one embodiment, S5 includes:

[0028] S51. Perform the first normalization on the voiceprint feature vector to obtain the first normalized vector;

[0029] S52. Calculate the square root of the voiceprint fusion weights to obtain the first scaling factor;

[0030] S53. Multiply each element of the first normalized vector by the first scaling factor to obtain the weighted voiceprint feature vector.

[0031] S54. Perform a second normalization on the earprint feature vector to obtain the second normalized vector;

[0032] S55. Calculate the square root of the earprint fusion weight to obtain the second scaling factor;

[0033] S56. Multiply each element of the second normalized vector by the second scaling factor to obtain the weighted earprint feature vector.

[0034] S57. By concatenating the weighted voiceprint feature vector and the weighted earprint feature vector, an environmentally robust joint feature vector is obtained.

[0035] In one embodiment, the first normalization and the second normalization are L2 norm normalizations.

[0036] Secondly, this application also provides a biometric identification system, comprising:

[0037] The data acquisition module is used to acquire voice signal data and ear canal echo signal data;

[0038] The feature extraction module is used to extract the voiceprint feature vector from the speech signal data and the earprint feature vector from the ear canal echo signal data, thus obtaining the voiceprint feature vector and the earprint feature vector.

[0039] The noise assessment module is used to assess the intensity of noise interference based on speech signal data, obtain assessment results, and generate a voiceprint quality score based on the assessment results.

[0040] The weight calculation module is used to dynamically calculate and generate voiceprint fusion weights and earprint fusion weights based on the voiceprint quality score.

[0041] The feature fusion module is used to perform weighted fusion of the voiceprint feature vector and the earprint feature vector through voiceprint fusion weight and earprint fusion weight to generate an environment-robust joint feature vector.

[0042] The identity recognition module is used to perform identity recognition based on the environment robust joint feature vector and obtain the identity recognition result.

[0043] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement a biometric identification method as described in the first aspect.

[0044] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a biometric identification method as described in the first aspect.

[0045] The aforementioned biometric identification method, system, device, and medium assess the intensity of noise interference on the speech signal in real time and generate a corresponding voiceprint quality score. Based on this score, appropriate fusion weights are dynamically assigned to voiceprint and earprint features. These weights are then used to adaptively weight and fuse the two feature vectors, thereby constructing a joint feature representation that can resist changes in environmental noise. Finally, identity recognition is completed based on this robust feature. This method enables the identity recognition system to intelligently adjust its dependence on different modal features according to the level of environmental noise, thus significantly improving the accuracy and stability of recognition in real-world scenarios with dynamically changing noise. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 A flowchart illustrating a biometric identification method provided by the present invention;

[0048] Figure 2 This is a schematic diagram of the structure of a biometric identification system provided by the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0050] In one embodiment, such as Figure 1 As shown, a biometric identification method is provided. This embodiment illustrates the application of this method to a terminal. It is understood that this method can also be applied to a server, and further to a system including both a terminal and a server, and is implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps S1 to S6:

[0051] S1. Acquire speech signal data and ear canal echo signal data.

[0052] Optionally, speech signal data and ear canal echo signal data are acquired simultaneously using a device with audio acquisition capabilities. The speech signal data is the audio stream generated by the user's natural vocalization, and the ear canal echo signal data is the echo audio stream formed by reflection after a preset excitation signal (such as Gaussian white noise) is played into the ear canal. During the acquisition process, a uniform sampling frequency and bit depth are used to ensure that the time axes of the two types of signals are aligned. The sampling frequency preferably follows the standard specifications commonly used in the field of audio processing, and the bit depth setting meets the signal amplitude accuracy requirements. After acquisition, the two types of signals are stored separately in standardized audio file formats.

[0053] S2. Extract the voiceprint feature vector from the speech signal data and the earprint feature vector from the ear canal echo signal data to obtain the voiceprint feature vector and the earprint feature vector.

[0054] Optionally, Mel Frequency Cepstral Coefficients (MFCCs) are used as the core method for feature extraction, performing feature extraction on both speech signal data and ear canal echo signal data. Specifically, the two types of signals are first subjected to pre-emphasis, framing, windowing, Fast Fourier Transform (FFT), Mel filter bank filtering, logarithmic operation, and Discrete Cosine Transform (DCT) operations in sequence. For voiceprint feature extraction of speech signal data, a traditional Mel filter bank adapted to the low-frequency energy concentration characteristics of speech is used; for earprint feature extraction of ear canal echo signal data, an improved Mel filter bank that fits the significant high-frequency differences in earprint signals is used. Finally, fixed-dimensional voiceprint feature vectors and earprint feature vectors are obtained through feature dimensionality reduction.

[0055] S3. Based on the speech signal data, evaluate the noise interference intensity, obtain the evaluation results, and generate a voiceprint quality score based on the evaluation results.

[0056] Optionally, noise interference intensity is assessed based on two key indicators: short-time energy and spectral flatness of the speech signal data. Short-time energy is obtained by calculating the sum of squares of the amplitudes within each frame after the signal is segmented into frames, and spectral flatness is obtained by calculating the ratio of the geometric mean to the arithmetic mean of the signal frame spectrum. These two indicators reflect the degree of noise interference from the perspectives of energy distribution and spectral shape, respectively. The short-time energy and spectral flatness are mapped to normalized values ​​in the [0,1] interval, and a weighted summation is used to obtain the comprehensive evaluation result. The weights are pre-set based on the differences in noise sensitivity between the two indicators. The voiceprint quality score is obtained by inverse mapping from the comprehensive evaluation result; that is, the lower the noise interference intensity, the higher the voiceprint quality score. The score range is also limited to the [0,1] interval, intuitively reflecting the reliability of the voiceprint features.

[0057] S4. Based on the voiceprint quality score, dynamically calculate and generate voiceprint fusion weights and earprint fusion weights.

[0058] Optionally, a dynamic weight calculation rule is constructed based on the voiceprint quality score. This rule follows the logic that voiceprint quality is positively correlated with voiceprint fusion weight and negatively correlated with earprint fusion weight. Specifically, the voiceprint fusion weight is directly adopted as the linear mapping result of the voiceprint quality score, and the earprint fusion weight is obtained by subtracting the voiceprint fusion weight from 1, ensuring that the sum of the two weights is always 1, satisfying the mathematical constraints of weighted fusion. Furthermore, to avoid fusion imbalance caused by excessively low or high voiceprint fusion weights under extreme noise environments, a reasonable value range can be set for the voiceprint fusion weights. The upper and lower limits of the value range are determined based on the stability requirements of feature fusion. The boundary constraints of the weights are implemented through the clip function, ensuring that the weight allocation conforms to the real-time environmental state and guarantees the effectiveness of the fused features.

[0059] S5. By using voiceprint fusion weights and earprint fusion weights, the voiceprint feature vector and earprint feature vector are weighted and fused to generate an environment-robust joint feature vector.

[0060] Optionally, the voiceprint feature vector and earprint feature vector are fused using an element-wise weighted summation method. Specifically, each element of the voiceprint feature vector is multiplied by its corresponding voiceprint fusion weight, and each element of the earprint feature vector is multiplied by its corresponding earprint fusion weight. The two weighted vectors are then added together along their respective dimensions to obtain the environment-robust joint feature vector. The dimensionality of the feature vectors remains unchanged during the fusion process. Both element-wise multiplication and addition operations follow the basic rules of vector operations, maintaining numerical precision to avoid distortion of feature information due to computational errors. The resulting environment-robust joint feature vector combines the convenience of voiceprint features with the stability of earprint features.

[0061] S6. Based on the environmental robust joint feature vector, perform identity recognition to obtain the identity recognition result.

[0062] Optionally, when performing identity recognition based on robust joint feature vectors, probabilistic linear discriminant analysis (PLDA) is used to complete feature matching and identity determination. First, a pre-stored library of robust joint feature vectors for registered users is accessed. The robust joint feature vector to be identified and the feature vectors of all registered users in the library are input one by one into the PLDA processing logic. By mining the statistical correlation between features, the similarity score of the two types of feature vectors in the low-dimensional discriminant space is calculated, quantifying the degree of matching between the feature to be identified and the registered features. Then, the calculated similarity score is compared with an offline optimized decision threshold, which is determined through cross-validation of the registered dataset to balance the false acceptance rate and the false rejection rate. When the similarity score is higher than the decision threshold, the user to be identified is determined to be a registered user; when the similarity score is lower than the decision threshold, the user to be identified is determined to be a non-registered user, ultimately outputting a clear identity recognition result.

[0063] In the aforementioned biometric recognition method, by simultaneously acquiring dual-modal signals, extracting corresponding features, dynamically adjusting the fusion weights based on noise intensity, and then using PLDA to complete feature matching and identity determination, adaptive adaptation to dynamic noise environments is achieved. This method retains the high discriminative advantage of voiceprint features in quiet environments while leveraging the anti-interference characteristics of earprint features in noisy environments. It effectively solves the problem of unstable recognition performance of fixed-weight fusion in noise fluctuation scenarios, and significantly improves the robustness and accuracy of the identity recognition system.

[0064] In one embodiment, S3 includes:

[0065] S31. Perform frame segmentation on the speech signal data to obtain multiple speech frame data.

[0066] Optionally, since speech signal data exhibits short-term stationarity, framing processing is implemented based on this characteristic to obtain multiple speech frame data. Specifically, an overlapping framing strategy is adopted. First, a fixed frame length and frame shift are set. The frame length is chosen to ensure approximately stationary signal duration within the frame, and the frame shift is set to half or one-third of the frame length to maintain the continuity of information between frames. Then, a sliding window with the frame shift as the step size is used to extract signal segments equal to the frame length from the speech signal data, each segment constituting one speech frame data. To avoid signal edge discontinuities caused by framing, a Hamming window is applied to each extracted speech frame data. The time-domain waveform of the speech frame data is weighted using a window function, ensuring a smooth transition of the amplitude to zero at frame edges. This effectively reduces spectral leakage during spectrum analysis, ensuring the integrity and validity of each speech frame data.

[0067] S32. Calculate the short-time energy of each speech frame data to obtain multiple frame energy values.

[0068] Optionally, short-time energy is a key parameter characterizing the energy strength of speech frame data, and its calculation is based on the temporal sampling values ​​of the speech frame data. For each speech frame data, first, the values ​​of all its temporal sampling points are obtained, then each sampling point value is squared to highlight energy differences, and finally, all squared sampling point values ​​are summed to obtain the frame energy value corresponding to that speech frame data. Preferably, the short-time energy can also be calculated by summing the absolute values ​​of the sampling points. Both calculation methods can reflect the energy level of the speech frame data, with the sum of squares method being more sensitive to changes in signal energy and more suitable for distinguishing between speech frames and noise frames. By performing the above operations sequentially on all speech frame data, multiple frame energy values ​​corresponding one-to-one with the speech frame data can be obtained.

[0069] S33. Based on the preset energy threshold, identify the energy value of the noisy frame from the energy values ​​of multiple frames.

[0070] Optionally, a preset energy threshold is used as a criterion to distinguish between the energy values ​​of speech frames and noise frames. The preset energy threshold is determined based on the energy difference characteristics between speech signals and noise signals. Specifically, the preset energy threshold is obtained by analyzing the energy statistics of a large number of silent signal segments without speech. The maximum energy value of the silent signal segment is taken as the initial threshold, and then fine-tuned in combination with the noise level fluctuation range in the actual application scenario to ensure that the threshold can accurately distinguish between low-energy noise frames and high-energy speech frames. After obtaining multiple frame energy values, each frame energy value is compared with the preset energy threshold one by one. If the energy value of a frame is lower than the preset energy threshold, it is determined to be a noise frame energy value, and the corresponding speech frame data is a noise frame; if the frame energy value is higher than or equal to the preset energy threshold, it is determined to be a speech frame energy value, and the corresponding speech frame data is a valid speech frame.

[0071] S34. Calculate the average value of the noise frame energy to obtain the noise energy estimate.

[0072] Optionally, the noise energy estimate is obtained by calculating the arithmetic mean of the energy values ​​of all noise frames to reflect the average energy intensity of noise in the current environment. First, the total number of identified noise frame energy values ​​is counted. Then, all noise frame energy values ​​are summed to obtain the total noise frame energy. Finally, the total noise frame energy is divided by the total number of noise frame energy values ​​to obtain the noise energy estimate. This calculation method effectively smooths out random fluctuations in the energy values ​​of individual noise frames, more accurately representing the overall energy level of environmental noise.

[0073] S35. Calculate the signal-to-noise ratio estimate based on the noise energy estimate and the overall energy of the speech signal data to obtain the signal-to-noise ratio estimate.

[0074] Optionally, the signal-to-noise ratio (SNR) estimate is calculated based on the ratio of signal energy to noise energy. The overall energy of the speech signal data specifically refers to the sum of the frame energy values ​​corresponding to valid speech frames. A valid speech frame is defined as a speech frame with a frame energy value higher than a preset energy threshold. First, the frame energy values ​​of all valid speech frames are summed to obtain the total valid speech energy. Then, the noise energy estimate is used as a noise energy reference. Finally, the following formula is applied: The signal-to-noise ratio (SNR) estimate in decibels (dB) is calculated. The core principle of this formula is to use logarithmic transformation to convert the energy ratio to a logarithmic scale, which better matches the human ear's perception of sound intensity and clearly reflects the relative strength of the effective speech signal compared to noise.

[0075] S36. Map the signal-to-noise ratio estimate to a predefined scoring interval to obtain the voiceprint quality score.

[0076] Optionally, the predefined scoring interval is a monotonically increasing mapping interval set according to the requirements of voiceprint recognition for voiceprint quality. Specifically, first, the common range of signal-to-noise ratio (SNR) estimates in actual application scenarios is determined, and then this range is evenly divided into multiple continuous sub-intervals. Each sub-interval corresponds to a fixed voiceprint quality score, preferably ranging from 0 to 1, with higher SNR estimates corresponding to higher scores. The mapping process uses a linear mapping method, first calculating the relative position of the SNR estimate within the preset range, and then converting this relative position into the corresponding voiceprint quality score. For example, if the preset range of the SNR estimate is a certain interval, when the SNR estimate reaches the upper limit of the range, the mapped voiceprint quality score is 1; when the SNR estimate drops to the lower limit of the range, the mapped voiceprint quality score is 0; intermediate values ​​correspond to the corresponding scores in a linear proportion, ensuring that the voiceprint quality score can accurately reflect the SNR level, thereby characterizing the reliability of the voiceprint features.

[0077] In the above embodiments, by framing the speech signal data, calculating the frame energy, identifying noise frames, estimating the noise energy, calculating the signal-to-noise ratio and mapping it to the voiceprint quality score, an accurate assessment of the reliability of voiceprint features is achieved. Its core logic is to use energy differences to distinguish speech from noise and to quantify voiceprint quality through the signal-to-noise ratio. This effectively avoids the performance defects of fixed weights in noise fluctuation scenarios and significantly improves the adaptability and robustness of the identity recognition system to dynamic noise environments.

[0078] In one embodiment, the energy value of each frame is compared with a preset energy threshold, and the energy values ​​of frames that are less than the preset energy threshold are determined to be noise frame energy values.

[0079] Optionally, the preset energy threshold is a key criterion determined based on the difference in energy distribution between the speech signal and background noise. Its acquisition requires statistical analysis of the energy characteristics of a large number of silent segments without speech activity. Specifically, silent segment signals from various typical environments are first collected, their frame energy values ​​are extracted and statistically modeled to obtain the probability distribution curve of the silent segment energy. Then, the upper limit of the high-confidence interval in this distribution curve is selected as the initial threshold. Preferably, it is fine-tuned in conjunction with the noise fluctuation range that may occur in actual application scenarios to ensure that the threshold can effectively filter background noise without misjudging low-energy speech frames, such as softly uttered segments.

[0080] The comparison process is performed frame-by-frame. First, all frame energy values ​​are read in chronological order to form an ordered frame energy sequence. Then, each frame energy value in the sequence is compared with a preset energy threshold. The comparison logic is a strict numerical comparison. If the current frame energy value is less than the preset energy threshold, the frame energy value is directly marked as a noise frame energy value, and its corresponding timestamp and frame index are recorded to associate it with the original speech frame data. If the frame energy value is greater than or equal to the preset energy threshold, it is determined to be a speech frame energy value and is not marked further.

[0081] Furthermore, to avoid misjudgment by a single threshold in complex noise environments, a dynamic threshold calibration mechanism can be introduced. By monitoring the energy change trend of multiple consecutive frames in real time, if multiple consecutive frames are at low energy levels and there is no obvious energy jump, the preset energy threshold is appropriately lowered to adapt to the weak noise environment. If a sudden strong noise is detected that causes a large number of frames to abnormally increase in energy values, the threshold is temporarily raised to avoid speech frames being misjudged as noise frames. The calibration process does not change the core judgment logic, but only dynamically adjusts the threshold size based on the statistical characteristics of the frame energy sequence.

[0082] In the above embodiments, by determining the preset energy threshold based on the statistical characteristics of the silent segment, and by combining frame-by-frame comparison and dynamic calibration mechanisms to distinguish between noise frame energy values ​​and speech frame energy values, the frame energy data corresponding to background noise can be accurately screened out, providing a clean and reliable input for noise energy estimation, effectively reducing the interference of noise on voiceprint feature extraction, thereby improving the accuracy of voiceprint quality scoring and enhancing the robustness of the identity recognition system based on the joint voiceprint and earprint in complex environments.

[0083] In one embodiment, S4 includes:

[0084] S41. Based on the voiceprint quality score, the voiceprint fusion weight is calculated using a preset weight mapping function to obtain the voiceprint fusion weight; the formula for calculating the voiceprint fusion weight is:

[0085]

[0086] in, For voiceprint fusion weights, Score the quality of your voiceprint. and These are predefined function parameters.

[0087] Optionally, the preset weight mapping function adopts the Sigmoid function. The core principle of this function is to map the input voiceprint quality score to the range of 0 to 1 through nonlinear transformation. Its output value can be directly used as the voiceprint fusion weight, which can accurately match the nonlinear correlation between voiceprint quality and feature contribution. The voiceprint quality score is a quantified numerical value representing the reliability of the voiceprint, and its value range is limited to 0 to 1. The predefined function parameters a and b are determined through offline calibration experiments. Specifically, voiceprint and earprint joint recognition sample data under various typical noise environments are collected. With the goal of maximizing the recognition accuracy, the parameter space is traversed by the grid search method to select the parameter combination that optimizes the fusion recognition performance under different noise scenarios. After determination, the values ​​of a and b are fixed and stored for real-time calculation. First, the voiceprint quality score s is obtained, and the exponential part is calculated according to the formula: the voiceprint quality score s is multiplied by parameter a, and then added to parameter b to obtain an intermediate result. The negative of the intermediate result is substituted into the natural exponential function exp for calculation; then the value 1 is added to the exponential result, and finally the reciprocal is taken to obtain the voiceprint fusion weight. .

[0088] S42. Calculate the difference between value 1 and voiceprint fusion weight to obtain the earprint fusion weight.

[0089] Optionally, the calculation of earprint fusion weights is based on the principle of complementary weights in bimodal feature fusion, aiming to ensure that the sum of voiceprint fusion weights and earprint fusion weights is always 1, thereby guaranteeing the mathematical rationality and physical consistency of the feature fusion process. Directly calling the voiceprint fusion weights... The value 1 is performed by basic subtraction operation and the voiceprint fusion weight is applied. The difference is calculated. The result of subtracting the two is the earprint fusion weight. Because the voiceprint fusion weight... The value range is between 0 and 1, so the value range of earprint fusion weight is also limited to between 0 and 1. This allows it to dynamically complement the voiceprint fusion weight. When the voiceprint quality is high, the earprint fusion weight automatically decreases, and when the voiceprint quality is low, the earprint fusion weight automatically increases, thus achieving an adaptive balance of the contribution of dual-modal features.

[0090] In the above embodiments, the voiceprint quality score is dynamically mapped to voiceprint fusion weights using the Sigmoid function, and then complementary earprint fusion weights are obtained through interpolation. The core logic is to use nonlinear mapping to adapt the correlation between voiceprint quality and feature contribution, and to ensure the rationality of fusion through complementary weights. This design enables the dual-modal fusion weights to be adaptively adjusted in real time according to voiceprint quality, effectively balancing the feature contributions of voiceprint and earprint under different noise environments, and significantly improving the adaptability and robustness of the identity recognition system based on voiceprint and earprint joint recognition to dynamic noise environments.

[0091] In one embodiment, function parameters and The value of is configured as follows: in response to the voiceprint quality score representing high-quality speech, the voiceprint fusion weight output approaches the first preset weight value; in response to the voiceprint quality score representing low-quality speech, the voiceprint fusion weight output approaches the second preset weight value, and the first preset weight is greater than the second preset weight.

[0092] Optionally, the first preset weight and the second preset weight are fixed values ​​predefined based on the complementary characteristics of the voiceprint and earprint bimodal features. Both are in the range of 0 to 1, and the first preset weight is greater than the second preset weight. The first preset weight is used to strengthen the contribution of the voiceprint features in the fusion when the voiceprint features are reliable, while the second preset weight is used to reduce the weight of the voiceprint features when they are contaminated by noise, so as to avoid invalid information interfering with the recognition results.

[0093] The values ​​of parameters a and b were determined through offline calibration experiments. Specifically, a set of speaker samples covering different noise environments was first constructed. Each sample was labeled with a corresponding speaker quality score and the expected speaker fusion weight. For samples labeled as high-quality speech, the expected speaker fusion weight was set to the first preset weight; for samples labeled as low-quality speech, the expected speaker fusion weight was set to the second preset weight. Then, using the mean square error between the Sigmoid function output and the expected weight as the optimization objective, the gradient descent algorithm was used to iteratively adjust the values ​​of parameters a and b until the function output met the preset conditions.

[0094] The core function of parameter 'a' is to adjust the steepness of the Sigmoid function curve, thereby controlling the sensitivity of the voiceprint fusion weights to changes in voiceprint quality scores. The larger the value of 'a', the steeper the transition of the weights with changes in scores, and the more accurately the weight outputs corresponding to high and low quality speech can be distinguished. The function of parameter 'b' is to shift the position of the function curve to ensure that the boundary value of the voiceprint quality score is precisely matched with the criteria for distinguishing between high and low quality speech. When the score just reaches the high quality threshold, the function output approaches the first preset weight, and when the score just reaches the low quality threshold, the output approaches the second preset weight.

[0095] After calibration, the effect needs to be verified by a validation sample set to ensure that after inputting the voiceprint quality score corresponding to all high-quality speech samples, the output of voiceprint fusion weight is stably close to the first preset weight, after inputting the score corresponding to all low-quality speech samples, the output is stably close to the second preset weight, and the score of intermediate quality level corresponds to a smooth transition of the weight between the two preset values, ultimately forming a weight adjustment logic that dynamically adapts to voiceprint quality.

[0096] In the above embodiments, the values ​​of parameters a and b are determined by offline calibration, so that the Sigmoid function can dynamically output the matched voiceprint fusion weights according to the voiceprint quality score. The voiceprint contribution is strengthened when the voiceprint is of high quality, and its interference is weakened when the voiceprint is of low quality. The complementary advantages of dual-modal features are fully utilized, which effectively solves the problem of unstable performance of fixed weight fusion in noise fluctuation scenarios and significantly improves the noise resistance and recognition reliability of the identity recognition system.

[0097] In one embodiment, S5 includes:

[0098] S51. Perform the first normalization on the voiceprint feature vector to obtain the first normalized vector.

[0099] Optionally, the first normalization employs L2 normalization (Euclidean Normalization). Its core principle is to standardize the magnitude of the voiceprint feature vector to 1, eliminating scale differences between different voiceprint feature vectors and ensuring fairness and effectiveness in the weighting operation. First, the sum of squares of each element of the voiceprint feature vector is calculated, then the square root of this sum is taken to obtain the L2 magnitude of the voiceprint feature vector. Subsequently, each element in the voiceprint feature vector is divided by this L2 magnitude to obtain the first normalized vector. L2 normalization effectively avoids weighting bias caused by the voiceprint feature vector itself being too large or too small, making the weights' adjustment effect on the features more balanced, while preserving the relative proportions of each dimension of the voiceprint feature vector. Preferably, if the L2 magnitude of the voiceprint feature vector is 0 (in extreme scenarios with no effective features), then a vector of all zeros is directly output as the first normalized vector to avoid computational anomalies.

[0100] S52. Calculate the square root of the voiceprint fusion weights to obtain the first scaling factor.

[0101] Optionally, the first scaling factor is obtained by taking the square root of the voiceprint fusion weights. The core reason for choosing the square root operation is to mitigate the impact of extreme weight values, making the weighted voiceprint feature vector more stable while maintaining the relative magnitude of the weights. The standard mathematical square root function is directly called to calculate the voiceprint fusion weights without the need for additional parameter adjustments. If the voiceprint fusion weight is 0, the first scaling factor is 0; if the voiceprint fusion weight is 1, the first scaling factor is 1, ensuring a reasonable scaling effect when the weights are at extreme values. This approach retains the intention of the voiceprint fusion weights to adjust the features while avoiding the problems of excessive feature amplification when the weights are too large or excessive feature suppression when the weights are too small.

[0102] S53. Multiply each element of the first normalized vector by the first scaling factor to obtain the weighted voiceprint feature vector.

[0103] Optionally, element-wise multiplication is used to multiply the first normalized vector by the first scaling factor to obtain a weighted voiceprint feature vector. Each element of the first normalized vector is iterated through, and each element is multiplied by the first scaling factor. The result of each operation is used as the element at the corresponding position in the weighted voiceprint feature vector. This multiplication method allows each dimension of the first normalized vector to be scaled according to the voiceprint fusion weights. When the voiceprint fusion weights are high, the contributions of each dimension of the normalized voiceprint features are enhanced, highlighting effective information; when the voiceprint fusion weights are low, the contributions of each dimension are weakened, reducing noise interference.

[0104] S54. Perform a second normalization on the earprint feature vector to obtain the second normalized vector.

[0105] Optionally, the second normalization uses the exact same L2 normalization method as the first normalization to ensure that the normalization standards of the voiceprint and earprint feature vectors are consistent, avoiding feature imbalance caused by differences in normalization methods during splicing. First, the sum of squares of each element of the earprint feature vector is calculated. Then, the square root of this sum is taken to obtain the L2 magnitude of the earprint feature vector. Finally, each element of the earprint feature vector is divided by this magnitude to obtain the second normalized vector. If the L2 magnitude of the earprint feature vector is 0, a vector of all zeros is also output as the second normalized vector.

[0106] S55. Calculate the square root of the earprint fusion weight to obtain the second scaling factor.

[0107] Optionally, the calculation logic of the second scaling factor is completely consistent with that of the first scaling factor, obtained by performing a square root operation on the earprint fusion weights. This unified calculation method ensures the consistency of the weighting logic for voiceprints and earprints, avoiding fusion deviations caused by different scaling factor calculation methods. The standard mathematical square root function is used to directly calculate the earprint fusion weights; when the earprint fusion weight is 0, the second scaling factor is 0, and when it is 1, it is 1. This mitigates the influence of extreme weights, making the weighting adjustment of the earprint feature vector more stable and adapting to the contribution requirements of earprint features under different noise environments.

[0108] S56. Multiply each element of the second normalized vector by the second scaling factor to obtain the weighted earprint feature vector.

[0109] Optionally, element-wise multiplication is used to multiply the second normalized vector by the second scaling factor to obtain the weighted earprint feature vector. Each element of the second normalized vector is iterated and multiplied by the second scaling factor, with the result serving as the element at the corresponding position in the weighted earprint feature vector. This operation allows the normalized earprint feature vector to be scaled according to its fusion weights. When the earprint fusion weights are high, such as in noisy environments, the contributions of each dimension of the earprint features are enhanced, compensating for the deficiencies of the voiceprint features; when the weights are low, the contributions of the earprint features are appropriately weakened, highlighting the role of the voiceprint features, ensuring reasonable complementarity between the two features in different scenarios.

[0110] S57. By concatenating the weighted voiceprint feature vector and the weighted earprint feature vector, an environmentally robust joint feature vector is obtained.

[0111] Optionally, the splicing operation employs a fixed-order dimensional splicing method, arranging all elements of the weighted voiceprint feature vector in their original order first, followed by all elements of the weighted earprint feature vector in their original order, and then concatenating them to form an environment-robust joint feature vector. During the splicing process, the element order of the two vectors remains unchanged, and the numerical values ​​of any element are not altered; only the weighted information of the two features is integrated through dimensional expansion. This fixed-order splicing ensures that the structure of the joint feature vector remains consistent after each fusion. The joint feature vector contains both weighted voiceprint features and integrated weighted earprint features, combining the advantages of both modalities and adapting to recognition needs in dynamic noise environments.

[0112] In the above embodiments, scale differences are eliminated by L2 normalization of the voiceprint and earprint feature vectors respectively, and the influence of extreme weights is mitigated by using the square root of the weights as a scaling factor. Then, element-wise weighting and fixed-order concatenation are used to obtain joint features. The core logic is to allow the two features to be adaptively fused according to dynamic weights under a unified standard. This design maintains the relative integrity of the features while achieving precise adjustment of the weights' contribution to the features, effectively compensating for the shortcomings of fixed fusion strategies and enabling the joint features to combine the advantages of both voiceprints and earprints.

[0113] In one embodiment, the first normalization and the second normalization are L2 norm normalizations.

[0114] In the aforementioned biometric identification method, system, device, and medium, the biometric identification method is based on the complementary features of voiceprint and earprint dual-modality features. It first simultaneously acquires speech signal data and ear canal echo signal data and extracts corresponding feature vectors. Then, it segments the speech signal, calculates short-time energy, identifies noise frames, estimates the signal-to-noise ratio, and maps it to a voiceprint quality score. Based on the voiceprint quality score, it dynamically generates voiceprint and earprint fusion weights using the Sigmoid function. After L2 normalization and weight square root scaling, the dual-modality features are weighted and concatenated to obtain an environment-robust joint feature vector. Finally, it completes identity recognition based on this environment-robust joint feature vector, forming a complete identity recognition process adapted to dynamic noise environments. By dynamically adjusting the fusion weights of the dual-modality features according to voiceprint quality, the fusion strategy can adapt to changes in environmental noise in real time, effectively solving the problem of unstable recognition performance in noise fluctuation scenarios with fixed-weight fusion, and significantly improving the anti-interference capability and recognition accuracy of the identity recognition system.

[0115] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0116] Based on the same inventive concept, this application also provides a system for implementing the biometric identification method described above. The solution provided by this system is similar to the implementation scheme described in the above method; therefore, the specific limitations of one or more biometric identification system embodiments provided below can be found in the limitations of the biometric identification method described above, and will not be repeated here.

[0117] In one exemplary embodiment, such as Figure 2 As shown, a biometric identification system 10 is provided, comprising:

[0118] Data acquisition module 11 is used to acquire speech signal data and ear canal echo signal data;

[0119] The feature extraction module 12 is used to extract the voiceprint feature vector of the speech signal data and the earprint feature vector of the ear canal echo signal data to obtain the voiceprint feature vector and the earprint feature vector.

[0120] The noise assessment module 13 is used to assess the noise interference intensity based on speech signal data, obtain the assessment results, and generate a voiceprint quality score based on the assessment results.

[0121] The weight calculation module 14 is used to dynamically calculate and generate voiceprint fusion weights and earprint fusion weights based on the voiceprint quality score.

[0122] The feature fusion module 15 is used to perform weighted fusion of the voiceprint feature vector and the earprint feature vector through voiceprint fusion weight and earprint fusion weight to generate an environment-robust joint feature vector.

[0123] The identity recognition module 16 is used to perform identity recognition based on the environment robust joint feature vector to obtain the identity recognition result.

[0124] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of a biometric identification method as described above.

[0125] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of a biometric identification method as described above.

[0126] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0127] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A biometric identification method, characterized in that, The method includes: S1. Acquire speech signal data and ear canal echo signal data; S2. Extract the voiceprint feature vector from the speech signal data and extract the earprint feature vector from the ear canal echo signal data to obtain the voiceprint feature vector and the earprint feature vector. S3. Based on the speech signal data, evaluate the noise interference intensity to obtain the evaluation result, and generate a voiceprint quality score based on the evaluation result; S4. Based on the voiceprint quality score, dynamically calculate and generate voiceprint fusion weights and earprint fusion weights; S5. The voiceprint feature vector and the earprint feature vector are weighted and fused using the voiceprint fusion weight and the earprint fusion weight to generate an environment robust joint feature vector. S6. Based on the robust joint feature vector of the environment, perform identity recognition to obtain the identity recognition result.

2. The method according to claim 1, characterized in that, S3 includes: S31. Perform frame segmentation processing on the speech signal data to obtain multiple speech frame data; S32. Calculate the short-time energy of each of the speech frame data to obtain multiple frame energy values; S33. Based on a preset energy threshold, identify the noise frame energy value from the plurality of frame energy values; S34. Calculate the average value of the noise frame energy to obtain the noise energy estimate; S35. Calculate the signal-to-noise ratio estimate based on the noise energy estimate and the overall energy of the speech signal data to obtain the signal-to-noise ratio estimate; S36. Map the signal-to-noise ratio estimate to a predefined scoring range to obtain the voiceprint quality score.

3. The method according to claim 2, characterized in that, Each frame energy value is compared with the preset energy threshold, and the frame energy values ​​that are less than the preset energy threshold are determined to be the noise frame energy values.

4. The method according to claim 1, characterized in that, S4 includes: S41. Based on the voiceprint quality score, the voiceprint fusion weight is calculated using a preset weight mapping function to obtain the voiceprint fusion weight; the formula for calculating the voiceprint fusion weight is: in The voiceprint fusion weights, The voiceprint quality is scored. and These are predefined function parameters; S42. Calculate the difference between value 1 and the voiceprint fusion weight to obtain the earprint fusion weight.

5. The method according to claim 4, characterized in that, The function parameters and The value of is configured as follows: in response to the voiceprint quality score representing high-quality speech, the voiceprint fusion weight output approaches a first preset weight value; in response to the voiceprint quality score representing low-quality speech, the voiceprint fusion weight output approaches a second preset weight value, wherein the first preset weight is greater than the second preset weight.

6. The method according to claim 1, characterized in that, S5 includes: S51. Perform a first normalization on the voiceprint feature vector to obtain a first normalized vector; S52. Calculate the square root of the voiceprint fusion weight to obtain the first scaling factor; S53. Multiply each element of the first normalized vector by the first scaling factor to obtain a weighted voiceprint feature vector. S54. Perform a second normalization on the earprint feature vector to obtain a second normalized vector; S55. Calculate the square root of the earprint fusion weight to obtain the second scaling factor; S56. Multiply each element of the second normalized vector by the second scaling factor to obtain the weighted earprint feature vector; S57. Concatenate the weighted voiceprint feature vector and the weighted earprint feature vector to obtain the environmental robust joint feature vector.

7. The method according to claim 6, characterized in that, The first normalization and the second normalization are L2 norm normalizations.

8. A biometric identification system, characterized in that, The system includes: The data acquisition module is used to acquire voice signal data and ear canal echo signal data; The feature extraction module is used to extract the voiceprint feature vector of the speech signal data and the earprint feature vector of the ear canal echo signal data to obtain the voiceprint feature vector and the earprint feature vector. The noise assessment module is used to assess the noise interference intensity based on the speech signal data, obtain the assessment result, and generate a voiceprint quality score based on the assessment result. The weight calculation module is used to dynamically calculate and generate voiceprint fusion weights and earprint fusion weights based on the voiceprint quality score. The feature fusion module is used to perform weighted fusion of the voiceprint feature vector and the earprint feature vector through the voiceprint fusion weight and the earprint fusion weight to generate an environment-robust joint feature vector; The identity recognition module is used to perform identity recognition based on the robust joint feature vector of the environment and obtain the identity recognition result.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images