Lightweight identity recognition method combining voiceprint and earprint features
By combining the fusion of voiceprint and earprint features on small intelligent devices and extracting high-level abstract features with multi-layer neural networks, the contradiction between model lightweighting and performance improvement is solved, and a significant reduction is achieved while reducing the error rate. The parameters and improve anti-interference, suitable for identification of Bluetooth headsets and other devices.
Patent Information
- Application Number
- CN202411124137.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-15
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2044-08-15
AI Technical Summary
On small smart devices such as Bluetooth headsets, the deep learning model has a large number of parameters and high calculations, making it difficult to achieve lightweight model while maintaining recognition performance, especially the problem of voiceprint recognition being susceptible to environmental noise interference and causing a decrease in accuracy.
Using a lightweight identity recognition method with combined voice thread ear thread features, the 13-dimensional Merle frequency cepspectral coefficients of speech and ear canal echo signals are extracted for fusion, and the lightweight identity recognition model is input, using multi-layer neural network and fully connected structure Feature extraction is performed, and the number of parameters is reduced by pre-training the model, and the similarity is calculated by using probability linear discriminant analysis for identity recognition.
While reducing the error rate, the number of parameters of the identity recognition model is greatly reduced, and the anti-interference and recognition accuracy of the model are improved. It is suitable for small devices with limited storage and computing capabilities.
Smart Images

Figure CN118972070A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of deep learning models, and in particular to a lightweight identity recognition method combining voiceprint and earprint features. Background Art
[0002] With the rapid development of artificial intelligence and other technologies, various identity recognition technologies have emerged one after another, such as iris recognition, fingerprint recognition, face recognition, voiceprint recognition and other biometric technologies. The accuracy of voiceprint recognition technology is relatively low compared to other biometric technologies. This is because the collection of voice data is easily interfered by various noises such as the environment, thereby reducing the recognition accuracy. Combining voiceprint data with earprint data can reduce interference such as environmental noise and improve the accuracy of identity recognition. It should be pointed out that the earprint features described in the claims of the present invention refer to the features of the echo signal in the auditory canal of the human ear, and do not refer to the features of the external pattern of the human ear like fingerprints.
[0003] Early patents provided methods for extracting earprint acoustic features (such as [1] Yu Anbo. An online identification method and system for earprint acoustic characteristics suitable for noise reduction headphone design [P]. Beijing: CN201610842206.8, 2017-08-22.), but did not involve how to use earprint features for identity recognition.
[0004] In recent years, some patents have proposed methods for using earprint features for identity recognition (such as patent [2] Huang Qinbo, Guo Quan, Li Jing. Privacy recognition method and device based on earprint [P]. Guangdong: CN201610751856.1, 2018-03-09; patent [3] J.P. Leso, W.E. Sherwood, P. Bardsley, et al. Biometric identification process, device and machine-readable medium [P]. United Kingdom: CN202080013151.4, 2021-09-17.). For example, patent [2] uses the identifiability of earprints between people to identify whether the person to be verified is a known authorized person by matching the earprints of the person to be verified with the earprints of the known authorized person. However, such patents do not involve methods for using voiceprints and earprints together for identity recognition.
[0005] Furthermore, patent [4] (Dai Zuhua, Zhang Yinong. Target user confirmation method, device, electronic device and readable storage medium [P]. Jiangsu Province: CN202311661230.8, 2024-03-26.) proposes a method for identity recognition using earprints and other biometric features (including voiceprints), but in this method, earprints and voiceprint features are respectively used for identity recognition through the first and second confirmation models. In contrast, the present invention fuses the earprint and voiceprint features and inputs them into a common identity recognition model, and focuses on the lightweight structure of the identity recognition model to meet the needs of using the recognition model in small smart devices such as headphones.
[0006] Small smart devices such as Bluetooth headsets have weak parameter storage and computing capabilities, while deep learning neural network models generally have large parameters and computing power. Therefore, if deep learning models are to be suitable for small devices with weak storage and computing power, it is necessary to reduce model parameters, that is, to make the model lightweight. However, the challenge faced by model lightweighting is how to balance the contradiction between reducing the number of model parameters and improving performance, because reducing the number of parameters will reduce model performance. Compared with existing methods, the method proposed in the present invention achieves a significant reduction in the number of model parameters under the premise of a better recognition error rate (EER). Summary of the invention
[0007] The purpose of the present invention is to solve the contradiction between balancing the reduction of model parameters and the improvement of performance faced by lightweight models when using voiceprints and earprints for identity recognition on small smart devices such as Bluetooth headsets. Compared with the method of performing identity recognition only with voiceprint data, the present invention greatly reduces the number of parameters of the identity recognition model while reducing the equal error rate (EER).
[0008] To achieve the above-mentioned purpose, the present invention provides a lightweight identity recognition method combining voiceprint and earprint features. The technical problem to be solved by the present invention is achieved through the following technical solutions:
[0009] The embodiment of the present invention provides a lightweight identity recognition method combining voiceprint and earprint features, and the identity recognition method includes:
[0010] Step 1: Acquire the voice and ear canal echo signals of a known registrant;
[0011] Step 2: extract the 13-dimensional Mel-Frequency Cepstral Coefficients (MFCC) of the known registered person's voice and ear canal echo signal respectively, and fuse the Mel-Frequency Cepstral feature coefficients of the two to obtain the fusion feature of the known registered person's voiceprint and earprint;
[0012] Step 3: Input the fusion features obtained in step 2 into the lightweight identity recognition model, and output a 128-dimensional embedded feature vector;
[0013] Step 4: Acquire the voice and ear canal echo signals of the person to be verified;
[0014] Step 5: extract the 13-dimensional Mel-frequency cepstrum coefficients of the voice and ear canal echo signal of the person to be verified respectively, and fuse the Mel-frequency cepstrum feature coefficients of the two to obtain the fusion feature of the voiceprint and earprint of the person to be verified;
[0015] Step 6: Input the fusion features of the voiceprint and earprint obtained in step 5 into the lightweight identity recognition model, and output a 128-dimensional embedded feature vector;
[0016] Step 7: Perform similarity comparison on the 128-dimensional embedded feature vectors outputted from Step 3 and Step 6, and if the similarity is greater than a threshold, the person to be verified is determined to be a known registrant; otherwise, the person to be verified is determined not to be a known registrant;
[0017] The lightweight identity recognition model is used to extract high-level abstract features of the voiceprint and earprint fusion features layer by layer to obtain representation features at the signal frame level; the lightweight identity recognition model includes a multi-layer neural network and a fully connected structure, wherein each layer of the neural network includes multiple fully connected layers, activation functions, and superposition layers in sequence, and the fully connected structure includes a fully connected layer, an activation function, a statistical pooling layer, and a fully connected output layer in sequence.
[0018] In one embodiment of the invention, the multi-layer neural network in the lightweight identity recognition model includes:
[0019] Overlay layer: concat, used to overlay the output features of the multiple fully connected layers, and the final output is 64-dimensional features.
[0020] In one embodiment of the invention, the fully connected structure in the lightweight identity recognition model includes:
[0021] Statistical pooling layer: 128×256, 128 means 128-dimensional input, 256 means 256-dimensional output, used to calculate the mean and standard deviation of the input features to aggregate frame features:
[0022] Fully connected output layer: 256×128Dense, 256 means 256-dimensional input, 128 means 128-dimensional output embedding feature.
[0023] In one embodiment of the invention, the similarity calculation in step 7 uses the probabilistic linear discriminant analysis (PLDA) method to calculate the similarity of the embedded layer features between the person to be verified and the registered person.
[0024] In one embodiment of the invention, the lightweight identity recognition model needs to be pre-trained, and the pre-training includes two models, one is a delayed neural network using ECAPA (Emphasized Channel Attention, Propagation and Aggregation) as a pre-training model 1, and the other is the lightweight identity recognition model as a pre-training model 2. The training process of the two pre-training models is as follows:
[0025] Acquire training sample speech and ear canal echo signals, and label the training sample speech and ear canal echo signals to form labeled training samples;
[0026] Extracting 13-dimensional Mel-frequency cepstral coefficients of each labeled speech and ear canal echo signal, and fusing the Mel-frequency cepstral feature coefficients of the two, that is, obtaining the fusion feature of voiceprint and earprint;
[0027] The fused features of all the labeled voiceprints and earprints are input into the pre-trained model 1 and the pre-trained model 2 respectively;
[0028] The total loss function of pre-trained model 1 and pre-trained model 2 is set as follows:
[0029]
[0030] in are the feature vectors output by pre-trained model 1 and pre-trained model 2, and score 1 and score 2 are the class probability scores of the output features of pre-trained model 1 and pre-trained model 2 respectively; N represents the number of samples, M represents the total number of categories; y k is a sign function, which takes 1 if the true category of sample i is equal to c, otherwise it takes 0; is the predicted probability value that sample i belongs to category c, and α (0<α<1) is the weight used to balance the two loss functions.
[0031] After the pre-trained model is trained, the parameters of the pre-trained model 2 are saved to obtain a trained lightweight identity recognition model.
[0032] Compared with the prior art, the present invention has the following beneficial effects:
[0033] (1) In order to solve the problem that voice signals are easily interfered with, which reduces the accuracy of voiceprint identification, the present invention proposes a method for fusing the features of voiceprint and earprint data to improve the anti-interference ability of the identification model.
[0034] (2) The present invention aims to solve the problem that a large number of identity recognition models cannot be deployed on small smart devices such as Bluetooth headsets, and constructs a lightweight identity recognition model. First, the dimension of the model input data features is reduced, that is, only the 13-dimensional Mel-frequency cepstral coefficients of voiceprint and earprint data are input; secondly, while reducing the number of model parameters, pre-learning is performed from the pre-trained model with a large number of parameters through the pre-training process to ensure the recognition accuracy of the lightweight model, thereby effectively balancing the contradiction between the number of model parameters and performance improvement.
[0035] (3) Compared with the method of performing identity recognition based on voiceprint data only, the present invention reduces the equal error rate (EER) and significantly reduces the number of parameters of the identity recognition model.
[0036] Description of the accompanying tables and graphs
[0037] Figure 1 This is a flowchart of a lightweight identity recognition method combining voiceprint and earprint features provided by an embodiment of the present invention;
[0038] Figure 2 It is a schematic diagram of the lightweight identity recognition model provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0039] The technical solution in the embodiment of this patent will be further described in detail below in combination with the drawings and specific embodiments of the present invention.
[0040] The specific embodiments of the present invention are described in detail below. It should be understood that the embodiments described in the drawings in the present invention are only used to explain the implementation method of the present invention, rather than being a limitation of the present invention.
[0041] A lightweight identity recognition method combining voiceprint and earprint features provided in an embodiment of the present invention can be applied to a terminal device, can be implemented in a combination of software and hardware devices, and integrated into a terminal device, such as a Bluetooth headset.
[0042] See also Figure 1 Now, a lightweight identity recognition method combining voiceprint and earprint features provided by an embodiment of the present invention is described. The specific implementation method of this embodiment includes the following steps:
[0043] Step 1: Acquire the voice and ear canal echo signals of a known registrant.
[0044] Specifically, the registrant wears a wired headset with playback and recording functions. The other end of the headset is connected to a sound card device to play a two-second Gaussian white noise. The headset recording is used to collect the echo signal obtained after the Gaussian white noise passes through the left and right ear canals of the registrant; the voice of the registrant speaking into the microphone connected to the headset is recorded for about 3 seconds.
[0045] The ear canal echo signal is Gaussian white noise and does not need to be preprocessed. The speech signal needs to be detected by double threshold endpoints to calculate the short-time energy and zero-crossing rate of each frame of speech signal. The voiced segments are retained according to the threshold, and the unvoiced and silent segments are removed.
[0046] Step 2: Extract the 13-dimensional Mel-Frequency Cepstral Coefficients (MFCC) of the known registered person's voice and ear canal echo signal respectively, and fuse the Mel-Frequency Cepstral feature coefficients of the two to obtain the fusion features of the known registered person's voiceprint and earprint.
[0047] Specifically, the process of extracting 13-dimensional Mel-frequency cepstral coefficients is as follows: pre-emphasis, framing, windowing, fast Fourier transform, Mel filter bank, logarithmic operation, discrete Fourier transform, and dynamic feature extraction are performed on the 2-second ear canal echo signal and the preprocessed 1-second speech signal, wherein the selected frame length is 256 points, the frame shift is 160 points, the window length is 512, the pre-emphasis coefficient is 0.97, and 26 Mel filters are used to extract the 13-dimensional Mel-frequency cepstral coefficients, including the energy of the first dimension;
[0048] Extract the 13-dimensional Mel-frequency cepstrum coefficients of the ear canal echo signal, denoted by A=[A1, A2, A3, ..., A 13 ], extract the 13-dimensional Mel-frequency cepstrum coefficients of the speech signal, denoted as B = [B1, B2, B3, ..., B 13 ], concatenate A and B by dimension, and obtain a 13-dimensional vector of the Mel-frequency cepstrum coefficients of the fused ear canal echo and speech signal, denoted as C = [C1, C2, C3, ..., C 13 ], where C i ={A i , B i}, the symbol {,} represents splicing, i∈{1, 2, ..., 13}.
[0049] Step 3: Input the fusion features obtained in step 2 into the lightweight identity recognition model and output a 128-dimensional embedded feature vector.
[0050] Specifically, if Figure 2 As shown, the lightweight identity recognition model includes a multi-layer neural network and a fully connected structure, wherein each layer of the neural network includes multiple fully connected layers, activation functions, and stacking layers in sequence, and the fully connected structure includes a fully connected layer, an activation function, a statistical pooling layer, and a fully connected output layer in sequence.
[0051] Among them, the first layer of the multi-layer neural network consists of 5 linear full connections and a superposition layer: the first 4 full connections are all 13×12Dense, and the fifth is 13×16Dense. 13×12 means 13-dimensional input and 12-dimensional output, and 13×16 means 13-dimensional input and 16-dimensional output; finally, the outputs of these 5 full connections are input to the superposition layer for concat splicing to obtain a 64-dimensional output. Setting up the neural network in this way is equivalent to decomposing a single 65×64Dense into the above 5 full connections, which can greatly reduce the number of parameters of the model. For example, the number of parameters of a 65×64Dense is 4160, and the total number of parameters of 4 13×12Dense and 1 13×16Dense is 832. It can be seen that the number of model parameters is reduced to 1 / 5 of that of a 65×64Dense, making the model more lightweight.
[0052] The second layer of the multi-layer neural network consists of three linear full connections and one stacking layer: the first two full connections are 64×20Dense, and the third is 64×24Dense. 64 is the output dimension of the first layer of the neural network, which should be equal to the input dimension of the second layer; finally, the outputs of these three full connections are input to the stacking layer for concat splicing to obtain a 64-dimensional output. Setting up the neural network in this way is equivalent to decomposing a single 192×64Dense into the three full connections mentioned above, as described in the first layer of the previous paragraph, which can greatly reduce the number of model parameters.
[0053] The third layer in the multi-layer neural network consists of three linear full connections and one stacking layer: the first two full connections are 64×20Dense, and the third is 64×24Dense. 64 is the output dimension of the first layer of the neural network, which should be equal to the input dimension of the second layer; finally, the outputs of these three full connections are input to the stacking layer concat to obtain a 64-dimensional output. Setting up the neural network in this way is equivalent to decomposing a single 192×64Dense into the three full connections mentioned above, as described in the first layer of the previous paragraph, which can greatly reduce the number of model parameters.
[0054] Furthermore, if Figure 2 As shown, the fully connected layer in the fully connected structure of the lightweight identity recognition model is composed of two fully connected layers: 64×64Dense and 64×128Dense. 64×64 means that both the input and output are 64 dimensions, and 64×128 means that the input and output are 64 and 128 dimensions respectively. The 64-dimensional features of the previous layer output are increased to 128 dimensions to extract high-level features.
[0055] The statistical pooling layer in the fully connected structure of the lightweight identity recognition model is: 128×256StatsPooling, 128×256 means that the input and output are 128 and 256 dimensions respectively, the mean and standard deviation of the input features are calculated, and a 256-dimensional output is obtained to aggregate frame features;
[0056] The fully connected output layer in the fully connected structure of the lightweight identity recognition model is a linear fully connected 256×128Dense, which uses the 256-dimensional output of the statistical pooling layer as the input of this layer and outputs a 128-dimensional embedded feature vector;
[0057] Step 4: Acquire the voice and ear canal echo signals of the person to be verified.
[0058] Specifically, the person to be verified wears a wired headset with playback and recording functions. The other end of the headset is connected to a sound card device to play a two-second Gaussian white noise. The headset recording collects the echo signal obtained after the Gaussian white noise passes through the left and right ear canals of the registrant; the voice of the registrant speaking into the microphone connected to the headset is recorded for about 3 seconds.
[0059] The ear canal echo signal is Gaussian white noise and does not need to be preprocessed. Double threshold endpoint detection is performed on the speech signal to calculate the short-time energy and zero-crossing rate of each frame of speech signal. The voiced segment is retained according to the threshold, and the unvoiced and silent segments are removed.
[0060] Step 5: respectively extract the 13-dimensional Mel-frequency cepstrum coefficients of the voice and ear canal echo signal of the person to be verified, and fuse the Mel-frequency cepstrum feature coefficients of the two to obtain the fusion features of the voiceprint and earprint of the person to be verified.
[0061] Specifically, the process of extracting 13-dimensional Mel-frequency cepstral coefficients and obtaining their fusion features is the same as described in step 2 above.
[0062] Step 6: Input the fusion features of the voiceprint and earprint obtained in step 5 into the lightweight identity recognition model, and output a 128-dimensional embedded feature vector;
[0063] Specifically, it is the same as described in the above step 3.
[0064] Step 7: The 128-dimensional embedded feature vectors outputted from Step 3 and Step 6 are compared for similarity. If the similarity is greater than a threshold, the person to be verified is determined to be a known registrant; otherwise, the person to be verified is determined not to be a known registrant.
[0065] Specifically, the similarity calculation in step 7 uses the probabilistic linear discriminant analysis (PLDA) method to calculate the similarity of the embedding layer features between the person to be verified and the registered person. If the similarity value exceeds the set threshold, the identity recognition is successful, that is, the registered person and the person to be verified are the same user; otherwise, the identity recognition fails, that is, the person to be verified is not the registered person.
[0066] In the above embodiment, the lightweight identity recognition model needs to be pre-trained, and the pre-training includes two models, one is a delayed neural network using ECAPA (Emphasized Channel Attention, Propagation and Aggregation) as pre-training model 1, and the other is the lightweight identity recognition model as pre-training model 2. The training process of the two pre-training models is as follows:
[0067] A training sample speech and an ear canal echo signal are obtained, and the training sample speech and the ear canal echo signal are labeled to form a labeled training sample.
[0068] Specifically, the process of obtaining training sample speech and ear canal echo signals is as follows: first, the training sample data collector (hereinafter referred to as the collected person) is numbered in order; secondly, the collected person wears a headset with playback and recording functions, and the other end of the headset is connected to a sound card device to play a two-second Gaussian white noise. The echo signal after the Gaussian white noise passes through the left and right ear canals of the collected person is collected through the headset recording; the collected person needs to wear the headset 15 times on each ear, and each time the headset is taken off and put on again to ensure that each time has a certain degree of randomness. Each person's data is 30, each 2 seconds, and at least 100 people are collected, including boys and girls; voice signal collection only needs to record. In order to increase the robustness of the model, the recording content of each person is different, and each person is recorded for 3 seconds. Finally, according to the number of the collected person, the speech and ear canal echo signals of each collected person are marked, and the number of each mark is unique and non-repeating, so as to form a label for the speech and ear canal echo signals of each collected person.
[0069] Furthermore, the ear canal echo signal is Gaussian white noise and does not require preprocessing. The speech signal needs to perform double threshold endpoint detection, calculate the short-time energy and zero-crossing rate of each frame of the speech signal, retain the voiced segment according to the set threshold, and remove the unvoiced and silent segments.
[0070] The 13-dimensional Mel-frequency cepstral coefficients of each labeled speech and ear canal echo signal are extracted, and the Mel-frequency cepstral feature coefficients of the two are fused to obtain the voiceprint and earprint fusion features.
[0071] Specifically, the process of extracting 13-dimensional Mel-frequency cepstral coefficients and obtaining their fusion features is the same as described in step 2 above.
[0072] All the labeled voiceprint and earprint fusion features are input into pre-training model 1 and pre-training model 2 respectively, and the total loss function of pre-training model 1 and pre-training model 2 is set as follows:
[0073]
[0074] in are the feature vectors output by pre-trained model 1 and pre-trained model 2, and score 1 and score 2 are the class probability scores of the output features of pre-trained model 1 and pre-trained model 2 respectively; N represents the number of samples, M represents the total number of categories; y k is a sign function, which takes 1 if the true category of sample i is equal to c, otherwise it takes 0; is the predicted probability value that sample i belongs to category c, and α (0<α<1) is the weight used to balance the two loss functions.
[0075] The pre-trained model is trained until the total loss function is less than the threshold or the output error rate (EER) no longer decreases. After training, the parameters of the pre-trained model 2 are saved to obtain a trained lightweight identity recognition model.
[0076] According to the above-mentioned embodiment of a lightweight identity recognition method combining voiceprint and earprint features, the lightweight model parameter quantity and recognition performance are shown in Table 1. It can be seen that compared with the method of identity recognition using only voiceprint data, the present invention significantly reduces the number of parameters of the identity recognition model while reducing the equal error rate (EER).
[0077] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
[0078] Table 1 is a lightweight performance comparison of a lightweight identity recognition method provided by an embodiment of the present invention
[0079]
[0080] [1] Casanova, Edresson, et al. "Speech2Phone: a novel and efficient method for training speaker recognition models." Brazilian Conference on IntelligentSystems. Cham: Springer International Publishing, 2021.
[0081] [2]Chung, Joon Son, et al. "In defence ofmetric learning for speakerrecognition." arxiv preprint arxiv:2003.11982,2020.
[0082] [3]Wan, Li, et al. "Generalized end-to-end loss for speaker verification." 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2018.
[0083] [4] Zhu Zhen, Wu Jing, Zhu Wenbo, et al. A lightweight single-channel voiceprint recognition method and system based on deep learning. 202310168991, 2024.
Claims
1. A lightweight identity recognition method combining voiceprint and earprint features, characterized in that: After the voiceprint and earprint features of the person to be verified are integrated, they are input into the lightweight identity recognition model to determine whether the person to be verified is a registered person. The specific steps include: Step 1: Acquire the voice and ear canal echo signals of a known registrant; Step 2: extract the 13-dimensional Mel-Frequency Cepstral Coefficients (MFCC) of the known registered person's voice and ear canal echo signal respectively, and fuse the Mel-Frequency Cepstral feature coefficients of the two to obtain the fusion feature of the known registered person's voiceprint and earprint; Step 3: Input the fusion features obtained in step 2 into the lightweight identity recognition model, and output a 128-dimensional embedded feature vector; Step 4: Acquire the voice and ear canal echo signals of the person to be verified; Step 5: extract the 13-dimensional Mel-frequency cepstrum coefficients of the voice and ear canal echo signal of the person to be verified respectively, and fuse the Mel-frequency cepstrum feature coefficients of the two to obtain the fusion feature of the voiceprint and earprint of the person to be verified; Step 6: Input the fusion features of the voiceprint and earprint obtained in step 5 into the lightweight identity recognition model, and output a 128-dimensional embedded feature vector; Step 7: Perform similarity comparison on the 128-dimensional embedded feature vectors outputted from Step 3 and Step 6, and if the similarity is greater than a threshold, the person to be verified is determined to be a known registrant; otherwise, the person to be verified is determined not to be a known registrant; The lightweight identity recognition model is used to extract high-level abstract features of the voiceprint and earprint fusion features layer by layer to obtain representation features at the signal frame level; the lightweight identity recognition model includes a multi-layer neural network and a fully connected structure, wherein each layer of the neural network includes multiple fully connected layers, activation functions, and superposition layers in sequence, and the fully connected structure includes a fully connected layer, an activation function, a statistical pooling layer, and a fully connected output layer in sequence.
2. According to claim 1, a lightweight identity recognition method combining voiceprint and earprint features is characterized by: The multi-layer neural network in the lightweight identity recognition model, wherein: Overlay layer: concat, used to overlay the output features of the multiple fully connected layers, and the final output is 64-dimensional features.
3. According to claim 1, a lightweight identity recognition method combining voiceprint and earprint features is characterized by: The fully connected structure in the lightweight identity recognition model, where Statistical pooling layer: 128×256, 128 means 128-dimensional input, 256 means 256-dimensional output, used to calculate the mean and standard deviation of the input features to aggregate frame features; Fully connected output layer: 256×128Dense, 256 means 256-dimensional input, 128 means 128-dimensional output embedding feature.
4. The lightweight identity recognition method combining voiceprint and earprint features according to claim 1 is characterized by: The similarity calculation in step 7 uses the probabilistic linear discriminant analysis (PLDA) method to calculate the similarity of the embedded layer features between the person to be verified and the registered person.
5. The lightweight identity recognition method combining voiceprint and earprint features according to claim 1 is characterized by: The lightweight identity recognition model needs to be pre-trained. The pre-training includes two models. One is a delayed neural network using ECAPA (Emphasized Channel Attention, Propagation and Aggregation) as pre-training model 1, and the other is the lightweight identity recognition model described in claim 1 as pre-training model 2. The training process of the two pre-training models is as follows: Acquire training sample speech and ear canal echo signals, and label the training sample speech and ear canal echo signals to form labeled training samples; Extracting 13-dimensional Mel-frequency cepstral coefficients of each labeled speech and ear canal echo signal, and fusing the Mel-frequency cepstral feature coefficients of the two, that is, obtaining the fusion feature of voiceprint and earprint; The fused features of all the labeled voiceprints and earprints are input into the pre-trained model 1 and the pre-trained model 2 respectively; The total loss function of pre-trained model 1 and pre-trained model 2 is set as follows: in are the feature vectors output by pre-trained model 1 and pre-trained model 2, and score 1 and score 2 are the class probability scores of the output features of pre-trained model 1 and pre-trained model 2 respectively; N represents the number of samples, M represents the total number of categories; y k is a sign function, which takes 1 if the true category of sample i is equal to c, otherwise it takes 0; is the predicted probability value that sample i belongs to category c, α (0<α<1) is the weight used to balance the two loss functions; After the pre-trained model is trained, the parameters of the pre-trained model 2 are saved to obtain a trained lightweight identity recognition model.
Citation Information
Patent Citations
Learning assistance method, learning assistance client, and electronic learning device
CN109285098A
Full-connection multi-scale residual network and voiceprint recognition method thereof
CN111833886A
Identity verification method and device and electronic equipment
CN113643707A
Target user confirmation method and device, electronic equipment and readable storage medium
CN117763524A
Cited By
Voice interaction method and system of AI intelligent robot
CN120673768A
