User authentication method and device, equipment and storage medium

Abstract

The invention discloses a user authentication method, device and equipment and a storage medium, and relates to the technical field of user authentication technology, the method is applied to intelligent glasses, and the method specifically comprises the following steps: extracting a feature signal corresponding to a bone voiceprint signal of a current user; the feature signal is input into a preset voiceprint recognition model, the voiceprint feature of the current user is obtained, and the preset voiceprint recognition model is a model obtained by performing supervision training on a preset small model through the voiceprint feature output by a preset large model; and determining the similarity between the voiceprint feature of the registered user and the voiceprint feature of the current user, and determining an authentication result of the current user according to the similarity. The bone voiceprint can be used for user authentication, the influence of environmental noise is small, and the large model is used for supervising and training the small model to construct the preset voiceprint recognition model, so that the recognition performance of the preset voiceprint recognition model is ensured, and the accuracy and reliability of user authentication are improved.

Description

Technical Field

[0001] The present application relates to the technical field of user authentication, and in particular to a user authentication method, apparatus, device and storage medium. Background Art

[0002] In today's digital age, biometric technology continues to evolve, providing new prospects for identity authentication and security. Since everyone has a unique way of pronunciation, tone, speaking speed and accent, this uniqueness provides a convenient and non-invasive way of identity authentication for voiceprint recognition. Especially when applied to augmented reality (AR) devices that lack traditional interaction methods such as mouse, keyboard or touch screen, AR devices can collect the user's voiceprint signal through the built-in microphone to achieve voiceprint authentication.

[0003] However, since AR devices are usually designed to be portable and lightweight, such as smart glasses, the location and performance of the built-in microphones are limited, and it is difficult to effectively isolate background noise like professional recording equipment. In a noisy environment, the voiceprint signal collected by the microphone may be contaminated by noise, which in turn affects the accuracy and reliability of authentication. Summary of the invention

[0004] The main purpose of this application is to provide a user authentication method, which aims to solve the technical problem that the existing method of user authentication based on voiceprint signals on AR devices such as smart glasses is easily subject to noise pollution, affecting the accuracy of user authentication results.

[0005] To achieve the above object, the present application proposes a user authentication method, which is applied to smart glasses and includes:

[0006] Extracting the characteristic signal corresponding to the bone voiceprint signal of the current user;

[0007] Inputting the characteristic signal into a preset voiceprint recognition model to obtain the voiceprint characteristics of the current user, wherein the preset voiceprint recognition model is a model obtained by supervising and training a preset small model through the voiceprint characteristics output by a preset large model;

[0008] Determine the similarity between the voiceprint feature of the registered user and the voiceprint feature of the current user, and determine the authentication result of the current user according to the similarity.

[0009] In one embodiment, the step of extracting the characteristic signal corresponding to the bone voiceprint signal of the current user includes:

[0010] Get the bone voiceprint signal of the current user;

[0011] Segmenting the bone voiceprint signal by a voice activity detection algorithm to obtain a voice signal, wherein the bone voiceprint signal is collected by a sensor installed at the nose pad of the smart glasses;

[0012] The speech signal is subjected to Fbank feature conversion to obtain a feature signal corresponding to the bone voiceprint signal.

[0013] In one embodiment, the step of inputting the characteristic signal into a preset voiceprint recognition model to obtain the voiceprint characteristics of the current user includes:

[0014] Inputting the characteristic signal into a preset voiceprint recognition model, wherein the preset voiceprint recognition model includes a plurality of convolutional layers, a global pooling layer, and a fully connected layer;

[0015] Extracting features from the feature signal through each of the convolutional layers to obtain high-level speech features;

[0016] Performing global pooling on the high-level speech features through the global pooling layer to obtain a speech feature vector;

[0017] The voice feature vector is linearly mapped through the fully connected layer to obtain the current user's voiceprint feature.

[0018] In one embodiment, the step of determining the similarity between the voiceprint feature of the registered user and the voiceprint feature of the current user, and determining the authentication result of the current user according to the similarity includes:

[0019] Calculating the cosine similarity between the registered user's voiceprint feature and the current user's voiceprint feature;

[0020] When the cosine similarity is greater than a preset threshold, it is determined that the current user authentication is passed;

[0021] When the cosine similarity is not greater than the preset threshold, it is determined that the current user authentication fails.

[0022] In one embodiment, before the step of calculating the cosine similarity between the registered user's voiceprint feature and the current user's voiceprint feature, the step further includes:

[0023] Obtaining a bone voiceprint registration signal generated by a registered user based on a preset registration guide word;

[0024] Extracting a registration feature signal corresponding to the bone voiceprint registration signal;

[0025] The registered characteristic signal is input into the preset voiceprint recognition model to obtain the registered user's voiceprint characteristic.

[0026] In one embodiment, the method further comprises:

[0027] Acquire a training sample set, wherein the training sample set includes microphone speech signal samples and bone voiceprint signal samples;

[0028] The preset large model is trained by using the microphone voice signal sample to obtain a first voiceprint recognition model;

[0029] The bone voiceprint signal samples are respectively input into the first voiceprint recognition model and the preset small model, and the preset small model is distilled using the first voiceprint recognition model to obtain a preset voiceprint recognition model.

[0030] In one embodiment, the step of inputting the bone voiceprint signal sample into the first voiceprint recognition model and the preset small model respectively, and using the first voiceprint recognition model to distill the preset small model to obtain the preset voiceprint recognition model includes:

[0031] Inputting the bone voiceprint signal sample into the first voiceprint recognition model and outputting soft label features;

[0032] Input the bone voiceprint signal sample into the preset small model, and output the predicted voiceprint feature;

[0033] Based on the loss function, determining the distillation loss according to the soft label features and the predicted voiceprint features;

[0034] The preset small model is adjusted based on the distillation loss, and when the loss function condition is met, a preset voiceprint recognition model is obtained.

[0035] In addition, to achieve the above purpose, the present application also proposes a user authentication device, the device comprising:

[0036] A signal acquisition module, used to extract a characteristic signal corresponding to the bone voiceprint signal;

[0037] A feature extraction module, used for inputting the feature signal into a preset voiceprint recognition model to obtain the voiceprint feature of the current user, wherein the preset voiceprint recognition model is a model obtained by supervised training of a preset small model through the voiceprint feature output by a preset large model;

[0038] The user authentication module is used to determine the similarity between the voiceprint feature of the registered user and the voiceprint feature of the current user, and determine the authentication result of the current user according to the similarity.

[0039] In addition, to achieve the above-mentioned purpose, the present application also proposes a user authentication device, which includes: a memory, a processor, and a user authentication program stored in the memory and executable on the processor, and when the user authentication program is executed by the processor, the steps of the user authentication method described above are implemented.

[0040] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, on which a user authentication program is stored, and when the user authentication program is executed by a processor, the steps of the user authentication method described above are implemented.

[0041] The present application discloses a user authentication method, which is applied to smart glasses, including: extracting a characteristic signal corresponding to the bone voiceprint signal of the current user; inputting the characteristic signal into a preset voiceprint recognition model to obtain the voiceprint feature of the current user, wherein the preset voiceprint recognition model is a model obtained by supervised training of a preset small model through the voiceprint feature output by a preset large model; determining the similarity between the voiceprint feature of the registered user and the voiceprint feature of the current user, and determining the authentication result of the current user based on the similarity. Since the present application can use bone voiceprints for user authentication, it is less affected by environmental noise than voiceprint signals transmitted through the air, and uses a large model to supervise the training of the small model, the small model can learn the high-level feature representation of the large model while maintaining a small volume, thereby improving the efficiency and performance of the preset voiceprint recognition model, which is conducive to improving the accuracy and reliability of user authentication. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0043] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0044] Figure 1 This is a flowchart of the first embodiment of the user authentication method of this application;

[0045] Figure 2 This is a schematic diagram of the entire authentication process for this application user authentication method.

[0046] Figure 3 This is a flow chart of the second embodiment of the user authentication method of this application;

[0047] Figure 4 This is a schematic diagram of the structure of the preset large model in the user authentication method of this application;

[0048] Figure 5 This is a schematic diagram of the structure of the preset small model in the user authentication method of this application;

[0049] Figure 6This is a schematic diagram of the module structure of the user authentication device for this application;

[0050] Figure 7 This is a schematic diagram of the device structure of the hardware operating environment involved in the user authentication method of this application.

[0051] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0052] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0053] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0054] The present application embodiment provides a user authentication method, referring to Figure 1 , Figure 1 This is a flowchart of the first embodiment of the user authentication method of the present application. In this embodiment, the method includes: Steps S10 to S30:

[0055] Step S10: extracting the characteristic signal corresponding to the bone voiceprint signal of the current user.

[0056] It should be noted that the method of this embodiment can be applied to the voice interaction scenario of the AR device for user authentication or payment authentication, and can also be applied to other scenarios that require user authentication, and this embodiment does not limit this. The execution subject of the method of this embodiment can be a computer service device with data processing, program running and network communication, such as a head-mounted AR device. The following takes the smart glasses in the head-mounted AR device as an example to illustrate this embodiment and the following embodiments.

[0057] It should be understood that the bone voiceprint signal is a vibration signal of the bone collected by the sensor. Bone vibration has a strong anti-interference ability to air noise. Using the bone voiceprint signal as the input signal for user authentication can reduce the impact of ambient sound on voiceprint recognition. In some embodiments, after the bone voiceprint signal of the current user is obtained by the bone voiceprint signal sensor, the characteristic signal corresponding to the bone voiceprint signal of the current user is extracted.

[0058] It is understandable that voice activity detection (VAD) can be used to process the bone voiceprint signal to accurately locate the start and end points of the speech to identify the speech activity segment in the bone voiceprint. It can also be further converted into Mel filterbanks (Fbank) features to better express the characteristics of the bone voiceprint signal and achieve signal preprocessing before inputting into the preset voiceprint recognition model.

[0059] Further, in order to illustrate the specific process of preprocessing the bone voiceprint signal to obtain the corresponding characteristic signal, step S10 specifically includes steps S101 to S103:

[0060] Step S101: Acquire the bone voiceprint signal of the current user.

[0061] It should be noted that the bone voiceprint signal can be collected by a sensor installed at the nose pad of the smart glasses. Since the sensor is installed at the nose pad, it can be close to the user's nose bridge when the current user uses the smart glasses, thereby being able to collect reliable and accurate bone voiceprint signals. And compared with the traditional bone voiceprint signal collection device set at the auricle position (for example, headphones paired with smart glasses, or set at the end of the temples of smart glasses), it can improve the device integration or reduce the pressure on the ears of users when wearing the smart glasses, thereby improving the user experience.

[0062] Step S102: segmenting the bone voiceprint signal using a voice activity detection algorithm to obtain a voice signal.

[0063] Step S103: Perform Fbank (Filter Bank) feature conversion on the speech signal to obtain a feature signal corresponding to the bone voiceprint signal.

[0064] In the specific implementation, when the current user wears the smart glasses, the sensor installed at the nose pad collects the bone voiceprint signal, and the VAD algorithm is used to perform endpoint detection on the bone voiceprint signal to find the starting point and end point of the speech segment in the bone voiceprint signal, and then intercept the voice signal. Then, the intercepted voice signal is converted to Fbank features to obtain a feature signal with a fixed feature length (for example, a feature length of 80).

[0065] Step S20: Input the characteristic signal into a preset voiceprint recognition model to obtain the voiceprint characteristics of the current user, wherein the preset voiceprint recognition model is a model obtained by supervised training of a preset small model using the voiceprint characteristics output by a preset large model.

[0066] It should be understood that the preset large model can be a model with higher performance and more complex structure, and trained on general voiceprint data. The preset small model can be a lightweight model with fewer parameters and model levels, which can be deployed and used on smart glasses with limited resources.

[0067] It should be noted that the training sample set can be simultaneously input into the above-mentioned preset large model and the preset small model, the output of the above-mentioned preset large model can be used to perform supervised training on the preset small model, and the final preset voiceprint recognition model can be obtained through distillation learning of the preset large model on the preset small model.

[0068] Further, in order to illustrate the specific process of extracting the voiceprint feature of the above-mentioned characteristic signal through the preset voiceprint feature, step S20 specifically includes steps S201 to S204:

[0069] Step S201: input the feature signal into a preset voiceprint recognition model, which includes several convolutional layers, a global pooling layer and a fully connected layer.

[0070] It should be understood that the preset voiceprint recognition model is based on the lightweight structure of the above-mentioned preset small model, which can be composed of, for example, 5 convolution units, 1 global pooling layer and 1 fully connected layer.

[0071] Step S202: extract features from feature signals through each convolutional layer to obtain high-level speech features.

[0072] It should be understood that the above five convolution units can continuously reduce the time length and feature length of the input feature signal (Fbank feature signal) to obtain high-level speech features.

[0073] Step S203: globally pooling the high-level speech features through the global pooling layer to obtain a speech feature vector.

[0074] It should be understood that the above-mentioned high-level speech features can be globally pooled through the global pooling layer, and the mean and variance are calculated over the time length to obtain a one-dimensional speech feature vector.

[0075] Step S204: linearly map the speech feature vector through a fully connected layer to obtain the voiceprint feature of the current user.

[0076] It should be understood that the one-dimensional speech feature vector is linearly mapped through a fully connected layer and activated using a nonlinear function, and finally a high-dimensional (eg, 192-dimensional) voiceprint feature, namely the current user voiceprint feature, is obtained.

[0077] Step S30: Determine the similarity between the voiceprint feature of the registered user and the voiceprint feature of the current user, and determine the authentication result of the current user based on the similarity.

[0078] It should be understood that the registered user voiceprint feature may be a registered user voiceprint feature pre-stored on the smart glasses. In order to determine whether the current user is a registered user of the smart glasses, the registered user voiceprint feature may be compared with the current user voiceprint feature.

[0079] It can be understood that during the registration stage, the bone voiceprint registration signal generated by the registered user based on the preset registration guide word can be first obtained, and then the registration feature signal corresponding to the bone voiceprint registration signal can be extracted, and finally the registration feature signal can be input into the preset voiceprint recognition model to obtain the voiceprint feature of the registered user.

[0080] It should be noted that the above-mentioned preset registration guide words can be one or more predefined words and phrases displayed to the registered user through the smart glasses. When the registered user reads these guide words, the corresponding bone voiceprint registration signal can be collected through the sensor installed at the nose pad of the smart glasses, and the corresponding registration feature signal can be extracted to input the preset voiceprint recognition model to obtain the corresponding registration feature signal.

[0081] In a specific implementation, the cosine similarity between the voiceprint feature of the registered user and the voiceprint feature of the current user is calculated; when the cosine similarity is greater than a preset threshold, the current user authentication is determined to be successful; when the cosine similarity is not greater than the preset threshold, the current user authentication is determined to be unsuccessful. The preset threshold may be a pre-set similarity threshold for determining whether the current user and the registered user are the same person. For example, after obtaining the voiceprint feature of the registered user, the cosine similarity between the voiceprint feature of the registered user and the voiceprint feature of the current user is calculated, and then the cosine similarity is compared with the preset threshold, and then the magnitude relationship between the cosine similarity and the preset threshold is determined.

[0082] In addition, you can refer to Figure 2 A full description of the user authentication method used in smart glasses. Figure 2 This is a schematic diagram of the entire authentication process for the user authentication method of this application.

[0083] exist Figure 2In the process, the smart glasses obtain the bone voiceprint signal of the current user, first perform VAD endpoint detection on it to obtain the voice signal; then convert the voice signal into Fbank feature, that is, obtain the feature signal corresponding to the voice signal; then input the feature signal (Fbank feature) into the preset voiceprint recognition model, so as to output the voiceprint feature of the current user; then calculate the similarity between the voiceprint feature of the current user and the voiceprint feature of the registered user. When the similarity is higher than the similarity threshold, the current user and the registered user are the same person, that is, the authentication is passed; otherwise, the current user and the registered user are not the same person, that is, the authentication is failed.

[0084] This embodiment extracts the characteristic signal corresponding to the bone voiceprint signal of the current user; inputs the characteristic signal into a preset voiceprint recognition model to obtain the voiceprint feature of the current user, wherein the preset voiceprint recognition model is a model obtained by supervised training of a preset small model using the voiceprint feature output by a preset large model; determines the similarity between the voiceprint feature of the registered user and the voiceprint feature of the current user, and determines the authentication result of the current user based on the similarity. Since this embodiment can use bone voiceprints for user authentication, it is less affected by environmental noise than voiceprint signals transmitted through the air, and uses a large model to supervise the training of the small model, the small model can learn the high-level feature representation of the large model while maintaining a small volume, thereby improving the efficiency and performance of the preset voiceprint recognition model, which is conducive to improving the accuracy and reliability of user authentication.

[0085] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can be referred to the above introduction, and will not be repeated in the following. Figure 3 , Figure 3 This is a flow chart of the second embodiment of the user authentication method of the present application.

[0086] In this embodiment, in order to specifically illustrate the training process of the preset voiceprint recognition model, the method further includes: Steps S01 to S03:

[0087] Step S01: obtaining a training sample set, wherein the training sample set includes microphone speech signal samples and bone voiceprint signal samples.

[0088] It should be understood that training samples from different users may be collected, which may include a large number of microphone speech signal samples and a small number of bone voiceprint signal samples.

[0089] It should be noted that the microphone voice signal samples can be directly obtained from the open source general voice data set on the Internet, and in order to ensure the standardization of microphone voice signal samples obtained from different sources, they can be processed such as timestamp alignment or data cleaning to obtain a large number of microphone voice signal samples.

[0090] It should be understood that since there are few general bone voiceprint data sets open sourced on the Internet, an acceleration sensor can be used to collect bone voiceprint signals of different users in different scenarios, thereby obtaining a small number of bone voiceprint signal samples.

[0091] It should also be noted that after obtaining the above-mentioned microphone speech signal samples and bone voiceprint signal samples, each training sample needs to be converted to Fbank to ensure that each training sample used for subsequent training has Fbank features.

[0092] Step S02: Train a preset large model using microphone voice signal samples to obtain a first voiceprint recognition model.

[0093] It is understandable that the preset large model can be constructed based on the Enhanced Residual Network (E-ResNet) model, and pre-trained with the above-mentioned large number of microphone voice signal samples to obtain the first voiceprint recognition model.

[0094] Further, please refer to Figure 4 The process of using the preset large model to extract voiceprints from the input Fbank features is described in detail. Figure 4 This is a schematic diagram of the structure of the preset large model in the user authentication method of this application.

[0095] exist Figure 4 In the paper, based on E-ResNet, the time length and feature length of the input Fbank feature are continuously reduced through four stages (Stage1~Stage4) of convolution units to obtain high-level speech features, and the attention feature fusion (Attentional Feature Fusion) is used to step-by-step fuse the features of each group to enhance the local information connection and obtain more fine-grained features. Then a more comprehensive feature representation is formed, and the fixed-dimensional (192-dimensional) speaker embedding feature is obtained through the global pooling layer and the fully connected layer (Pooling+fc).

[0096] Step S03: input the bone voiceprint signal samples into the first voiceprint recognition model and the preset small model respectively, and use the first voiceprint recognition model to distill the preset small model to obtain the preset voiceprint recognition model.

[0097] It should be understood that the preset small model can use the lightweight MobileOne model, which can also be referenced here Figure 5 The process of using the preset small model to extract voiceprints from the input Fbank features is described in detail. Figure 5 This is a schematic diagram of the structure of the preset small model in the user authentication method of this application.

[0098] exist Figure 5 In the method, based on Mobileone, the time length and feature length of the input Fbank feature are continuously reduced through 5 stages (Stage0~Stage4) of convolutional units to obtain high-level speech features; finally, the fixed-dimensional (192-dimensional) voiceprint feature (speaker embedding) is obtained through the global pooling layer and the fully connected layer (Pooling+fc).

[0099] Furthermore, the model training process can also be performed on a cloud server associated with the smart glasses. To specifically illustrate the process of distilling the preset small model using the first voiceprint recognition model, step S03 specifically includes: steps S031 to S034:

[0100] Step S031: input the bone voiceprint signal sample into the first voiceprint recognition model, and output the soft label feature.

[0101] It should be understood that the bone voiceprint signal sample (Fbank feature) is input into the first voiceprint recognition model, and the voiceprint feature (speaker embedding) is output, which is the soft label feature. It can then be converted into Logits (class prediction raw score), and a Softmax layer is used to convert Logits into probability distribution and calculate the classification loss.

[0102] It should also be noted that the classification function used to calculate the classification loss can be additive margin softmax (AM-Softmax), and its calculation process is as follows:

[0103]

[0104] Where N is the total number of categories, s is the scaling factor used to stretch the classification boundary, and m is the angle interval parameter used to further separate the boundaries of different categories. You can customize the settings according to the task requirements.

[0105] Step S032: Input the bone voiceprint signal sample into a preset small model and output the predicted voiceprint feature.

[0106] In the specific implementation, the bone voiceprint signal sample (Fbank feature) is input into the preset small model, the predicted voiceprint feature (speaker embedding) is output, and the predicted voiceprint feature is converted into Logits (class prediction raw score).

[0107] Step S033: Based on the loss function, the distillation loss is determined according to the soft label features and the predicted voiceprint features.

[0108] It should be understood that the cosine similarity between the soft label features output by the preset large model (the first voiceprint recognition model) and the predicted voiceprint features output by the preset small model can be calculated, and the cosine similarity can be used as the distillation loss. The calculation process is as follows:

[0109]

[0110] In the formula, x t is the i-th output element in the soft label feature output by the preset large model, x s It is the i-th output element in the predicted voiceprint feature output by the preset small model.

[0111] Step S034: adjusting the preset small model based on the distillation loss, and obtaining the preset voiceprint recognition model when the loss function condition is met.

[0112] It should be understood that the distillation loss measures the inconsistency between the output of the preset small model and the output of the preset large model (the first voiceprint recognition model), so the preset small model can be optimized by the distillation loss.

[0113] Specifically, back propagation can be performed in combination with the above-mentioned classification loss and distillation loss to adjust the parameters of the preset small model until the distillation loss reaches an acceptably low level, or when a specific stopping condition is met, the training is completed to obtain the preset voiceprint recognition model.

[0114] It should also be noted that the above back propagation can be performed based on the final loss function, which can be obtained by combining the classification loss and the distillation loss, which can be expressed as:

[0115] L combine =L m +100*L d

[0116] It should be understood that during the entire training process, AM-Softmax loss is mainly used to train the preset large model to ensure that it has good classification performance. The cosine similarity distillation loss is used to train the preset small model so that it can learn the knowledge of the preset large model and ensure that the structure of the preset small model may be simpler. Through the above training process based on distillation learning, the preset small model can imitate the behavior of the preset large model as much as possible while maintaining a small model size, thereby achieving efficient knowledge transfer.

[0117] In this embodiment, a training sample set is obtained, which includes microphone voice signal samples and bone voiceprint signal samples; the preset large model is trained by the microphone voice signal samples to obtain the first voiceprint recognition model; the bone voiceprint signal samples are respectively input into the first voiceprint recognition model and the preset small model, and the preset small model is distilled using the first voiceprint recognition model to obtain the preset voiceprint recognition model. In this embodiment, the distillation process realizes the knowledge transfer from the large model to the small model, so that the small model can inherit the recognition ability of the large model, and can perform well even when there is less data, while maintaining the model size and efficiency while achieving a high recognition accuracy. The distilled small model, that is, the preset voiceprint recognition model, can achieve fast response on smart glasses, thereby meeting the needs of real-time user authentication.

[0118] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the user authentication method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0119] The present application also provides a user authentication device, referring to Figure 6 , Figure 6 This is a schematic diagram of the module structure of the user authentication device of this application, and the user authentication device includes:

[0120] The signal acquisition module 601 is used to extract the characteristic signal corresponding to the bone voiceprint signal of the current user;

[0121] The feature extraction module 602 is used to input the feature signal into a preset voiceprint recognition model to obtain the voiceprint feature of the current user, wherein the preset voiceprint recognition model is a model obtained by supervising and training a preset small model through the voiceprint feature output by a preset large model;

[0122] The user authentication module 603 is used to determine the similarity between the voiceprint feature of the registered user and the voiceprint feature of the current user, and determine the authentication result of the current user according to the similarity.

[0123] Furthermore, the signal acquisition module 601 is also used to obtain the bone voiceprint signal of the current user; segment the bone voiceprint signal through a voice activity detection algorithm to obtain a voice signal, wherein the bone voiceprint signal is collected by a sensor installed at the nose pad of the smart glasses; perform Fbank feature conversion on the voice signal to obtain a feature signal corresponding to the bone voiceprint signal.

[0124] Furthermore, the feature extraction module 602 is also used to input the feature signal into a preset voiceprint recognition model, which includes several convolution layers, a global pooling layer and a fully connected layer; extract the feature signal through each convolution layer to obtain high-level speech features; globally pool the high-level speech features through the global pooling layer to obtain a speech feature vector; linearly map the speech feature vector through the fully connected layer to obtain the current user's voiceprint features.

[0125] Furthermore, the user authentication module 603 is also used to calculate the cosine similarity between the voiceprint feature of the registered user and the voiceprint feature of the current user; when the cosine similarity is greater than a preset threshold, it is determined that the current user authentication is passed; when the cosine similarity is not greater than the preset threshold, it is determined that the current user authentication is not passed.

[0126] Furthermore, the user authentication module 603 is also used to obtain the bone voiceprint registration signal generated by the registered user based on the preset registration guide word; extract the registration feature signal corresponding to the bone voiceprint registration signal; input the registration feature signal into the preset voiceprint recognition model to obtain the voiceprint feature of the registered user.

[0127] Furthermore, the feature extraction module 602 is also used to obtain a training sample set, which includes microphone voice signal samples and bone voiceprint signal samples; train the preset large model through the microphone voice signal samples to obtain a first voiceprint recognition model; input the bone voiceprint signal samples into the first voiceprint recognition model and the preset small model respectively, and use the first voiceprint recognition model to distill the preset small model to obtain a preset voiceprint recognition model.

[0128] Furthermore, the feature extraction module 602 is also used to input the bone voiceprint signal sample into the first voiceprint recognition model and output the soft label feature; input the bone voiceprint signal sample into the preset small model and output the predicted voiceprint feature; based on the loss function, determine the distillation loss according to the soft label feature and the predicted voiceprint feature; adjust the preset small model based on the distillation loss, and obtain the preset voiceprint recognition model when the loss function conditions are met.

[0129] This embodiment can use bone voiceprints for user authentication, which is less affected by environmental noise than airborne voiceprint signals. The large model is used to supervise the training of the small model, so that the small model can learn the high-level feature representation of the large model while maintaining a small volume, thereby improving the efficiency and performance of the preset voiceprint recognition model, which is beneficial to improving the accuracy and reliability of user authentication.

[0130] The present application also provides a user authentication device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the user authentication method in the above-mentioned embodiment one.

[0131] Reference below Figure 7 , Figure 7 The device structure diagram of the hardware operating environment involved in the user authentication method of the present application. The user authentication device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The user authentication device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0132] like Figure 7As shown, the user authentication device may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the user authentication device are also stored. The processing device 1001, ROM1002, and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the user authentication device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a user authentication device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0133] The present application also provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, wherein the computer-readable program instructions are used to execute the user authentication method in the above-mentioned embodiment.

[0134] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0135] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned user authentication method, and can solve the technical problem of user authentication. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the user authentication method provided by the above-mentioned embodiment, and will not be repeated here.

[0136] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A user authentication method, characterized in that: The method is applied to smart glasses, and the method comprises: Extracting the characteristic signal corresponding to the bone voiceprint signal of the current user; Inputting the characteristic signal into a preset voiceprint recognition model to obtain the voiceprint characteristics of the current user, wherein the preset voiceprint recognition model is a model obtained by supervising and training a preset small model through the voiceprint characteristics output by a preset large model; Determine the similarity between the voiceprint feature of the registered user and the voiceprint feature of the current user, and determine the authentication result of the current user according to the similarity.

2. The method according to claim 1, characterized in that The step of extracting the characteristic signal corresponding to the bone voiceprint signal of the current user comprises: Get the bone voiceprint signal of the current user; Segmenting the bone voiceprint signal by a voice activity detection algorithm to obtain a voice signal, wherein the bone voiceprint signal is collected by a sensor installed at the nose pad of the smart glasses; The speech signal is subjected to Fbank feature conversion to obtain a feature signal corresponding to the bone voiceprint signal.

3. The method according to claim 2, characterized in that The step of inputting the characteristic signal into a preset voiceprint recognition model to obtain the voiceprint characteristics of the current user includes: Inputting the characteristic signal into a preset voiceprint recognition model, wherein the preset voiceprint recognition model includes a plurality of convolutional layers, a global pooling layer, and a fully connected layer; Extracting features from the feature signal through each of the convolutional layers to obtain high-level speech features; Performing global pooling on the high-level speech features through the global pooling layer to obtain a speech feature vector; The voice feature vector is linearly mapped through the fully connected layer to obtain the current user's voiceprint feature.

4. The method according to claim 1, characterized in that The step of determining the similarity between the voiceprint feature of the registered user and the voiceprint feature of the current user, and determining the authentication result of the current user according to the similarity comprises: Calculating the cosine similarity between the registered user's voiceprint feature and the current user's voiceprint feature; When the cosine similarity is greater than a preset threshold, it is determined that the current user authentication is passed; When the cosine similarity is not greater than the preset threshold, it is determined that the current user authentication fails.

5. The method according to claim 4, characterized in that Before the step of calculating the cosine similarity between the registered user's voiceprint feature and the current user's voiceprint feature, the method further includes: Obtaining a bone voiceprint registration signal generated by a registered user based on a preset registration guide word; Extracting a registration feature signal corresponding to the bone voiceprint registration signal; The registered characteristic signal is input into the preset voiceprint recognition model to obtain the registered user's voiceprint characteristic.

6. The method according to claim 1, characterized in that The method further comprises: Acquire a training sample set, wherein the training sample set includes microphone speech signal samples and bone voiceprint signal samples; The preset large model is trained by using the microphone voice signal sample to obtain a first voiceprint recognition model; The bone voiceprint signal samples are respectively input into the first voiceprint recognition model and the preset small model, and the preset small model is distilled using the first voiceprint recognition model to obtain a preset voiceprint recognition model.

7. The method according to claim 6, characterized in that The step of inputting the bone voiceprint signal sample into the first voiceprint recognition model and the preset small model respectively, and using the first voiceprint recognition model to distill the preset small model to obtain the preset voiceprint recognition model includes: Inputting the bone voiceprint signal sample into the first voiceprint recognition model and outputting soft label features; Input the bone voiceprint signal sample into the preset small model, and output the predicted voiceprint feature; Based on the loss function, determining the distillation loss according to the soft label features and the predicted voiceprint features; The preset small model is adjusted based on the distillation loss, and when the loss function condition is met, a preset voiceprint recognition model is obtained.

8. A user authentication device, characterized in that: The device comprises: A signal acquisition module, used to extract a characteristic signal corresponding to the bone voiceprint signal; A feature extraction module, used for inputting the feature signal into a preset voiceprint recognition model to obtain the voiceprint feature of the current user, wherein the preset voiceprint recognition model is a model obtained by supervised training of a preset small model through the voiceprint feature output by a preset large model; The user authentication module is used to determine the similarity between the voiceprint feature of the registered user and the voiceprint feature of the current user, and determine the authentication result of the current user according to the similarity.

9. A user authentication device, characterized in that: The user authentication device comprises: a memory, a processor, and a user authentication program stored in the memory and executable on the processor, wherein the user authentication program implements the steps of the user authentication method according to any one of claims 1 to 7 when executed by the processor.

10. A storage medium, characterized in that: The storage medium stores a user authentication program, and when the user authentication program is executed by the processor, the steps of the user authentication method according to any one of claims 1 to 7 are implemented.

Images