Authentication method, device and electronic device
By combining the vocalprint features and the external auditory canal acoustic reflex features, the problem of low accuracy of voiceprint recognition in the prior art is solved, and the accuracy and user experience of identity verification are improved.
Patent Information
- Application Number
- CN202010328026.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-04-23
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2040-04-23
AI Technical Summary
The verification accuracy of the authenticity authentication scheme based on voiceprint recognition in the prior art is low, especially in noise and reverberation scenarios, resulting in a decrease in user experience.
An authentication method combining voiceprint features and external auditory canal acoustic reflex features is adopted to generate audio verification information through the voiceprint features and external auditory canal acoustic reflex features of registered users, and match them during the verification process to improve the accuracy of identity verification.
While maintaining the convenience of voice interaction, the accuracy of authentication is improved, the impact of noise and reverb on authentication is reduced, and the user experience is improved.
Smart Images

Figure CN113643707B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of intelligent terminals, and in particular to an identity authentication method, device and electronic device. Background Art
[0002] In the field of identity authentication technology, a common identity authentication scheme is to use a text password as verification information, and obtain the text password entered by the user through a text input interface during the identity authentication process to perform identity authentication.
[0003] Since text passwords are easy to leak and forget, in many identity authentication application scenarios, the user's biometrics are used as verification information, and identity authentication is implemented based on biometric recognition, thereby improving the security of identity authentication and the user experience of identity authentication applications. For example, identity authentication is based on face recognition, fingerprint recognition, iris recognition, and voiceprint recognition.
[0004] Among the many different biometric recognition solutions, voiceprint recognition is more convenient and easier to implement than other biometric recognition solutions because voice interaction is a contactless and non-directional interaction method, and the hardware requirements for voice collection are very low.
[0005] However, compared with biometric recognition such as face recognition and fingerprint recognition, the accuracy of voiceprint recognition is relatively low. In addition, since voiceprints are extracted from speech, voiceprint recognition is sensitive to the interference of noise and reverberation during speech collection, which further reduces the accuracy of voiceprint recognition in actual applications. Therefore, in the prior art, the verification accuracy of identity authentication schemes based on voiceprint recognition is much lower than that of other identity authentication schemes. In the application scenarios of identity authentication based on voiceprint recognition, legitimate users often need to perform voice verification input operations multiple times before a voiceprint that can pass verification can be extracted, which greatly reduces the user experience. Summary of the invention
[0006] In view of the problem that the authentication scheme based on voiceprint recognition in the prior art has low authentication accuracy, the present application provides an authentication method, device and electronic device, as well as a computer-readable storage medium.
[0007] The present application embodiment adopts the following technical solutions:
[0008] In a first aspect, an embodiment of the present application provides an identity authentication method, including:
[0009] Collecting a voiceprint feature of a user to be verified, where the user to be verified is a user who initiates identity authentication for the first user;
[0010] Collecting the acoustic reflex characteristics of the external auditory canal of the user to be verified;
[0011] Calling the audio verification information of the first user, wherein the audio verification information of the first user is the audio verification information registered according to the voiceprint characteristics and external auditory canal acoustic reflection characteristics of the first user;
[0012] Verify whether the user to be verified is the first user based on the voiceprint characteristics, external auditory canal acoustic reflection characteristics of the user to be verified and the audio verification information of the first user.
[0013] In a feasible implementation manner of the first aspect above, the audio verification information of the first user includes a voiceprint feature, an external auditory canal acoustic reflection feature, and a matching result verification standard of the first user;
[0014] The verifying whether the user to be verified is the first user according to the voiceprint feature, the external auditory canal acoustic reflection feature of the user to be verified and the audio verification information of the first user includes:
[0015] Match the voiceprint features of the user to be verified with the voiceprint features of the first user to generate a first matching result, match the external auditory canal acoustic reflex features of the user to be verified with the external auditory canal acoustic reflex features of the first user to generate a second matching result, merge the first matching result and the second matching result to generate a matching result to be verified, and determine whether the matching result to be verified meets the verification standard based on the matching result verification standard.
[0016] In a feasible implementation manner of the first aspect, the audio verification information of the first user includes an audio feature of the first user and an audio feature matching determination criterion, and the audio feature of the first user is generated by fusing a voiceprint feature and an external auditory canal acoustic reflection feature of the first user;
[0017] The verifying whether the user to be verified is the first user according to the voiceprint feature, the external auditory canal acoustic reflection feature of the user to be verified and the audio verification information of the first user includes:
[0018] The voiceprint feature and the external auditory canal acoustic reflection feature of the user to be verified are integrated to generate the audio feature of the user to be verified, and whether the audio feature of the user to be verified matches the audio feature of the first user is determined based on the audio feature matching judgment standard.
[0019] In a feasible implementation of the first aspect above, for different users, the similarity between the audio features is lower than the similarity between the voiceprint features, and the similarity between the audio features is lower than the similarity between the external auditory canal acoustic reflection features.
[0020] In a feasible implementation manner of the first aspect, the audio feature is generated by fusing the voiceprint feature and the external auditory canal acoustic reflection feature, wherein the process of fusing the voiceprint feature and the external auditory canal acoustic reflection feature includes:
[0021] Performing feature splicing on the voiceprint feature and the external auditory canal acoustic reflection feature to obtain a feature sample set;
[0022] Based on a linear discriminant analysis algorithm, projecting the feature sample set in a first dimension to obtain a first dimension projection result, wherein for different users, the similarity between the first dimension projection results is lower than the similarity between the projection results in other dimensions;
[0023] The first dimensional projection result is used as the audio feature.
[0024] In a feasible implementation manner of the first aspect above, the voiceprint features include air conduction voiceprint features and / or bone conduction voiceprint features.
[0025] In a feasible implementation manner of the first aspect, collecting the acoustic reflection characteristics of the external auditory canal of the user to be verified includes:
[0026] Playing incident sound to the external auditory canal of the user to be authenticated;
[0027] Picking up the reflected sound generated by the incident sound being reflected through the external auditory canal of the user to be authenticated, and generating a reflected sound digital signal;
[0028] The reflected sound digital signal is converted into the external auditory canal acoustic reflection feature.
[0029] In a feasible implementation manner of the first aspect, converting the reflected sound digital signal into the external auditory canal acoustic reflection feature includes:
[0030] Extracting an external auditory canal reflected sound amplitude spectrum of the reflected sound digital signal as the external auditory canal acoustic reflection feature;
[0031] or,
[0032] The acoustic reflection transfer function of the external auditory canal is calculated, and the acoustic reflection transfer function is used as the acoustic reflection feature of the external auditory canal.
[0033] In a feasible implementation of the first aspect above, the incident sound is a single-frequency signal, and the reflected sound generated by the incident sound being reflected through the external auditory canal of the user to be verified is picked up, wherein the reflected sound is picked up by scanning a frequency range corresponding to the incident sound.
[0034] In a second aspect, an embodiment of the present application provides a method for registering audio verification information, including:
[0035] Collecting the voiceprint features of the first user;
[0036] collecting external auditory canal acoustic reflection characteristics of the first user;
[0037] The audio verification information of the first user is registered according to the voiceprint feature and the external auditory canal acoustic reflection feature of the first user.
[0038] In a third aspect, an embodiment of the present application provides an identity authentication device, including:
[0039] A first collection module, which is used to collect voiceprint features of a user to be verified, where the user to be verified is a user who initiates identity authentication for the first user;
[0040] A second acquisition module, which is used to acquire the acoustic reflection characteristics of the external auditory canal of the user to be verified;
[0041] A verification information calling module, which is used to call the audio verification information of the first user, wherein the audio verification information of the first user is the audio verification information registered according to the voiceprint characteristics and external auditory canal acoustic reflection characteristics of the first user;
[0042] A verification module is used to verify whether the user to be verified is the first user based on the voiceprint characteristics, external auditory canal acoustic reflection characteristics of the user to be verified and the audio verification information of the first user.
[0043] In a fourth aspect, an embodiment of the present application provides a device for registering audio verification information, including:
[0044] A first collection module, which is used to collect the voiceprint features of the first user;
[0045] A second acquisition module, which is used to acquire the acoustic reflection characteristics of the external auditory canal of the first user;
[0046] A verification information registration module is used to register the audio verification information of the first user according to the voiceprint characteristics and external auditory canal acoustic reflection characteristics of the first user.
[0047] In a fifth aspect, an embodiment of the present application provides an identity authentication system, including:
[0048] The device for registering audio verification information as described in the fourth aspect above, which is used to register the audio verification information of the first user;
[0049] The identity authentication device as described in the third aspect above is used to verify whether the user to be authenticated is the first user based on the audio verification information of the first user.
[0050] In a sixth aspect, an embodiment of the present application provides an electronic device, comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute the method steps described in the first and second aspects above.
[0051] In a seventh aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the method described in the first and second aspects above.
[0052] According to the above technical solution proposed in the embodiment of the present application, at least the following technical effects can be achieved: According to the method of the embodiment of the present application, identity authentication based on voiceprint features and external auditory canal acoustic reflection features can be realized, eliminating the negative impact of the decline in voiceprint recognition performance in noise and reverberation scenarios on the accuracy of identity authentication when identity authentication is based on voiceprint features alone, and improving the accuracy of identity authentication while maintaining the convenience of identity authentication based on voice interaction. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] Figure 1 Shown is a schematic diagram of the human external auditory canal;
[0054] Figure 2 It is a flow chart of an embodiment of a registration method of audio verification information according to the present application;
[0055] Figure 3 Shown is a flow chart of an embodiment of an identity verification method according to the present application;
[0056] Figure 4 Shown is a logic diagram of scoring identity authentication according to an embodiment of the present application;
[0057] Figure 5 The figure is a schematic diagram showing the effect of performing scoring identity verification according to an embodiment of the present application;
[0058] Figure 6 Shown is a flow chart of a registration method for audio verification information and an identity verification method according to an embodiment of the present application;
[0059] Figure 7 Shown is a logical schematic diagram of identity authentication based on feature fusion according to an embodiment of the present application;
[0060] Figure 8 The figure is a schematic diagram showing the effect of identity authentication based on feature fusion according to an embodiment of the present application;
[0061] Fig. 9Shown is a flow chart of a registration method for audio verification information and an identity verification method according to an embodiment of the present application;
[0062] Fig.10 It is a structural diagram of an embodiment of a registration device for audio verification information according to the present application;
[0063] Fig.11 Shown is a structural diagram of an embodiment of an identity verification device according to the present application;
[0064] Fig.12 Shown is a structural diagram of an embodiment of an identity verification system according to the present application;
[0065] Fig.13 Shown is a structural diagram of an embodiment of an identity verification system according to the present application;
[0066] Fig.14 Shown is a structural diagram of an embodiment of an identity authentication system according to the present application. DETAILED DESCRIPTION
[0067] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.
[0068] The terms used in the implementation section of this application are only used to explain the specific embodiments of this application and are not intended to limit this application.
[0069] In view of the problem that the authentication scheme based on voiceprint recognition in the prior art has a low verification accuracy, the present application provides a new scheme for forming audio verification information. In order to propose the scheme of the present application, the inventor first analyzes the specific application scenarios of the authentication scheme based on voiceprint recognition.
[0070] In the prior art, the implementation process of the identity authentication scheme based on voiceprint recognition is mainly as follows: when user A registers, the voice information of user A is collected, voiceprint A is extracted from the user's voice information and voiceprint A is saved; when a user to be verified wants to verify that he or she is user A, the voice information of the user to be verified is collected, voiceprint B is extracted from the user's voice information, and the extracted voiceprint B is compared with the pre-stored voiceprint A. If voiceprint B is consistent with voiceprint A, the user to be verified passes the identity authentication and is user A.
[0071] Ideally, the voiceprints of the same user in all application scenarios are the same. However, due to the limitations of the voiceprint extraction algorithm, the voiceprints in the voice cannot be perfectly extracted. In addition, due to the presence of noise and reverberation in the voice, there may be large differences between the two voiceprints extracted from two different voices of the same user. This will cause the identity authentication system to determine that the two voiceprints are inconsistent when comparing them, and the two voiceprints do not correspond to the same user, resulting in identity authentication failure.
[0072] One of the feasible solutions to the above problems is to ignore the voiceprint details during the voiceprint extraction process, thereby ignoring the voiceprint differences caused by the voiceprint extraction algorithm and the interference of noise and reverberation in the voice, so that the same voiceprint can be extracted from two different voices of the same user. Alternatively, a comparison rule with a larger fuzziness is used when comparing two voiceprints, for example, a larger threshold range is set, and when the difference between the two voiceprints is within the threshold range, the two voiceprints are considered to be consistent.
[0073] Although the above scheme allows users to successfully implement identity authentication in different application scenarios, when the voiceprint details are ignored, there is a situation where the same voiceprint is extracted from the voices of different users; and when a comparison rule with a large fuzziness is adopted, there is a situation where the voiceprints of different users are judged to correspond to the same user, which in turn reduces the verification accuracy of the identity authentication system.
[0074] In view of the above situation, in a solution of an embodiment of the present application, instead of starting with voiceprint recognition, new verification information is introduced. During the identity authentication process, the voiceprint and the newly introduced verification information are combined to perform identity authentication, thereby improving the accuracy of identity authentication.
[0075] Figure 1 The figure shows a schematic diagram of the human external auditory canal. Figure 1 The external auditory canal 12 shown is a semi-closed passage, and the tympanic membrane 11 is located on the inner side to isolate the external auditory canal 12 from the inner auditory canal. The speaker 13 emits an incident sound 14. When the incident sound 14 is transmitted into the external auditory canal 12, transmission, reflection and absorption phenomena will occur at the tympanic membrane 11. The transmittance, reflectivity and absorptivity of the incident sound 14 are determined by the shape of the external auditory canal 12, the material properties of the tympanic membrane 11 and the frequency of the incident sound 14. Therefore, the individual external auditory canal acoustic reflection characteristics are stable and unique. In addition, based on the microphone 16, the reflected sound 15 generated by the incident sound 14 reflected through the external auditory canal 12 can be collected, so as to calculate and obtain the external auditory canal acoustic reflection characteristics. Therefore, the external auditory canal acoustic reflection characteristics can be used as a biometric feature to characterize identity, and the external auditory canal acoustic reflection characteristics are a stable static biometric feature with good robustness to noise.
[0076] Therefore, in a solution of an embodiment of the present application, the acoustic reflection characteristics of the external auditory canal are introduced on the basis of voiceprint recognition, and the voiceprint features and the acoustic reflection characteristics of the external auditory canal are combined to realize identity authentication.
[0077] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.
[0078] To achieve identity authentication, an embodiment of the present application first proposes a method for registering audio verification information, where a user registers audio verification information when registering an account, so that identity authentication can be performed based on the registered audio verification information when performing identity authentication later.
[0079] Figure 2 FIG. 1 is a flow chart of an embodiment of a registration method for audio verification information according to the present application. Figure 2 As shown, in one embodiment of the present application, when a user registers an account in the identity authentication system, the identity authentication system performs the following steps:
[0080] Step 2010, collecting the voiceprint features of the first user;
[0081] Step 220, collecting the acoustic reflection characteristics of the external auditory canal of the first user;
[0082] Step 2030: register the audio verification information of the first user according to the voiceprint characteristics and external auditory canal acoustic reflection characteristics of the first user.
[0083] Furthermore, based on the registration method of audio verification information proposed in an embodiment of the present application, an embodiment of the present application also proposes an identity authentication method. Figure 3 FIG. 1 is a flow chart of an embodiment of an identity verification method according to the present application. Figure 3 As shown, in one embodiment of the present application, when the user to be authenticated initiates identity authentication for the first user (the user to be authenticated expects to verify whether he is the first user), the identity authentication system performs the following steps:
[0084] Step 3010, collecting the voiceprint feature of the user to be verified, where the user to be verified is the user who initiates identity authentication for the first user, that is, the user to be verified wishes to verify whether he or she is the first user;
[0085] Step 3020, collecting the acoustic reflection characteristics of the external auditory canal of the user to be verified;
[0086] Step 3030: calling the audio verification information of the first user, wherein the audio verification information of the first user is the audio verification information registered according to the voiceprint feature and the acoustic reflection feature of the external auditory canal of the first user, that is, the audio verification information of the first user is based on Figure 1 The audio verification information registered by the method for registering audio verification information of the illustrated embodiment;
[0087] Step 3040: Verify whether the user to be verified is the first user based on the voiceprint characteristics, external auditory canal acoustic reflection characteristics of the user to be verified and the audio verification information of the first user.
[0088] According to the registration method of audio verification information and the identity authentication method of the embodiments of the present application, identity authentication based on voiceprint features and external auditory canal acoustic reflection features can be realized, eliminating the negative impact of the decline in voiceprint recognition performance in noise and reverberation scenarios on the accuracy of identity authentication when identity authentication is based on voiceprint features alone, and improving the accuracy of identity authentication while maintaining the convenience of identity authentication based on voice interaction.
[0089] Furthermore, in actual application scenarios, Figure 2 as well as Figure 3 Each step of the illustrated embodiment may have a variety of different implementation methods.
[0090] Specifically, in an implementation of step 2010 or step 3010, the process of collecting voiceprint features includes:
[0091] Acquire the user's voice digital signal;
[0092] The acquired voice digital signal is converted into voiceprint features based on the voiceprint extraction algorithm.
[0093] In actual application scenarios, voice collection can be achieved through air conduction collection and bone conduction collection. Therefore, in an embodiment of the present application, the voiceprint feature is one of the air conduction voiceprint feature and the bone conduction voiceprint feature.
[0094] Furthermore, the bone conduction microphone can pick up the sound vibrations conducted by the bones, thereby converting the voice vibrations into voice digital signals. After bone conduction, the voice picked up by the bone conduction microphone carries different information from the voice picked up by the air conduction microphone. Therefore, combining the voiceprint features extracted by the bone conduction microphone with the voiceprint features extracted by the air conduction microphone can improve the richness of the voiceprint feature information and thus improve the accuracy of recognition.
[0095] Therefore, in one embodiment of the present application, in an implementation of step 2010 or step 3010, the collected voiceprint features include air conduction voiceprint features and / or bone conduction voiceprint features.
[0096] Specifically, in an implementation of step 2010 or step 3010, the extraction of air conduction voiceprint features includes the following steps:
[0097] The air conduction microphone converts the speaker's voice sound pressure signal into a voice electrical signal and samples it into a voice digital signal;
[0098] Based on the air conduction voiceprint extraction algorithm, the speech digital signal is converted into an air conduction voiceprint feature vector.
[0099] Specifically, in an implementation of step 2010 or step 3010, the extraction of bone conduction voiceprint features includes the following steps:
[0100] The speaker wears a host device equipped with a bone conduction microphone. The speaker's voice is transmitted to the bone conduction microphone through the body. The bone conduction microphone converts the speaker's voice vibration signal into a voice electrical signal and samples it into a bone conduction voice digital signal.
[0101] Based on the bone conduction voiceprint extraction algorithm, the bone conduction voice digital signal is converted into a bone conduction voiceprint feature vector.
[0102] Specifically, in an implementation of step 2020 or step 3020, the process of collecting the external auditory canal acoustic reflection characteristics includes:
[0103] Playing incident sound into the user's external auditory canal;
[0104] Picking up the reflected sound generated by the incident sound reflected by the user's external auditory canal, and sampling the reflected sound into a reflected sound digital signal;
[0105] The reflected sound digital signal is converted into the external auditory canal acoustic reflection characteristics.
[0106] Specifically, in one implementation of step 2020 or step 3020, in the process of collecting the acoustic reflection characteristics of the user's external auditory canal, the incident sound is a single-frequency signal, and the reflected sound generated by the incident sound reflected by the user's external auditory canal is picked up, wherein the reflected sound is picked up by scanning the frequency range corresponding to the incident sound.
[0107] Specifically, in an implementation of step 2020 or step 3020, in the process of collecting the acoustic reflection feature of the external auditory canal, the external auditory canal reflected sound amplitude spectrum of the reflected sound digital signal is extracted as the external auditory canal acoustic reflection feature. Specifically, the reflected sound can be expressed as a convolution of a reflection pulse response function of the external auditory canal and the incident sound, and the amplitude spectrum of the reflected sound can be expressed as the product of the amplitude spectrum of the incident sound and the transfer function of the external auditory canal incident-reflection path.
[0108] r(t)=i(t)*h(t), (1)
[0109] R(w)=I(w)H(w), (2)
[0110] In formulas (1) and (2), i(t) and r(t) are the sound pressure signals of the incident sound and the reflected sound, respectively; I(w) and R(w) are the amplitude spectra of the incident sound and the reflected sound, respectively; h(t) and H(w) are biometric features determined by the shape of the external auditory canal and the material properties of the eardrum. Since the shape of each person's external auditory canal and the material of the eardrum are unique, H(w) = R(w) / I(w) can be used as an identity representation for identity registration and verification. When the incident sound is fixed, R(w) can be directly used as a reflection feature for identity representation.
[0111] Furthermore, in practical application scenarios, the signals actually stored in the circuit are digital signals i'(n) and r'(n), so the gain and sensitivity of the incident sound playback and echo sampling paths should also be considered when calculating H(w). The path transfer function is denoted as H0(w), and H'(w) is the transfer function between i'(n) and r'(n). Then H(w) = H'(w) / H0(w). When the speaker and microphone device are not consistent, the incident sound playback and echo sampling path H0(w) should be estimated before calculating H(w). When the speaker and microphone device are consistent, the transfer function between i(n) and r(n) can be directly used as the external auditory canal acoustic reflection feature. That is, in one implementation of step 2020, in the process of collecting the external auditory canal acoustic reflection feature, the acoustic reflection transfer function of the user's external auditory canal is calculated, and the acoustic reflection transfer function is used as the external auditory canal acoustic reflection feature.
[0112] Furthermore, in one implementation of step 2030, the voiceprint feature and the external auditory canal acoustic reflection feature are used as audio verification information. When performing identity verification, a comparison verification is performed based on the registered voiceprint feature and the external auditory canal acoustic reflection feature, and the comparison verification results are comprehensively compared to determine whether the identity verification is passed.
[0113] Specifically, in an implementation of step 2030, the audio verification information of the first user includes the voiceprint feature, the external auditory canal acoustic reflection feature, and the matching result verification standard of the first user, wherein:
[0114] The matching result verification standard is used to determine whether the matching result to be verified by the user to be verified meets the verification standard;
[0115] The to-be-verified matching result of the to-be-verified user is generated by fusing the matching result of the voiceprint feature of the to-be-verified user with the voiceprint feature of the first user and the matching result of the external auditory canal acoustic reflex feature of the to-be-verified user with the external auditory canal acoustic reflex feature of the first user.
[0116] Correspondingly, in an implementation of step 3040, the process of verifying whether the user to be verified is the first user includes:
[0117] Matching the voiceprint feature of the user to be verified with the voiceprint feature in the audio verification information of the first user to generate a first matching result;
[0118] Matching the external auditory canal acoustic reflection feature of the user to be verified with the external auditory canal acoustic reflection feature in the audio verification information of the first user to generate a second matching result;
[0119] Fusion the first matching result and the second matching result to generate a matching result to be verified;
[0120] Based on the matching result verification standard in the audio verification information of the first user, it is determined whether the matching result to be verified meets the verification standard. If so, it is determined that the user to be verified is the first user.
[0121] Specifically, in an implementation of step 2030, the matching result verification standard is a preset scoring threshold. In an implementation of step 3040, in the process of matching the voiceprint feature of the user to be verified with the voiceprint feature in the audio verification information of the first user and matching the external auditory canal acoustic reflection feature of the user to be verified with the external auditory canal acoustic reflection feature in the audio verification information of the first user, the voiceprint feature and the external auditory canal acoustic reflection feature are scored respectively by scoring, and then the total score is calculated based on the calculated multiple scores, and finally the total score is compared with the scoring threshold for judgment.
[0122] Specifically, in one implementation of step 3040, scoring is performed by similarity calculation, that is, the similarity scores of the external auditory canal acoustic reflection features and voiceprint features of the user to be verified and the external auditory canal acoustic reflection features and voiceprint features in the audio verification information of the first user are calculated respectively, the total score is calculated based on multiple similarity scores, and then the decision is made based on the total score.
[0123] Specifically, in an implementation of step 3040, in the process of calculating the total score based on the scores of the voiceprint features and the external auditory canal acoustic reflection features, multiple scores are generally used as inputs of a fusion function S to calculate the total score. For example:
[0124] S=f(S1,S2,S3). (3)
[0125] In formula (3), S1, S2, and S3 are the scores of the external auditory canal acoustic reflection characteristics, air conduction voiceprint characteristics, and bone conduction voiceprint characteristics, respectively.
[0126] Take simple weighted fusion as an example, for example:
[0127] S=S1+S2+S3. (4)
[0128] Figure 4 FIG. 1 is a logic diagram of performing scoring identity authentication according to an embodiment of the present application. Figure 4As shown, 41, 42, and 43 refer to the registered external auditory canal acoustic reflection features, air conduction voiceprint features, and bone conduction voiceprint features, respectively; 44, 45, and 46 refer to the external auditory canal acoustic reflection features, air conduction voiceprint features, and bone conduction voiceprint features, respectively. The three features of external auditory canal acoustic reflection features, air conduction voiceprint features, and bone conduction voiceprint features are scored separately, that is, 41-44, 42-45, and 43-46 are calculated respectively for similarity scores 47, 48, and 49, and then the fusion score 410 is calculated. Finally, the decision is made based on the fusion score 410 and the scoring threshold.
[0129] Figure 5 The figure shows the effect of scoring identity verification according to an embodiment of the present application. Assume that an imposter wants to impersonate user A to pass identity verification. Figure 5 As shown, 51 refers to the voiceprint feature score of the counterfeiter when performing identity authentication, 52 refers to the voiceprint feature score of user A when performing identity authentication, 53 refers to the external auditory canal acoustic reflex feature score of the counterfeiter when performing identity authentication, 54 refers to the external auditory canal acoustic reflex feature score of user A when performing identity authentication, 55 refers to the weighted total score of the voiceprint feature score of the counterfeiter and the external auditory canal acoustic reflex feature score when performing identity authentication, and 56 refers to the weighted total score of the voiceprint feature score of the counterfeiter and the external auditory canal acoustic reflex feature score when performing identity authentication. Compared with the comparison between 51 and 52 and the comparison between 53 and 54, the distribution center distance between 55 and 56 is larger, so it is easier to identify who is the counterfeiter and who is user A based on 55 and 56, thereby effectively improving the recognition rate.
[0130] Figure 6 The flowchart of the registration method of audio verification information and the identity verification method according to the embodiment of the present application is shown. Figure 6 As shown in the figure, when a user registers an account, the identity verification system performs the following steps:
[0131] Step 610, collecting a first air conduction voiceprint feature, a first bone conduction voiceprint feature, and a first external auditory canal acoustic reflection feature of the first user;
[0132] Step 630, registering a first air conduction voiceprint feature, a first bone conduction voiceprint feature, and a first external auditory canal acoustic reflection feature.
[0133] During the user identity authentication stage, the identity authentication system performs the following steps for the user to be authenticated who initiates the identity authentication for the first user:
[0134] Step 640, collecting the air conduction voiceprint features to be verified, the bone conduction voiceprint features to be verified, and the external auditory canal acoustic reflection features to be verified of the user to be verified;
[0135] Step 650, calling the first audio verification information of the first user, wherein the first audio verification information includes a first air conduction voiceprint feature, a first bone conduction voiceprint feature, a first external auditory canal acoustic reflection feature, and a scoring threshold;
[0136] Step 660, respectively comparing the air conduction voiceprint feature to be verified and the first air conduction voiceprint feature, the bone conduction voiceprint feature to be verified and the first bone conduction voiceprint feature, and the external auditory canal acoustic reflex feature to be verified and the first external auditory canal acoustic reflex feature, and calculating a first similarity score, a second similarity score, and a third similarity score;
[0137] Step 670, weighted sum of the first similarity score, the second similarity score and the third similarity score to calculate a total score;
[0138] Step 680, compare the total score and the scoring threshold, and make an identity authentication decision based on the comparison result, wherein when the total score is higher than the scoring threshold, the identity authentication is passed.
[0139] Furthermore, in an implementation of step 2030, the voiceprint feature and the external auditory canal acoustic reflection feature are fused based on feature fusion, and the fusion result is used as audio verification information.
[0140] Specifically, the audio verification information of user A includes the audio features of user A and the audio feature matching judgment criteria (depending on actual needs, different users can correspond to the same audio feature matching judgment criteria or different audio feature matching judgment criteria). The audio feature matching judgment criteria are used to determine whether the audio features of the user to be verified match the audio features of user A (for example, the audio feature matching judgment criteria is set to a specific threshold. When the similarity between the audio features of the user to be verified and the audio features of user A exceeds the audio feature matching judgment criteria, it is determined that the audio features of the user to be verified match the audio features of user A). The audio features of user A are generated by fusion of the voiceprint features of user A and the external auditory canal acoustic reflex features of user A.
[0141] That is, in one implementation of step 2030, the process of registering the audio verification information of user A includes:
[0142] The voiceprint features of user A and the acoustic reflection features of the external auditory canal of user A are integrated to generate the audio features of user A;
[0143] Register the audio features of user A.
[0144] Correspondingly, in an implementation of step 3040, the process of verifying whether the user to be verified is the first user includes:
[0145] The voiceprint features of the user to be verified and the acoustic reflection features of the external auditory canal are integrated to generate the audio features to be verified;
[0146] Determining whether the audio feature of the user to be verified matches the audio feature in the audio verification information of the first user based on the audio feature matching determination criterion in the audio verification information of the first user;
[0147] When the audio feature of the user to be authenticated matches the audio feature in the audio authentication information of the first user, it is determined that the user to be authenticated is the first user.
[0148] Furthermore, in actual application scenarios, ideally, for a characteristic parameter of the same object, if the characteristic parameter is individually unique, then for two parameter quantities of the characteristic parameter obtained from different channels, if the two parameter quantities are the same, then they correspond to the same object. Therefore, in actual application scenarios, whether two parameter quantities correspond to the same target object is determined by comparing whether the two parameter quantities are consistent.
[0149] For example, the voiceprint feature of user A is extracted based on the voice information A1 of user A as feature quantity A11; the voiceprint feature extracted based on the voice information A2 of an unidentified user is feature quantity A12. If feature quantity A11 is the same as feature quantity A12, then the unidentified user is user A.
[0150] However, since the algorithm cannot be perfect, in different computing environments, the calculation of parameter quantities for a feature parameter of the same object may result in multiple different but similar results. Therefore, in many application scenarios, fuzzy judgment is used to determine whether two feature quantities are consistent, that is, when the similarity of two parameter quantities is higher than the preset similarity threshold, the two parameter quantities can be considered to be consistent.
[0151] For example, the voiceprint features of user A are extracted. The voiceprint features extracted based on user A's voice information A1 are feature quantity A11; the voiceprint features extracted based on user A's voice information A2 are feature quantity A12. Ideally, feature quantity A11 is the same as feature quantity A12. However, in actual situations, there may be differences between feature quantity A11 and feature quantity A12. Generally, when comparing feature quantity A11 and feature quantity A12, as long as the similarity between the two is higher than a preset similarity threshold, the two feature quantities can be considered to correspond to the same user.
[0152] One of the prerequisites for the above fuzzy judgment is that the similarity between multiple different parameter quantities obtained for the same object is higher than the similarity between multiple different parameter quantities obtained for different objects. In this way, by setting the similarity threshold to be higher than the similarity between multiple different parameter quantities obtained for different objects, it is possible to avoid judging multiple different parameter quantities obtained for different objects as corresponding to the same object.
[0153] For example, the voiceprint feature of user A is extracted. The voiceprint feature extracted based on user A's voice information A1 is feature quantity A11; the voiceprint feature extracted based on user A's voice information A2 is feature quantity A12; the voiceprint feature extracted based on user B's voice information A3 is feature quantity A13. The similarity between feature quantity A11 and feature quantity A12 should be higher than the preset similarity threshold; the similarity between feature quantity A12 and feature quantity A13 should be lower than the preset similarity threshold.
[0154] Furthermore, based on a certain characteristic parameter, when the difference between the similarity between multiple different parameter quantities obtained for different objects and the similarity between multiple different parameter quantities obtained for the same object is greater, the characteristic parameter is easier to use to distinguish different objects, and the distinguishability of the characteristic parameter is better; when the similarity between multiple different parameter quantities obtained for different objects is much smaller than the similarity between multiple different parameter quantities obtained for the same object, it is difficult to judge multiple different parameter quantities obtained for different objects as corresponding to the same object.
[0155] Therefore, in an embodiment of the present application, the parameter type of the audio verification information is preferably selected to reduce the similarity between the audio verification information obtained for different users as much as possible.
[0156] Specifically, in one implementation of step 2030, in the process of obtaining the audio verification information to be registered based on feature fusion, the voiceprint feature is fused with the external auditory canal acoustic reflection feature to generate an audio feature with better resolution than the voiceprint feature and the external auditory canal acoustic reflection feature. That is, for different users, the similarity between the audio features is lower than the similarity between the voiceprint features and the similarity between the external auditory canal acoustic reflection features. In this way, compared with using the voiceprint feature or the external auditory canal acoustic reflection feature as the audio verification information for verifying the user's identity, the verification accuracy of using the audio feature as the audio verification information for verifying the user's identity will be improved.
[0157] Specifically, in one implementation of step 2030, the acoustic reflection features of the external auditory canal and the voiceprint features are subjected to multimodal machine learning to obtain a more resolvable identity representation vector (audio feature), the identity representation vector is registered, and the identity representation vector is stored in a memory. In the identity authentication stage, the identity representation vector to be verified is extracted using the same method, and the identity representation vector to be verified is similarly calculated with the registered identity representation vector, and the identity authentication is determined based on the similarity calculation result.
[0158] Specifically, in one implementation of step 2030, a machine learning model is used to implement feature fusion. Feature fusion is to feed multiple feature vectors into the same machine learning algorithm network to extract features with stronger discrimination. Figure 7 FIG. 1 is a logic diagram of identity authentication based on feature fusion according to an embodiment of the present application. Figure 7 As shown, 71, 72, and 73 refer to the registered external auditory canal acoustic reflection features, air conduction voiceprint features, and bone conduction voiceprint features, respectively; 74, 75, and 76 refer to the external auditory canal acoustic reflection features, air conduction voiceprint features, and bone conduction voiceprint features to be verified, respectively. First, 71-72-73 and 74-75-76 are fused to extract more distinguishable registered representations 77 and representations to be verified 78, and then 77 and 78 are used to calculate the similarity score 79, and identity verification is performed based on 79.
[0159] Specifically, the development and use of identity authentication schemes based on feature fusion is divided into three stages: training, registration, and verification. Taking the neural network learning method as an example, the training stage uses identity labels and training corpus to train the recognition ability of the machine learning model, such as Figure 7 As shown, in the registration phase, the registration data is used to extract multiple feature vectors of the target speaker, and the vectors are fused into an identity vector and saved as a template. In the identity authentication phase, the identity vector of the user to be authenticated is extracted from the user's data to be authenticated, and compared with the registered identity vector template. A score is given based on the comparison result, and finally a judgment is made based on the score to give an identity authentication result.
[0160] Furthermore, in an implementation of step 2030 or step 3040, feature fusion is implemented based on a feature concatenation algorithm and a linear discriminant analysis (LDA) algorithm. Specifically, in an implementation of step 2030 or step 3040, the process of fusing the voiceprint feature of the first user or the user to be verified and the acoustic reflection feature of the external auditory canal includes:
[0161] Perform feature splicing on the voiceprint features and the external auditory canal acoustic reflection features to obtain a feature sample set;
[0162] Based on the linear discriminant analysis algorithm, the feature sample set is projected on the first dimension to obtain the first dimension projection result, wherein for different users, the similarity between the first dimension projection results is lower than the similarity between the projection results of other dimensions;
[0163] The first dimension projection result is used as the audio feature.
[0164] Figure 8 FIG. 1 is a schematic diagram showing the effect of identity authentication based on feature fusion according to an embodiment of the present application. Assume that an imposter wants to impersonate user A to pass identity authentication. Figure 8As shown, dots refer to samples of the counterfeiter, and stars refer to samples of user A. Samples of the counterfeiter after the feature splicing of external auditory canal acoustic reflection features 86, air conduction voiceprint features 87, and bone conduction voiceprint features 88 when the counterfeiter performs identity authentication; samples of user A after the feature splicing of external auditory canal acoustic reflection features, air conduction voiceprint features, and bone conduction voiceprint features when user A performs identity authentication. 81 is the distribution density function of the counterfeiter sample projected on the LD2 dimension, 82 is the distribution density function of the user A sample projected on the LD2 dimension, 83 is the distribution density function of the counterfeiter sample projected on the LD1 dimension, and 84 is the distribution density function of the user A sample projected on the LD1 dimension. After the LDA algorithm transformation, the LD1 component with high discrimination can be extracted as the final identity vector component, and the LD2 component with low discrimination can be removed to improve the recognition rate.
[0165] Fig. 9 The flowchart of the registration method of audio verification information and the identity verification method according to the embodiment of the present application is shown. Fig. 9 As shown in the figure, when a user registers an account, the identity verification system performs the following steps:
[0166] Step 910, collecting a first air conduction voiceprint feature, a first bone conduction voiceprint feature, and a first external auditory canal acoustic reflection feature of the first user;
[0167] Step 931, performing feature splicing on the first air conduction voiceprint feature, the first bone conduction voiceprint feature, and the first external auditory canal acoustic reflection feature to obtain a first feature sample set;
[0168] Step 932, projecting the first feature sample set on the first dimension based on the linear discriminant analysis algorithm to obtain a first dimension projection result;
[0169] Step 933, register the first dimension projection result.
[0170] During the user identity authentication stage, the identity authentication system performs the following steps for the user to be authenticated who initiates the identity authentication for the first user:
[0171] Step 940, collecting the to-be-verified air conduction voiceprint features, to-be-verified bone conduction voiceprint features, and to-be-verified external auditory canal acoustic reflection features of the to-be-verified user;
[0172] Step 950, calling the first audio verification information of the first user, wherein the first audio verification information includes the first dimension projection result and the scoring threshold;
[0173] Step 961, performing feature splicing on the air conduction voiceprint features to be verified, the bone conduction voiceprint features to be verified, and the external auditory canal acoustic reflection features to be verified, to obtain a feature sample set to be verified;
[0174] Step 962, based on the linear discriminant analysis algorithm, project the feature sample set to be verified on the first dimension to obtain the projection result of the dimension to be verified;
[0175] Step 970, comparing the first dimension projection result and the to-be-verified dimension projection result, and calculating a first similarity score;
[0176] Step 980, comparing the first similarity score and the score threshold, and performing identity authentication determination based on the comparison result, wherein when the total score is higher than the score threshold, the identity authentication is passed.
[0177] It is to be understood that some or all of the steps or operations in the above embodiments are merely examples, and the present application embodiments may also perform other operations or variations of various operations. In addition, the various steps may be performed in different orders presented in the above embodiments, and it is possible that not all of the operations in the above embodiments need to be performed.
[0178] Furthermore, based on the method for registering audio verification information proposed in one embodiment of the present application, one embodiment of the present application also proposes a device for registering audio verification information. Fig.10 The structure diagram of an embodiment of a registration device for audio verification information according to the present application is shown. In an embodiment of the present application, Fig.10 As shown, in one embodiment of the present application, the registration device 1000 for audio verification information includes:
[0179] A collection module 1011 is used to collect a first voiceprint feature of a first user;
[0180] A collection module 1012, which is used to collect a first external auditory canal acoustic reflection feature of a first user;
[0181] The verification information registration module 1020 is used to register the first audio verification information of the first user according to the first voiceprint feature and the first external auditory canal acoustic reflection feature.
[0182] Furthermore, an embodiment of the present application also proposes an identity authentication device. Fig.11 The structure diagram of an embodiment of the identity verification device according to the present application is shown. In an embodiment of the present application, Fig.11 As shown, in one embodiment of the present application, the identity authentication device 1100 includes:
[0183] A collection module 1111, which is used to collect the voiceprint features of the user to be verified, where the user to be verified is the user who initiates the identity authentication for the first user;
[0184] A collection module 1112, which is used to collect the external auditory canal acoustic reflection characteristics of the user to be verified;
[0185] The verification information calling module 1120 is used to call the first audio verification information of the first user, wherein the audio verification information of the first user is the audio verification information registered according to the voiceprint feature and the external auditory canal acoustic reflection feature of the first user;
[0186] The verification module 1130 is used to verify whether the user to be verified is the first user according to the voiceprint feature to be verified, the external auditory canal acoustic reflection feature to be verified and the first audio verification information.
[0187] Fig.10 as well as Fig.11 The device provided in the embodiment of the present application shown can be used to execute the technical solution of the method embodiment of the embodiment of the present application. Its implementation principle and technical effects can be further referred to the relevant description in the method embodiment.
[0188] Furthermore, in the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the access party programming the device. Designers can "integrate" a digital device on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.
[0189] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.
[0190] In the description of the embodiments of the present application, for the convenience of description, the device is described by being divided into various modules / units according to their functions. The division of each module / unit is merely a division of logical functions. When implementing the embodiments of the present application, the functions of each module / unit can be implemented in the same or one or more software and / or hardware.
[0191] Specifically, the device proposed in the embodiment of the present application can be fully or partially integrated into a physical entity during actual implementation, or it can be physically separated. And these modules can all be implemented in the form of software calling through processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software calling through processing elements, and some modules can be implemented in the form of hardware. For example, the detection module can be a separately established processing element, or it can be integrated in a chip of an electronic device. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, or they can be implemented independently. In the implementation process, each step of the above method or each of the above modules can be completed by an integrated logic circuit of hardware in a processor element or instructions in the form of software.
[0192] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more digital signal processors (DSP), or one or more field programmable gate arrays (FPGA), etc. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
[0193] Furthermore, an embodiment of the present application also proposes an identity authentication system, which includes:
[0194] like Fig.10 The audio verification information registration device of the illustrated embodiment is used to register the audio verification information of the first user;
[0195] like Fig.11 The identity authentication device of the illustrated embodiment is used to verify whether the user to be authenticated is the first user based on the audio authentication information of the first user.
[0196] Specifically, in a feasible implementation, the identity authentication system includes several host devices and an optional receiving device, and the host device can be bound to the receiving device or an account in the receiving device through wireless communication.
[0197] The air conduction microphone, bone conduction microphone, and feedback microphone can be installed on 1-3 host devices in different ways. That is, all three microphones can be installed on one host device, or they can be installed on three different host devices, or two of the microphones can be installed on one host device, and the third microphone can be installed on a second host device.
[0198] The feedback microphone and the speaker must be installed on the same host device to ensure that the feedback microphone can pick up the reflected sound played by the speaker and reflected through the external auditory canal.
[0199] The host device where the three microphones are located is connected to the receiving device through wireless communication. The host device sends the collected audio digital signal to the receiving device, or the host device converts the collected audio digital signal into voiceprint features and then sends it to the receiving device. The three microphones collect three audio signals, and the three audio signals are extracted into three features respectively. The three features are saved, fused and judged on the receiving device whether the verification is passed. When all sensors are installed on one host device, feature extraction, storage and fusion judgment can be performed on the host device.
[0200] The solution can also be implemented using only one of the feedback microphone and the air conduction microphone / bone conduction microphone. The feedback microphone and the air conduction microphone / bone conduction microphone are installed in one device or two independent devices.
[0201] If the operation can be performed directly on the host device, there is no need for a receiving device, that is, the host device directly performs the operation according to the judgment result without sending any signal, feature or result.
[0202] Fig.12 FIG. 1 is a structural diagram of an embodiment of an identity verification system according to the present application. Fig.12 As shown, in a feasible implementation of the identity authentication system, the host device in the identity authentication system is a wireless headset 112 with a remote communication function, the receiving device is a remote server 118, the wireless headset 112 communicates with the remote server 118 wirelessly through a wireless communication network, and the wireless headset 112 is bound to a user account in the remote server. The air conduction microphone 121, the speaker 119, the feedback microphone 120, and the bone conduction microphone 122 are all installed on the wireless headset 112, and the wireless headset 112 also includes a processor 111. The processor 111 integrates an external auditory canal acoustic reflection feature extraction algorithm 105, an air conduction voiceprint feature extraction algorithm 106, and a bone conduction voiceprint feature extraction algorithm 107. Each time identity authentication is required, the processor 111 will scan the digital signal 101 and play it on the speaker 119, while the feedback microphone 120 picks up the auditory canal reflected sound and converts it into a reflected sound digital signal 102, and the processor 111 controls the air conduction microphone 121 to collect the voice digital signal 103, and controls the bone conduction microphone 122 to collect the voice digital signal 104.
[0203] The external auditory canal acoustic reflection feature extraction algorithm 105 extracts the external auditory canal acoustic reflection feature 108 from the incident sound digital signal 101 and the reflected sound digital signal 102, the air conduction voiceprint extraction algorithm 106 extracts the air conduction voiceprint feature 109 from the digital signal 103, and the bone conduction voiceprint extraction algorithm 107 extracts the bone conduction voiceprint feature 110 from the bone conduction digital signal 104. The external auditory canal acoustic reflection feature 108, the air conduction voiceprint feature 109, and the bone conduction voiceprint feature 110 are sent to the server 118 through wireless communication, and extracted as the identity representation vector 114 in the multimodal fusion machine learning module 113. The identity representation vector 114 and the identity representation vector 115 saved when the user registers are sent to the scoring judgment module 116 for cosine distance scoring judgment, and a judgment result 117 is generated. The server 118 determines the identity authentication result according to the judgment result 117. When the cosine distance score exceeds the threshold, the identity authentication is considered to be passed, otherwise the identity authentication is considered to be failed. The loudspeaker 101 plays a continuously changing single-frequency sound, and the acoustic reflection characteristics of the external auditory canal are represented by extracting the envelope of the reflected sound.
[0204] Fig.13 FIG. 1 is a structural diagram of an embodiment of an identity verification system according to the present application. Fig.13In a feasible implementation of the identity authentication system, the host device in the identity authentication system is a Bluetooth wireless headset 213, and the receiving device is a mobile phone terminal 214. The Bluetooth wireless headset 213 and the mobile phone terminal 214 communicate wirelessly via Bluetooth and are bound to each other. The air conduction microphone 218, the speaker 216, and the feedback microphone 217 are all installed on the Bluetooth wireless headset 213, and the Bluetooth wireless headset 213 also includes a processor 215. The processor 215 stores the external auditory canal acoustic reflection extraction algorithm 204, the voiceprint extraction algorithm 205, the feature fusion algorithm 208, and the scoring judgment module 211. Each time verification is required, the voiceprint extraction algorithm 205 extracts the air conduction voiceprint feature 207 from the voice digital signal 203 collected by the air conduction microphone 218, and the external auditory canal acoustic reflection feature extraction algorithm extracts the external auditory canal acoustic reflection feature based on the incident sound digital signal 201 and the reflected sound digital signal 202. The air conduction voiceprint feature and the external auditory canal acoustic reflection feature are extracted as an identity representation vector 210 in the multimodal fusion machine learning module 208. The identity representation vector 210 and the identity representation vector 209 saved during registration are sent to the scoring and judgment module 211 for cosine distance scoring and judgment, and a judgment result 212 is generated. The judgment result 212 is sent to the mobile phone terminal 214 via Bluetooth, and the mobile phone terminal 214 determines whether the identity authentication is passed based on the judgment result. When the cosine distance score exceeds the threshold, the identity authentication is considered to be passed, otherwise the identity authentication is considered to be failed. The speaker 201 plays a continuously changing single-frequency sound, and the external auditory canal acoustic reflection feature is represented by extracting the envelope of the reflected sound. During registration, the identity representation vector is stored in the memory as a template, and during verification, the cosine distance similarity score is calculated between the new identity representation vector and the template.
[0205] Fig.14 FIG. 1 is a structural diagram of an embodiment of an identity verification system according to the present application. Fig.14As shown, in a feasible implementation of the identity authentication system, the host device of the identity authentication system is a multifunctional headset 313, and there is no receiving device. The air conduction microphone 318, the speaker 316, and the feedback microphone 317 are all installed on the multifunctional headset 313, and the multifunctional headset 313 also includes a processor 315. The processor 315 stores the external auditory canal acoustic reflection extraction algorithm 304, the voiceprint extraction algorithm 305, the feature fusion algorithm 308, and the scoring judgment module 311. Each time verification is required, the voiceprint extraction algorithm 305 extracts the air conduction voiceprint feature 307 from the voice digital signal 303 collected by the air conduction microphone 318, and the external auditory canal acoustic reflection feature extraction algorithm extracts the external auditory canal acoustic reflection feature based on the incident sound digital signal 301 and the reflected sound digital signal 302. The air conduction voiceprint feature and the external auditory canal acoustic reflection feature are extracted as the identity representation vector 310 in the multimodal fusion machine learning module 8. The identity representation vector 310 and the identity representation vector 309 saved during registration are sent to the scoring and judgment module 311 for cosine distance scoring and judgment, and a judgment result 312 is generated. The multifunctional earphone 313 determines whether the identity authentication is passed according to the judgment result. When the cosine distance score exceeds the threshold, the identity authentication is considered to be passed, otherwise the identity authentication is considered to be failed. The speaker 301 plays a continuously changing single-frequency sound, and the external auditory canal acoustic reflection feature is represented by extracting the envelope of the reflected sound. During registration, the identity representation vector is stored in the memory as a template, and during verification, the new identity representation vector and the template are calculated for the cosine distance similarity score.
[0206] An embodiment of the present application further proposes an electronic device, which includes a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute the method steps described in the embodiment of the present application.
[0207] Specifically, in one embodiment of the present application, the above-mentioned one or more computer programs are stored in the above-mentioned memory, and the above-mentioned one or more computer programs include instructions. When the above-mentioned instructions are executed by the above-mentioned device, the above-mentioned device executes the method steps described in the embodiment of the present application.
[0208] Specifically, in one embodiment of the present application, the processor of the electronic device may be a SOC device on a chip, which may include a central processing unit (CPU), and may further include other types of processors. Specifically, in one embodiment of the present application, the processor of the electronic device may be a PWM control chip.
[0209] Specifically, in an embodiment of the present application, the processor involved may include, for example, a CPU, a DSP, a microcontroller or a digital signal processor, and may also include a GPU, an embedded neural network processor (Neural-network Process Units, NPU) and an image signal processor (Image Signal Processing, ISP). The processor may also include necessary hardware accelerators or logic processing hardware circuits, such as ASICs, or one or more integrated circuits for controlling the execution of the program of the technical solution of the present application. In addition, the processor may have the function of operating one or more software programs, and the software programs may be stored in a storage medium.
[0210] Specifically, in one embodiment of the present application, the memory of the electronic device may be a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage devices, or may be any computer-readable medium that can be used to carry or store desired program codes in the form of instructions or data structures and can be accessed by a computer.
[0211] Specifically, in an embodiment of the present application, the processor and the memory can be combined into a processing device, more commonly, they are independent components, and the processor is used to execute the program code stored in the memory to implement the method described in the embodiment of the present application. In specific implementation, the memory can also be integrated into the processor, or independent of the processor.
[0212] Furthermore, the devices, apparatuses, means, modules or units illustrated in the embodiments of the present application may be specifically implemented by a computer chip or entity, or by a product having a certain function.
[0213] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code.
[0214] In the several embodiments provided in the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application.
[0215] Specifically, in one embodiment of the present application, a computer-readable storage medium is further provided, in which a computer program is stored. When the computer program is run on a computer, the computer executes the method provided in the embodiment of the present application.
[0216] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is run on a computer, the computer executes the method provided by the embodiment of the present application.
[0217] The embodiments described in this application are described with reference to the flowcharts and / or block diagrams of the methods, devices (apparatus), and computer program products according to the embodiments of the application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0218] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0219] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0220] It should also be noted that in the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.
[0221] In the embodiments of the present application, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, method, commodity or device including the element.
[0222] The present application may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.
[0223] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0224] Those skilled in the art will appreciate that the various units and algorithm steps described in the embodiments of the present application can be implemented by a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0225] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, apparatuses and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0226] The above is only a specific implementation of the present application. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. The protection scope of the present application should be based on the protection scope of the claims.
Claims
1. An identity authentication method, characterized in that: include: Collecting a voiceprint feature of a user to be verified, where the user to be verified is a user who initiates identity authentication for the first user; Collecting the acoustic reflection characteristics of the external auditory canal of the user to be verified, including: obtaining a transfer function between the incident sound and the reflected sound; when the gain and sensitivity of the incident sound playback and the echo sampling path are inconsistent, obtaining the path transfer function of the incident sound playback and the echo sampling path, and taking the ratio of the transfer function between the incident sound and the reflected sound to the path transfer function as the acoustic reflection characteristic of the external auditory canal; when the gain and sensitivity of the incident sound playback and the echo sampling path are consistent, taking the transfer function between the incident sound and the reflected sound as the acoustic reflection characteristic of the external auditory canal; Calling the audio verification information of the first user, wherein the audio verification information of the first user is the audio verification information registered according to the voiceprint characteristics and external auditory canal acoustic reflection characteristics of the first user; Verify whether the user to be verified is the first user based on the voiceprint characteristics, external auditory canal acoustic reflection characteristics of the user to be verified and the audio verification information of the first user.
2. The method according to claim 1, characterized in that The audio verification information of the first user includes the voiceprint feature, external auditory canal acoustic reflection feature and matching result verification standard of the first user; The verifying whether the user to be verified is the first user according to the voiceprint feature, the external auditory canal acoustic reflection feature of the user to be verified and the audio verification information of the first user includes: Match the voiceprint features of the user to be verified with the voiceprint features of the first user to generate a first matching result, match the external auditory canal acoustic reflex features of the user to be verified with the external auditory canal acoustic reflex features of the first user to generate a second matching result, merge the first matching result and the second matching result to generate a matching result to be verified, and determine whether the matching result to be verified meets the verification standard based on the matching result verification standard.
3. The method according to claim 1, characterized in that The audio verification information of the first user includes an audio feature of the first user and an audio feature matching determination criterion, wherein the audio feature of the first user is generated by fusing a voiceprint feature and an external auditory canal acoustic reflection feature of the first user; The verifying whether the user to be verified is the first user according to the voiceprint feature, the external auditory canal acoustic reflection feature of the user to be verified and the audio verification information of the first user includes: The voiceprint feature and the external auditory canal acoustic reflection feature of the user to be verified are integrated to generate the audio feature of the user to be verified, and whether the audio feature of the user to be verified matches the audio feature of the first user is determined based on the audio feature matching judgment standard.
4. The method according to claim 3, characterized in that For different users, the similarity between the audio features is lower than the similarity between the voiceprint features, and the similarity between the audio features is lower than the similarity between the external auditory canal acoustic reflection features.
5. The method according to claim 4, characterized in that The audio feature is generated by fusing the voiceprint feature and the external auditory canal acoustic reflection feature, wherein the process of fusing the voiceprint feature and the external auditory canal acoustic reflection feature includes: Performing feature splicing on the voiceprint feature and the external auditory canal acoustic reflection feature to obtain a feature sample set; Based on a linear discriminant analysis algorithm, projecting the feature sample set in a first dimension to obtain a first dimension projection result, wherein for different users, the similarity between the first dimension projection results is lower than the similarity between the projection results in other dimensions; The first dimensional projection result is used as the audio feature.
6. The method according to any one of claims 1 to 5, characterized in that The voiceprint features include air conduction voiceprint features and / or bone conduction voiceprint features.
7. The method according to any one of claims 1 to 5, characterized in that Collecting the acoustic reflection characteristics of the external auditory canal of the user to be verified includes: Playing incident sound to the external auditory canal of the user to be authenticated; Picking up the reflected sound generated by the incident sound being reflected through the external auditory canal of the user to be authenticated, and generating a reflected sound digital signal; The reflected sound digital signal is converted into the external auditory canal acoustic reflection feature.
8. The method according to claim 7, characterized in that The step of converting the reflected sound digital signal into the external auditory canal acoustic reflection feature comprises: Extracting an external auditory canal reflected sound amplitude spectrum of the reflected sound digital signal as the external auditory canal acoustic reflection feature; or, The acoustic reflection transfer function of the external auditory canal is calculated, and the acoustic reflection transfer function is used as the acoustic reflection feature of the external auditory canal.
9. The method according to claim 7, characterized in that: The incident sound is a single-frequency signal, and the reflected sound generated by the incident sound being reflected by the external auditory canal of the user to be authenticated is picked up, wherein the reflected sound is picked up by scanning a frequency range corresponding to the incident sound.
10. A method for registering audio verification information, characterized in that: include: Collecting the voiceprint features of the first user; Collecting the acoustic reflection characteristics of the first user's external auditory canal includes: obtaining a transfer function between incident sound and reflected sound; when the gain and sensitivity of the incident sound playback and the echo sampling path are inconsistent, obtaining the path transfer function of the incident sound playback and the echo sampling path, and taking the ratio of the transfer function between the incident sound and the reflected sound to the path transfer function as the acoustic reflection characteristic of the external auditory canal; when the gain and sensitivity of the incident sound playback and the echo sampling path are consistent, taking the transfer function between the incident sound and the reflected sound as the acoustic reflection characteristic of the external auditory canal; The audio verification information of the first user is registered according to the voiceprint feature and the external auditory canal acoustic reflection feature of the first user.
11. An identity verification device, characterized in that: include: A first collection module, which is used to collect voiceprint features of a user to be verified, where the user to be verified is a user who initiates identity authentication for the first user; The second acquisition module is used to acquire the acoustic reflection characteristics of the external auditory canal of the user to be verified, including: obtaining the transfer function between the incident sound and the reflected sound; when the gain and sensitivity of the incident sound playback and the echo sampling path are inconsistent, obtaining the path transfer function of the incident sound playback and the echo sampling path, and taking the ratio of the transfer function between the incident sound and the reflected sound to the path transfer function as the acoustic reflection characteristic of the external auditory canal; when the gain and sensitivity of the incident sound playback and the echo sampling path are consistent, taking the transfer function between the incident sound and the reflected sound as the acoustic reflection characteristic of the external auditory canal; A verification information calling module, which is used to call the audio verification information of the first user, wherein the audio verification information of the first user is the audio verification information registered according to the voiceprint characteristics and external auditory canal acoustic reflection characteristics of the first user; A verification module is used to verify whether the user to be verified is the first user based on the voiceprint characteristics, external auditory canal acoustic reflection characteristics of the user to be verified and the audio verification information of the first user.
12. A device for registering audio verification information, characterized in that: include: A first collection module, which is used to collect the voiceprint features of the first user; A second acquisition module, which is used to acquire the acoustic reflection characteristics of the first user's external auditory canal, including: acquiring a transfer function between an incident sound and a reflected sound; when the gain and sensitivity of the incident sound playback and the echo sampling path are inconsistent, acquiring a path transfer function of the incident sound playback and the echo sampling path, and taking a ratio of the transfer function between the incident sound and the reflected sound to the path transfer function as the acoustic reflection characteristic of the external auditory canal; when the gain and sensitivity of the incident sound playback and the echo sampling path are consistent, taking the transfer function between the incident sound and the reflected sound as the acoustic reflection characteristic of the external auditory canal; A verification information registration module is used to register the audio verification information of the first user according to the voiceprint characteristics and external auditory canal acoustic reflection characteristics of the first user.
13. An identity authentication system, characterized in that: include: The device for registering audio verification information as claimed in claim 12, which is used to register the audio verification information of the first user; The identity authentication device as described in claim 11 is used to verify whether the user to be verified is the first user based on the audio verification information of the first user.
14. An electronic device, characterized in that: The electronic device comprises a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to execute the method steps as claimed in any one of claims 1 to 10.
15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed on a computer, enables the computer to execute the method according to any one of claims 1 to 10.
