Authentication method, device, computer equipment and storage medium

Abstract

The present application relates to an identity authentication method, device, computer equipment and storage medium. The method comprises: outputting a first sound wave signal, the first sound wave signal pointing to the lips moving according to the verification password; obtaining a second sound wave signal reflected by the lips from the first sound wave signal; extracting a password signal segment based on the lip movement signal in the second sound wave signal; performing activity detection on the password signal segment, and when the result of the activity detection indicates that the lip movement is valid, extracting lip movement features based on the password signal segment; determining the identity authentication result based on the lip movement features and the registered lip movement features corresponding to the verification password. The use of this method can effectively improve the accuracy and security of identity authentication.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an identity authentication method, apparatus, computer device and storage medium. Background Art

[0002] With the rapid development of computer technology, network information security has attracted much attention, and more and more application scenarios require identity authentication. For example, in scenarios such as terminal unlocking, online payment, and access control, the user's identity needs to be verified, and a variety of biometric identity authentication methods such as voice recognition, fingerprint recognition, and face recognition have emerged.

[0003] Traditional identity authentication methods usually compare biometric features collected on-site with pre-built biometric features. However, this authentication method is vulnerable to attacks. For example, attackers can deceive deep learning-based recognition systems by generating adversarial sample features, resulting in recognition errors. Attackers can also obtain facial images or videos of target users to synthesize attack videos to conduct facial feature forgery attacks. The accuracy and security of traditional identity authentication methods cannot be effectively guaranteed. Summary of the invention

[0004] Based on this, it is necessary to provide an identity authentication method, apparatus, computer device and storage medium that can effectively improve the accuracy and security of the identity authentication method in response to the above technical problems.

[0005] An identity authentication method, the method comprising:

[0006] Outputting a first sound wave signal, wherein the first sound wave signal is directed to the lips moving according to the verification password;

[0007] Acquire a second sound wave signal obtained by reflecting the first sound wave signal through the lip;

[0008] extracting a command signal segment based on the lip movement signal in the second sound wave signal;

[0009] Performing activity detection on the command signal segment, and when a result of the activity detection indicates that the lip movement is valid, extracting lip movement features according to the command signal segment;

[0010] An identity authentication result is determined based on the lip movement feature and the registered lip movement feature corresponding to the verification password.

[0011] An identity verification device, comprising:

[0012] A signal output module, configured to output a first sound wave signal directed to the lips moving according to the verification password;

[0013] A signal acquisition module, used for acquiring a second sound wave signal generated by the first sound wave signal being reflected by the lip;

[0014] A signal extraction module, configured to extract a command signal segment based on a lip movement signal in the second sound wave signal;

[0015] an activity detection module, configured to perform activity detection on the command signal segment, and extract lip movement features according to the command signal segment when the result of the activity detection indicates that the lip movement is valid;

[0016] The identity authentication module is used to determine the identity authentication result based on the lip movement feature and the registered lip movement feature corresponding to the verification password.

[0017] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0018] Outputting a first sound wave signal, wherein the first sound wave signal is directed to the lips moving according to the verification password;

[0019] Acquire a second sound wave signal obtained by reflecting the first sound wave signal through the lip;

[0020] extracting a command signal segment based on the lip movement signal in the second sound wave signal;

[0021] Performing activity detection on the command signal segment, and when a result of the activity detection indicates that the lip movement is valid, extracting lip movement features according to the command signal segment;

[0022] An identity authentication result is determined based on the lip movement feature and the registered lip movement feature corresponding to the verification password.

[0023] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0024] Outputting a first sound wave signal, wherein the first sound wave signal is directed to the lips moving according to the verification password;

[0025] Acquire a second sound wave signal obtained by reflecting the first sound wave signal through the lip;

[0026] extracting a command signal segment based on the lip movement signal in the second sound wave signal;

[0027] Performing activity detection on the command signal segment, and when a result of the activity detection indicates that the lip movement is valid, extracting lip movement features according to the command signal segment;

[0028] An identity authentication result is determined based on the lip movement feature and the registered lip movement feature corresponding to the verification password.

[0029] The above-mentioned identity authentication method, device, computer equipment and storage medium, after outputting the first sound wave signal, the first sound wave signal points to the lips moving according to the verification password, thereby effectively obtaining the second sound wave signal reflected by the lips of the first sound wave signal. Since the second sound wave signal is a sound wave signal obtained by the verification object based on the verification password during identity authentication, the authenticity of the user can be effectively confirmed. By extracting the password signal segment based on the lip movement signal in the second sound wave signal and performing activity detection on the password signal segment, the validity of the lip movement signal can be accurately identified. When the result of the activity detection indicates that the lip movement is valid, the lip movement feature is extracted according to the password signal segment to effectively extract the dynamic lip movement feature of the verification object during the identity authentication process. The identity authentication result is determined by comparing the lip movement feature with the registered lip movement feature corresponding to the verification password. By obtaining the real-time lip movement signal of the user that is difficult to forge and extracting the corresponding lip movement feature identity authentication, the accuracy and security of the identity authentication can be effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A diagram of an application environment of an identity authentication method in an embodiment;

[0031] Figure 2 is an interactive flow chart of identity authentication between a terminal and a server in one embodiment;

[0032] Figure 3 A schematic diagram of a process flow of an identity authentication method in an embodiment;

[0033] Figure 4 A schematic flow chart of the steps of extracting lip movement signals in one embodiment;

[0034] Figure 5 It is a flowchart of the steps of performing activity detection on a password signal segment in one embodiment;

[0035] Figure 6 A schematic diagram of a flow chart of an identity authentication method in another embodiment;

[0036] Figure 7 A schematic diagram of a process flow of an identity authentication method in a specific embodiment;

[0037] Figure 8 is a structural block diagram of an identity verification device in an embodiment;

[0038] Fig. 9 is a structural block diagram of an identity verification device in another embodiment;

[0039] Fig.10 is an internal structure diagram of a server in one embodiment;

[0040] Fig.11 FIG. 4 is a diagram showing the internal structure of a terminal in an embodiment. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0042] The solution provided in the embodiment of the present application relates to technologies such as biometric recognition, liveness detection and speech recognition based on artificial intelligence. Artificial Intelligence (AI) is a theory, method, technology and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that the machines have the functions of perception, reasoning and decision-making. The key technologies of speech technology (Speech Technology) include speech separation (SS), speech enhancement (SE) and automatic speech recognition technology (ASR). Allowing computers to listen, see, speak and feel is the development direction of human-computer interaction in the future, among which speech has become one of the most promising human-computer interaction methods in the future.

[0043] The identity authentication method provided in the present application can be applied to a computer device. The computer device can be a terminal or a server. It is understandable that the identity authentication method provided in the present application can be applied to a terminal, can be applied to a server, can be applied to a system including a terminal and a server, and can be implemented through the interaction between the terminal and the server.

[0044] In one embodiment, the computer device may be a terminal, and the terminal may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, payment devices, smart home devices, and portable wearable devices. When a user uses the terminal to authenticate the identity, after the terminal outputs a first sound wave signal, the user outputs a preset verification password through the lips; after the terminal obtains a second sound wave signal reflected by the lips from the first sound wave signal, the password signal segment is extracted based on the lip movement signal in the second sound wave signal; activity detection is performed on the password signal segment, and when the result of the activity detection indicates that the lip movement is valid, the lip movement feature is extracted, and identity authentication is performed based on the lip movement feature and the registered lip movement feature corresponding to the verification password.

[0045] In one embodiment, the computer device may be a server. The identity authentication method provided by the present application may be applied to Figure 1 The application environment diagram shown in the figure includes a system of terminals and servers, and is implemented through the interaction between the terminals and the servers. Among them, the terminal 102 communicates with the server 104 through the network. The server 104 outputs a first sound wave signal through the terminal 102. The terminal 102 can also generate a first sound wave signal and then play the first sound wave signal through the speaker of the terminal. The first sound wave signal points to the lips moving according to the verification password. The microphone of the terminal 102 collects the second sound wave signal reflected by the lips of the first sound wave signal, and the server 104 obtains the second sound wave signal collected by the terminal 102, and then extracts the password signal segment based on the lip movement signal in the second sound wave signal, and performs activity detection on the password signal segment. When the result of the activity detection indicates that the lip movement is valid, the lip movement features are extracted according to the password signal segment. The server 104 determines the identity authentication result based on the lip movement features and the registered lip movement features corresponding to the verification password. Among them, the server 104 can be implemented as an independent server or a server cluster composed of multiple servers. Refer to Figure 2 , Figure 2 The figure is a flow chart of interaction between a terminal and a server for identity authentication in one embodiment.

[0046] In one embodiment, Figure 3 As shown, an identity authentication method is provided, which is illustrated by applying it to a computer device, and the computer device can specifically be a terminal or a server. Figure 3 , the authentication method includes the following steps:

[0047] S302, output a first sound wave signal, the first sound wave signal is directed to the lips moving according to the verification password.

[0048] Among them, the first sound wave signal is an ultrasonic signal. Sound wave is a mechanical wave. The propagation of vibrations generated by a sound-emitting body in the air or other substances is called sound wave, which is a form of sound propagation. Ultrasonic waves refer to mechanical waves with a vibration frequency greater than 20,000 Hz. Their vibration frequency per second is high, exceeding the general upper limit of human hearing (20,000 Hz). People usually cannot hear the propagation of ultrasonic waves. Ultrasonic signals have high frequency and short wavelength, and have good beaming and directionality when propagated within a certain distance.

[0049] Specifically, the computer device can output a first sound wave signal based on the identity authentication instruction, and specifically, the first sound wave signal can be played through the speaker of the terminal. During the identity authentication process, after the first sound wave signal is played through the speaker of the terminal, a preset verification password is output through the user's lips. Specifically, during identity authentication, the user can silently recite the verification password through the lips, and the first sound wave signal propagates in the direction of the user's lips. The first sound wave signal points to the lips moving according to the verification password, indicating that the first sound wave signal propagates in the direction of the lips, so that the lips can reflect the first sound wave signal to form a second sound wave signal. Among them, the verification password can include multiple characters or words. The password can represent evidence of verification, which is used to identify whether the verification object can pass a specific test.

[0050] By obtaining the ultrasonic signal reflected by the user's lips to detect the user's liveness, the authenticity of the user's identity can be effectively verified. Among them, liveness detection is a method of determining the real physiological characteristics of the object in some identity verification scenarios. In face recognition applications, liveness detection can verify whether the user is the real live person through combined actions such as blinking, opening the mouth, shaking the head, and nodding, using technologies such as facial key point positioning and face tracking. It can effectively resist common attack methods such as photos, face swapping, masks, occlusion, and screen reshoots, thereby helping users identify fraudulent behavior and protect their interests.

[0051] In one embodiment, before outputting the first sound wave signal, the method further includes: acquiring a preset audio signal; and performing carrier frequency randomization on the preset audio signal to generate the first sound wave signal.

[0052] The preset audio signal is a pre-configured ultrasonic signal.

[0053] Before outputting the first sound wave signal, the computer device may randomize the carrier frequency of the preset audio signal through a signal generator to generate and output the first sound wave signal. Specifically, after the computer device obtains the preset audio signal, when randomizing the carrier frequency of the preset audio signal, the preset audio signal may be toned. The specific expression may be as follows:

[0054]

[0055] Where 2A is the amplitude, f k is the carrier frequency of the signal, and N is the total number of subcarriers. We use a random number generator to generate the frequency f k. In order to avoid interference between adjacent frequency signals, the frequency interval △f between any two tones can be specified, for example, the frequency interval can be at least 300Hz. The computer device generates a first sound wave signal after tones are superimposed and carrier frequency is varied on the preset audio signal. The carrier frequency of the generated audio is randomized to resist audio playback attacks. Therefore, attackers cannot pass the identity authentication based on liveness detection by replaying previously recorded audio signals, which effectively ensures the accuracy and security of identity authentication.

[0056] In one of the embodiments, since the human ear cannot hear audio signals with frequencies higher than 18KHz, and the audio hardware of most terminal devices is not very sensitive to sounds higher than 21KHz, the frequency of the ultrasonic signal can be set within the range of 18 to 21KHz. This can effectively ensure that the output audio signal is a sound wave signal that is inaudible to the human ear, and can effectively ensure that it can be collected by the audio hardware of the terminal, thereby effectively ensuring the validity of the output first sound wave signal, so as to further effectively authenticate the user based on liveness detection.

[0057] S304: Acquire a second sound wave signal obtained by reflecting the first sound wave signal through the lips.

[0058] When a sound wave encounters a medium during propagation, it will be reflected. When a sound wave is incident from one medium to another medium with different acoustic properties, reflection will occur at the interface between the two media, causing part of the energy of the incident sound wave to return to the first medium. The second sound wave signal is the sound wave signal obtained by reflecting the first sound wave signal from the user's lips.

[0059] During the identity authentication process, the terminal used for verification is pointed at the user's face. After the first sound wave signal is output, the lips of the user will move accordingly when the user recites the verification password. Therefore, after the first sound wave signal points to the lips moving according to the verification password, the first sound wave signal will be reflected by the user's lips, and the reflected sound wave signal is the second sound wave signal. The second sound wave signal reflected by the lips of the first sound wave signal can be collected by the microphone of the terminal.

[0060] In one of the embodiments, a sound may be made when the user speaks the verification password. If the user makes a sound when speaking the verification password, the collected signal includes multiple signals such as ultrasonic signals and voice signals reflected by the user's lips. The computer device needs to perform corresponding signal processing on the collected signal to separate the signal and extract only the reflected ultrasonic signal. Specifically, the computer device can separate the collected signal according to a preset frequency band by identifying the frequency of the collected signal, thereby extracting the second sound wave signal reflected by the lips from the first sound wave signal. For example, the frequency spectrum of the voice signal is usually in the range of 100Hz to 7kHz. When the terminal collects the signal during the identity authentication process, it can collect the signal reflected by the user's lips according to a preset sampling threshold (such as 18kHz) during the signal sampling process. The collected signal is an ultrasonic signal with the voice signal filtered out.

[0061] S306: Extract a command signal segment based on the lip movement signal in the second sound wave signal.

[0062] Among them, after the first sound wave signal is output, multiple propagation paths will be generated during the propagation process. Therefore, the collected second sound wave signal reflected by the lips includes some interference sound wave signals.

[0063] When the user performs identity authentication, the user sequentially outputs a verification password containing one or more words through the lips. In order to detect effective lip movements, the sound signal needs to be segmented into signal segments corresponding to each word of the password.

[0064] The password signal segment may specifically be a sound wave signal segment corresponding to each word in the verification password. The multiple is at least two or more. The verification password may include Chinese, numbers, English and other characters. The word represents a character or string with an independent meaning, which may specifically be a word or a phrase.

[0065] Specifically, after the computer device obtains the second sound wave signal, it performs signal preprocessing and signal extraction on the second sound wave signal to extract the lip movement signal corresponding only to the user's lip movement in the second sound wave signal, and then extracts the password signal segment based on the lip movement signal in the second sound wave signal. Specifically, the computer device extracts the password signal segment by performing endpoint detection on the lip movement signal, for example, VAD (Voice activity detection) can be used. Voice activity detection is a technology used for speech processing, and its purpose is to detect whether a speech signal exists. The lip movement signal is segmented by performing endpoint detection on the lip movement signal, thereby extracting a segment of each word in the verification password.

[0066] S308, performing activity detection on the command signal segment, and when the result of the activity detection indicates that the lip movement is valid, extracting lip movement features according to the command signal segment.

[0067] Among them, the liveness detection process is used to detect whether each password signal segment is valid. The liveness detection method can be to perform lip movement detection on the password signal segment corresponding to the lip movement to confirm whether the password signal segment is a valid lip movement. By judging the validity of the lip movement, it can be determined whether the verification object is a real living user. The lip movement feature is a biometric feature that represents the lip movement feature of the user's face when the user speaks. Since each user's lip movement is unique, it is difficult for an attacker to forge this biometric feature.

[0068] Specifically, the computer device can perform activity detection on the password signal segment through a preset lip movement detection model. After the computer device detects the password signal segment corresponding to each word, it can also add a corresponding detection tag to each password signal segment. The detection tag may include two tags: valid and invalid. The computer device then uses the detection tag of each password signal segment to generate the result of the activity detection. When the number of valid detection tags reaches a preset threshold, it means that the password signal segment meets the activity detection condition, and then it can be determined that the result of the activity detection indicates that the lip movement is valid. For example, in a specific environment, because the signal-to-noise ratio may be low, when the number of valid detection tags exceeds half, it can be determined that the result of the activity detection indicates that the lip movement is valid.

[0069] In one of the embodiments, when the detection tags of each password signal segment satisfy the activity detection condition, it is determined that the result of the activity detection indicates that the lip movement is valid.

[0070] Since the number of words in each verification password is known, it is necessary to detect the password signal segment corresponding to each word in the verification password to obtain the activity detection result of each password signal segment. In order to ensure the effectiveness and security of password verification, the user is considered to have passed the activity detection only when all the segments corresponding to the words in the verification password are detected. Specifically, when the detection tags of each password signal segment are valid detection tags, it is determined that the password signal segment meets the activity detection conditions, and at this time it can be determined that the result of the activity detection indicates that the lip movement is valid.

[0071] In this embodiment, by performing activity detection on all password signal segments, it is possible to accurately and effectively detect whether the user's lip movement is valid, thereby effectively enhancing the accuracy and security of identity authentication.

[0072] When the result of the activity detection indicates that the lip movement is valid, it means that the user's lip movement is valid, that is, the user has passed the activity detection. The computer device further extracts lip movement features based on the command signal segment.

[0073] Specifically, the computer device extracts features from the command signal segments corresponding to the lip movement signals, and specifically, all the command signal segments may be fused and then extracted to obtain the user's lip movement features. The extracted lip movement features may be energy band time-frequency features.

[0074] In one embodiment, extracting lip movement features based on command signal segments includes: splicing the command signal segments corresponding to the lip movement signal to obtain a spliced ​​lip movement command signal; and extracting features from the spliced ​​lip movement command signal to obtain lip movement features.

[0075] The computer device extracts multiple command signal segments from the second sound wave signal, and after the result of the activity detection indicates that the lip movement is valid, the command signal segments corresponding to all words are further spliced ​​together to obtain a spliced ​​signal, and then feature extraction is performed on the spliced ​​signal to extract the lip movement features corresponding to the lip movement command signal.

[0076] Specifically, the computer device merges and splices all the command signal segments corresponding to the words to obtain the spliced ​​command signal, and performs frequency domain transformation on the spliced ​​command signal to obtain the command signal after frequency domain transformation. For example, after obtaining the spliced ​​command signal, the spliced ​​signal can be converted from the time domain to the frequency domain using a short-time Fourier transform with a window size of 1000ms and an overlap size of 875ms. The computer device further extracts the energy band time-frequency features of the command signal after frequency domain transformation. For example, the energy value in the spectrum can be first normalized to between 0 and 1; then the cumulative energy value within the range of 0.03-0.99 at each time point is calculated, where the cumulative energy value is half of the cumulative energy value of all frequencies; finally, the frequency domain centroids of all frequencies are obtained, and the frequency domain centroids of all time points are combined together to obtain the energy band time-frequency features. The obtained energy band time-frequency features are the lip movement features, so that the lip movement features corresponding to the lip movement command signal can be effectively extracted.

[0077] S310, determining an identity authentication result based on the lip movement features and the registered lip movement features corresponding to the verification password.

[0078] The computer device stores the registered lip motion features of the verification object. The registered lip motion features are the lip motion features corresponding to the verification password that the user authenticates in advance during registration. The registered lip motion features are used to verify the lip motion features obtained during verification.

[0079] Specifically, after the computer device obtains the lip movement features of the user during identity authentication, the lip movement features are compared with the registered lip movement features corresponding to the verification password to obtain a comparison result. Specifically, the computer device can calculate the similarity between the current lip movement features and the registered lip movement features, and determine the comparison result according to the similarity. The comparison result includes a consistent comparison and an inconsistent comparison. For example, when the similarity between the lip movement features and the registered lip movement features reaches a similarity threshold, the comparison result is determined to be a consistent comparison. When the comparison result is a consistent comparison, the identity authentication result is determined to be a successful verification; when the comparison result is an inconsistent comparison, the identity authentication result is determined to be a failed verification. By obtaining the lip movement features of the user during authentication and comparing the lip movement features with the registered lip movement features corresponding to the verification password, the user's identity can be effectively authenticated.

[0080] In one of the embodiments, during the registration process, the user constructs a user authentication file using the extracted registered lip movement features, and each legitimate user has his or her own unique file. The user authentication file is used to verify whether the corresponding authentication object is legitimate when performing identity authentication. After confirmation of legitimacy, that is, when the identity authentication result is a successful authentication, the computer device can also use the lip movement features extracted from this identity authentication to update the user authentication file. Specifically, the computer device can integrate the lip movement features of this time into the registered lip movement features of the user authentication file according to preset weights to enhance the validity and security of the user authentication file. Due to the uniqueness of the user's lip movements, it is almost impossible for an attacker to bypass consistency verification by impersonating a legitimate user.

[0081] In the above identity authentication method, after the computer device outputs the first sound wave signal, the first sound wave signal points to the lips moving according to the verification password, thereby effectively obtaining the second sound wave signal reflected by the lips of the first sound wave signal. Since the second sound wave signal is a sound wave signal obtained by the verification object based on the verification password during identity authentication, the authenticity of the user can be effectively confirmed. By extracting the password signal segment based on the lip movement signal in the second sound wave signal and performing activity detection on the password signal segment, the validity of the lip movement signal can be accurately identified. When the result of the activity detection indicates that the lip movement is valid, the lip movement feature is extracted according to the password signal segment to effectively extract the dynamic lip movement feature of the verification object during the identity authentication process. The identity authentication result is determined by comparing the lip movement feature with the registered lip movement feature corresponding to the verification password. By obtaining the real-time lip movement signal of the user that is difficult to forge and extracting the corresponding lip movement feature identity authentication, the accuracy and security of the identity authentication can be effectively improved.

[0082] In one embodiment, extracting the password signal segment based on the lip movement signal in the second sound wave signal includes: extracting the lip movement signal in the second sound wave signal; and extracting the password signal segment corresponding to the verification password in the lip movement signal.

[0083] The second sound wave signal reflected by the lips of the first sound wave signal includes sound wave signals propagated through multiple paths, such as a reflection path of the user's lips, a propagation path of solids (such as the user's face, etc.), an air propagation path, and a reflection path of surrounding objects. Some interfering sound wave signals are included. Therefore, the computer device needs to extract the lip movement signal related only to the user's lip movement from the second sound wave signal.

[0084] Specifically, the computer device can down-convert and demodulate the acquired second sound wave signal to obtain a baseband signal, and then eliminate multipath interference to obtain the sound signal component related only to the lip movement, thereby extracting the lip movement signal in the second sound wave signal. The computer device performs endpoint detection on the lip movement signal to extract the password signal segment. Specifically, when the user speaks the verification password through the lips, the words in the verification password are spoken in sequence according to the verification password. After the computer device extracts the lip movement signal, it further extracts the password signal segment corresponding to the words in the verification password. In this way, the lip movement signal related only to the user's lip movement and the password signal segment corresponding to the verification password in the reflected ultrasonic signal are accurately and effectively extracted, so that the authenticity of the user's identity can be verified.

[0085] In one embodiment, extracting the lip movement signal in the second sound wave signal includes: demodulating the second sound wave signal to obtain component signals of the second sound wave signal; and eliminating interference on the component signals to obtain the lip movement signal in the second sound wave signal.

[0086] Among them, the component signal is the signal component of the analog signal, and the component signal represents the second sound wave signal split into two or more parts. The signal can be divided into in-phase component and orthogonal component, DC component and AC component, even component and odd component, sinusoidal component and pulse component, etc. Among them, the in-phase component is the signal component with the same direction as the vector; the orthogonal component is orthogonal to the vector signal (that is, perpendicular to the in-phase component). The component signal of the second sound wave signal can specifically include the in-phase component and the orthogonal component corresponding to the second sound wave signal.

[0087] Specifically, during the face recognition process, the second sound wave signal reflected by the lips of the first sound wave signal includes multiple paths of propagation. After the computer device obtains the second sound wave signal, it can use coherent phase detection to perform frequency down-conversion demodulation to obtain the sound signal component related only to the lip movement.

[0088] For example, assuming that there are M paths in the acquired second sound wave signal Rec(t), the acquired second sound wave signal can be described by the following formula:

[0089]

[0090] Where i represents the i-th path, 2Ai(t) represents the amplitude of the sound signal in the i-th path, represents the phase shift caused by propagation delay, Represents the phase shift caused by system delay.

[0091] The original first sound wave signal output by the speaker can be regarded as a carrier signal, and the second sound wave signal Rec(t) collected by the microphone can be regarded as the superposition of multiple baseband signals after phase shift modulation. Since the generated ultrasonic signal is the superposition of audio signals with different frequencies, the audio played by the speaker can be regarded as the superposition of baseband signals with different frequencies. Since the collected signal is basically synchronized with the played output signal, the collected second sound wave signal can be demodulated using coherent detection to obtain the carrier frequency f k The component signal corresponding to the baseband signal of the second sound wave signal. The component signal includes an in-phase component I and an orthogonal component Q. The expression for calculating the in-phase component I and the orthogonal component Q can be as follows:

[0092]

[0093] Among them, F low is a low-pass filter, F down is the downsampling function. In the in-phase component I, R k (t)×cos2πf k The t part is as follows:

[0094]

[0095] The computer device then passes through a low-pass filter F low Remove R k (t)×cos2πf k t high frequency terms, and then through F down The computer device further calculates the in-phase component I of the baseband signal of the second sound wave signal. The calculation formula of the in-phase component I can be as follows:

[0096]

[0097] Similarly, the calculation formula for the orthogonal component Q can be as follows:

[0098]

[0099] By using coherent phase detection to down-convert and demodulate the acquired second sound wave signal, the collected signal can be effectively processed to extract the sound signal component corresponding to the baseband signal of the second sound wave signal. By further eliminating interference from the sound signal component, the lip movement signal related only to the user's lip movement can be accurately and effectively extracted.

[0100] In one embodiment, Figure 4 As shown, the step of extracting the lip movement signal from the second sound wave signal specifically includes the following contents:

[0101] S402: demodulate the second sound wave signal to obtain component signals of the second sound wave signal.

[0102] S404, performing dynamic interference elimination on the component signal based on the preset interception frequency to obtain the component signal after dynamic interference elimination.

[0103] S406, extracting a static component from the component signal after dynamic interference elimination, performing static interference elimination on the static component, and obtaining a lip movement signal in the second sound wave signal.

[0104] Among them, interference elimination includes dynamic interference signals and static interference signals in the second sound wave signal. Dynamic interference signals refer to signals reflected by other nearby moving objects other than the verification object in the identity authentication environment; static interference signals include signals reflected by solid propagation paths, air propagation paths, and nearby stationary objects other than the verification object in the identity authentication environment.

[0105] For the obtained in-phase component I and orthogonal component Q, in order to improve the recognition accuracy, it is necessary to remove the interference signals of other paths to retain only the signals related to the lip movement. The computer device demodulates the acquired second sound wave signal, extracts the sound signal component corresponding to the baseband signal of the second sound wave signal, and then further eliminates interference on the extracted component signal. The computer device can perform dynamic interference elimination and static interference elimination on the component signal respectively.

[0106] Specifically, the computer device can set a preset interception frequency of the filter, and perform dynamic interference elimination on the component signal based on the preset interception frequency, thereby filtering out the dynamic interference signal, and thus obtaining the component signal after dynamic interference elimination. Among them, the computer device can also eliminate dynamic interference while demodulating the baseband signal of the second sound wave signal, or it can. For example, after demodulating the second sound wave signal to obtain the corresponding component signal, dynamic interference elimination is performed. Since the movement of the human body trunk usually causes a signal frequency shift in the range of 50-200Hz, and the maximum frequency shift caused by lip movement usually does not exceed 40Hz, the low-pass filter F used for coherent detection is used. lowThe cutoff frequency is set to 40Hz, which can effectively filter out dynamic interference signals in the component signal.

[0107] After dynamic interference elimination, the component signal obtained is the superposition of the sound wave signal reflected by the user's lips and the static interference signal. The computer device further performs static interference elimination on the component signal after dynamic interference elimination.

[0108] Specifically, the I / Q component can be expressed as a constant static component I s (t) / Q s The sum of (t) and the signal reflected from the lip can be expressed as follows:

[0109]

[0110] Among them A lip (t) is the amplitude of the lip reflection signal, d lip is the propagation delay, v is the speed of sound in air, θ lip is the phase shift caused by system delay. It can also be simplified as:

[0111]

[0112] In order to eliminate the static component, the I / Q component I can be further calculated g (t) / Q g The gradient of (t):

[0113] I g (t) = A lip (t)cos(φ lip (t))-A lip (t)φ lip (t)sinφ lip (t))

[0114] Q g (t) = -A lip (t)sin(φ lip (t))-A lip (t)φ lip (t)cosφ lip (t))

[0115] Among them, A lip (t) and Φ lip (t) are A lip (t) and Φ lip (t) is the differential coefficient. Since the coefficient A lip (t) is inversely proportional to the square of the propagation distance, and the lip movement is subtle, so A lip The value of (t) hardly changes, so A lipThe value of (t) is close to zero. Therefore, I s (t) / Q s (t) can be expressed as:

[0116] I g (t) = -A lip (t)φ lip (t)sin(φ lip (t))

[0117] Q g (t) = -A lip (t)φ lip (t)cos(φ lip (t))

[0118] Finally, the minimum mean square error is used to eliminate I g (t) and Q g After the processing is completed, the signal representing the user's lip movement information can be finally obtained. In the case of no lip movement, g (t) and Q g The size of (t) is close to zero. By performing dynamic interference elimination and static interference elimination on the extracted acoustic signal components respectively, the lip motion signal only related to the user's lip motion can be accurately and effectively extracted.

[0119] In one embodiment, extracting the password signal segment corresponding to the verification password in the lip movement signal includes: performing voice activity detection on the lip movement signal to obtain the endpoints of each word in the lip movement signal; segmenting the lip movement signal according to the endpoints of each word to obtain the password signal segment corresponding to each word.

[0120] The endpoint refers to the starting point where the voice appears and the ending point where the voice disappears in the audio signal, that is, the endpoint includes the starting point and the end point of the audio signal.

[0121] The computer device obtains the second sound wave signal, extracts the lip movement signal corresponding to the user's lip movement in the second sound wave signal, and then further performs voice activity detection on the lip movement signal to obtain the endpoints of each word in the lip movement signal. The lip movement signal is then segmented according to the endpoints of each word, and the command signal segment corresponding to each word is extracted.

[0122] Specifically, the computer device can use the voice activity detection algorithm to roughly locate the lip movement segment corresponding to the active voice segment, and then determine the endpoint of each word based on the preset VAD threshold, so that the lip movement signal can be segmented according to the endpoints of each word, and the command signal segment corresponding to each word can be extracted according to the segmentation result. By identifying the endpoints corresponding to each word in the lip movement signal, the command signal segment corresponding to the verification command in the lip movement signal is extracted, so that the silent part and the noise part can be removed, so that the truly effective command signal segment in the lip movement signal can be accurately extracted.

[0123] In one embodiment, the endpoints include a starting point and an end point, and segmenting the lip movement signal according to the endpoints of each word includes: obtaining the envelope feature corresponding to the lip movement signal; obtaining the envelope difference of the envelope feature, and determining the starting point and end point of each word based on the envelope difference and a preset threshold; and extracting the command signal segment corresponding to each word according to the starting point and end point of the word.

[0124] The envelope feature refers to the spectrum envelope corresponding to the lip movement signal. The spectrum envelope is a curve formed by connecting the highest amplitude points of different frequencies. The formed curve is the spectrum envelope. The spectrum is a collection of many different frequencies, forming a wide frequency range, and the amplitudes of different frequencies may be different. The spectrum envelope can include the upper and lower envelopes corresponding to the curve formed by connecting the amplitude fixed points, and the envelope difference can be expressed as the difference between the upper and lower envelopes of the spectrum envelope.

[0125] The voice signal emitted by the user's lips is a complex multi-frequency signal, and each frequency component has a different amplitude. The signal is arranged according to the frequency, and the curve connected at the top is the spectrum envelope. The shape of the envelope changes with the sound emitted, and the shape of the spectrum envelope varies from person to person. The spectrum envelope includes multiple characteristic parameters, such as several peaks and valleys, frequency and amplitude, etc. These parameters can be used for voice endpoint recognition.

[0126] After the computer device acquires the second sound wave signal, it demodulates the second sound wave signal and eliminates interference, and extracts the lip movement signal corresponding to the user's lip movement in the second sound wave signal. Since there is an obvious time interval between two consecutive words in the signal waveform after eliminating interference, the lip movement signal can be detected. The computer device performs voice activity detection on the lip movement signal, locates the endpoints of each word, and segments the lip movement signal. The computer device can use the voice active segment detection algorithm to roughly locate the lip movement segment, and then implement word segmentation based on the signal envelope.

[0127] Specifically, the computer device can extract the spectrum envelope of the lip motion signal by using a time domain extraction method, a frequency domain extraction method, or a sub-band division extraction method, thereby extracting the envelope feature corresponding to the lip motion signal from the spectrum envelope. The computer device obtains the envelope difference of the envelope feature, determines the starting point and end point of each word based on the envelope difference and a preset threshold, and then segments the lip motion signal according to the starting point and end point of the word.

[0128] For example, you can use I g (t) and Q g (t) represents the signal waveform. The computer device first uses the voice activity detection algorithm to detect the component signal I after the interference is eliminated. g (t) and Q g (t) Perform speech endpoint detection to roughly locate the lip movement segments corresponding to the active speech segments so that the starting point and end point of each word are within these segments.

[0129] In order to accurately locate the starting point and end point of each word, it is necessary to estimate I g (t) and Q g The upper and lower envelopes of (t) are used to extract the envelope features corresponding to the lip motion signal. The difference between the upper and lower envelopes will change over time. For example, when the user's lips are open, the difference becomes larger, and when the lips are closed, the difference is close to zero. A threshold T can be set in advance for the difference between the upper and lower envelopes. d , set a threshold T for the duration of a word w When the difference becomes larger than T d When the difference is less than T, the signal point may be the starting point of the word. d And the interval between the current position and the starting point of the word is greater than T w When , the signal point can be the end point of the word. Adjust T by repeated testing d and T w to accurately determine the start and end points of each word.

[0130] Specifically, the computer device obtains the envelope difference of the envelope feature, and based on the envelope difference and the preset threshold T d and T w Determine the starting point and end point of each word, segment the lip movement signal according to the starting point and end point of the word, and extract the command signal segment corresponding to each word according to the starting point and end point of each word, thereby accurately and effectively extracting the command signal segment corresponding to each word.

[0131] In one embodiment, performing activity detection on a password signal segment includes: inputting the password signal segment into a trained lip movement detection model, extracting the lip motion vector of each password signal segment through the lip movement detection model; generating a detection label for each password signal segment according to the lip motion vector; the detection label is used to indicate whether the lip movement is valid; and outputting the activity detection result according to the detection label of each password signal segment.

[0132] Among them, the lip movement detection model can be a neural network model pre-trained based on a deep learning algorithm, such as a CNN (Convolutional Neural Network) model, an LSTM (Long Short-Term Memory) model, a DNN (Deep Neural Network) model, and an RNN (Recurrent Neural Network) model, or a combination of multiple neural network models.

[0133] The computer device inputs the extracted command signal segment into the lip movement detection model, and each command signal segment can be regarded as a feature vector. The computer device extracts the lip movement vector of each command signal segment through the lip movement detection model. Among them, one or more feature vectors corresponding to each command signal segment can be extracted respectively through one or more feature extraction layers in the lip movement detection model, and the extracted one or more feature vectors are connected and fused to obtain the lip movement vector of each command signal segment.

[0134] The computer device can determine whether the password signal segment is a valid lip movement based on the extracted lip movement vector, and add a detection tag to each password signal segment, and the detection tag is used to indicate whether the lip movement is valid. The lip movement detection model then generates the result of the activity detection of the lip movement signal in this identity authentication based on the detection tag of each password signal segment, and outputs the result of the activity detection. By using the neural network-based lip movement detection model to perform activity detection on the password signal segment, it is possible to accurately identify whether the lip movement of the verification object is valid, so as to effectively identify the validity of the lip movement signal, thereby effectively verifying the authenticity of the living user.

[0135] In one embodiment, the lip movement detection model includes a first network layer, a second network layer, and a feature connection layer, and the step of performing activity detection on the command signal segment specifically includes the following contents:

[0136] S502, inputting the command signal segment into the trained lip movement detection model.

[0137] S504: Extract a first signal feature of the password signal segment through the first network layer.

[0138] S506: Extract a second signal feature of the password signal segment through the second network layer.

[0139] S508, using a feature connection layer to perform feature connection on the first signal feature and the second signal feature to obtain a lip motion vector of the command signal segment.

[0140] S510, generating a detection tag for each command signal segment according to the lip motion vector; the detection tag is used to indicate whether the lip motion is valid; and outputting the result of activity detection according to the detection tag of each command signal segment.

[0141] The neural network-based lip movement detection model includes a first network layer, a second network layer and a feature connection layer, wherein the first network layer and the second network layer are two different neural networks for extracting different feature vectors from the command signal segment. For example, the first network layer may be a CNN model, and the second network layer may be an LSTM model.

[0142] The lip movement detection model may further include an input layer, and the computer device first inputs the password signal segment into the lip movement detection model through the input layer. When the first network layer is a CNN model and the second network layer is an LSTM model, since the length of the feature vector input into the CNN network layer must be consistent, the input layer of the lip movement detection model can convert and sample all the password signal segments into 128-dimensional feature vectors, and then input the vector corresponding to each converted password signal segment into the CNN network layer and the LSTM network layer.

[0143] Among them, the CNN network layer may include 4 convolution layers and 2 maximum pooling layers. The convolution kernel extracts the lip movement signal features in time sequence, and uses the maximum pooling layer to downsample the feature map, so that the first signal feature of the command signal segment can be extracted through CNN, and the first signal feature can be a time series sampling feature. The LSTM network layer may include a multi-layer neural network, for example, it may be two layers, each with 64 hidden neurons. Therefore, the LSTM network layer can be used as a feature extractor with a 64-dimensional output, and the output of the LSTM network layer can be a 16x128-dimensional feature. The time series features of each command signal segment can be abstracted through the LSTM network layer, so that the second signal feature of the command signal segment can be extracted through the LSTM network layer, and the second signal feature can be a signal feature based on time series.

[0144] After the computer device extracts the first signal feature and the second signal feature of each command signal segment, it further performs feature connection on the first signal feature and the second signal feature through the feature connection layer in the lip movement detection model. The feature connection layer may be a fully connected layer, which is used to output a two-dimensional vector. The first signal feature and the second signal feature are connected and fused through the feature connection layer to obtain the lip movement vector corresponding to each command signal segment. The generated lip movement vector can be used to indicate whether the lip movement of each command signal segment is valid.

[0145] In this embodiment, the lip movement detection model including multiple network layers can accurately extract features of the password signal segments, thereby accurately and effectively detecting the activity of the user's lip movement signals, thereby effectively verifying the authenticity of the living user.

[0146] In one embodiment, Figure 6 As shown, an identity authentication method is provided, which specifically includes the following steps:

[0147] S602, obtaining a facial image of a verification object.

[0148] S604: extracting current facial features of the facial image.

[0149] S606, performing face recognition on the face image based on the current face features and the target face features corresponding to the verification object.

[0150] S608, when the face recognition is successful, output a first sound wave signal, the first sound wave signal points to the lips moving according to the verification password.

[0151] S610, obtaining a second sound wave signal obtained by reflecting the first sound wave signal through the lips.

[0152] S612: Extract a command signal segment based on the lip movement signal in the second sound wave signal.

[0153] S614, performing activity detection on the command signal segment, and when the result of the activity detection indicates that the lip movement is valid, extracting lip movement features based on the command signal segment.

[0154] S616, determining the identity authentication result based on the lip movement features and the registered lip movement features corresponding to the verification password.

[0155] Among them, face recognition is a biometric technology that identifies people based on their facial features. It uses a camera or a camera to collect images or video streams containing faces, and automatically detects and follows faces in the images, and then performs face recognition on the detected faces.

[0156] When authenticating a user, the user's face may be first recognized. After successful face recognition, the user's identity may be further confirmed through lip movement detection to enhance the accuracy and security of identity authentication.

[0157] Specifically, the computer device can obtain the face image of the verification object based on the identity authentication instruction, extract the current face features of the face image using the face recognition algorithm, and compare the current face features with the target face features corresponding to the verification object to perform face recognition on the face image. Among them, the face recognition algorithm can adopt algorithms based on face feature point recognition, image recognition based on the entire face image, recognition based on a neural network model, recognition based on an illumination model, etc. Face recognition is a relatively mature technology and will not be described in detail here.

[0158] After the computer device performs face recognition on the face image, a face recognition result is obtained. The face recognition result includes face recognition success and face recognition failure. When face recognition is successful, the computer device outputs a first sound wave signal to further detect the lip movement of the user. Specifically, after the computer device outputs the first sound wave signal, the first sound wave signal points to the lips moving according to the verification password to obtain a second sound wave signal reflected by the lips of the first sound wave signal. Since the second sound wave signal is a sound wave signal obtained by the verification object based on the verification password during identity authentication, the authenticity of the living user can be effectively confirmed. By extracting the password signal segment based on the lip movement signal in the second sound wave signal and performing activity detection on the password signal segment, when the result of the activity detection indicates that the lip movement is valid, the lip movement feature is extracted according to the password signal segment, and the lip movement feature is compared with the registered lip movement feature corresponding to the verification password, thereby determining the identity authentication result.

[0159] In this embodiment, by first performing face recognition on the user and then performing lip movement detection on the user, the authenticity of the live user can be effectively confirmed. By obtaining the real-time lip movement signal of the user that is difficult to forge and extracting the corresponding lip movement feature identity verification, the accuracy of identity verification can be effectively improved. By performing dual identity verification on the user, the accuracy and security of identity verification are effectively enhanced.

[0160] In a specific embodiment, Figure 7 As shown, the authentication method includes the following steps:

[0161] S702, output a first sound wave signal, where the first sound wave signal is directed to the lips moving according to the verification password.

[0162] S704: demodulate the second sound wave signal to obtain component signals of the second sound wave signal.

[0163] S706, performing dynamic interference elimination on the component signal based on the preset interception frequency to obtain the component signal after dynamic interference elimination.

[0164] S708, extracting a static component from the component signal after dynamic interference elimination, performing static interference elimination on the static component, and obtaining a lip movement signal in the second sound wave signal.

[0165] S710, performing voice activity detection on the lip movement signal to obtain the endpoints of each word in the lip movement signal, and segmenting the lip movement signal according to the endpoints of each word.

[0166] S712, obtaining envelope features corresponding to the lip movement signal, obtaining envelope differences of the envelope features, and determining the starting point and end point of each word based on the envelope differences and a preset threshold.

[0167] S714, extracting the password signal segment corresponding to each word according to the starting point and the end point of the word.

[0168] S716, input the command signal segment into the trained lip movement detection model.

[0169] S718: Extract a first signal feature of the password signal segment through the first network layer.

[0170] S720: Extract a second signal feature of the password signal segment through the second network layer.

[0171] S722, use a feature connection layer to perform feature connection on the first signal feature and the second signal feature to obtain a lip motion vector of the command signal segment.

[0172] S724, generating a detection tag for each command signal segment according to the lip motion vector; the detection tag is used to indicate whether the lip motion is valid; and outputting the result of the activity detection according to the detection tag of each command signal segment.

[0173] S726, splicing the command signal segments corresponding to the lip movement signal, and extracting features from the spliced ​​lip movement command signal to obtain lip movement features.

[0174] S728, determining the identity authentication result based on the lip movement features and the registered lip movement features corresponding to the verification password.

[0175] In the above identity authentication method, after obtaining the second sound wave signal reflected by the lips of the first sound wave signal, the second sound wave signal is demodulated and interference eliminated, thereby accurately extracting the lip movement signal in the second sound wave signal. The password signal segment in the lip movement signal can be accurately extracted by voice activity detection, and the password signal segment is detected for activity using the lip movement detection model, thereby accurately identifying the validity of the lip movement signal. When the result of the activity detection indicates that the lip movement is valid, the lip movement features are extracted by splicing the password signal segments. The identity authentication result can be effectively determined based on the lip movement features and the registered lip movement features corresponding to the verification password. By obtaining the real-time lip movement signal of the user that is difficult to forge and extracting the corresponding lip movement feature identity authentication, the authenticity of the living user can be effectively confirmed, thereby effectively improving the accuracy and security of identity authentication.

[0176] The present application also provides an application scenario, which applies the above-mentioned identity authentication method to realize online payment. Specifically, when a user uses an application running in a terminal to shop online or pay, a payment request is initiated through the corresponding application, and the user needs to perform identity authentication when paying. The terminal generates an identity authentication instruction based on the payment request, and the terminal outputs a first sound wave signal through a speaker based on the identity authentication instruction. During identity authentication, the user faces the terminal with his face, and the user can silently recite a verification password through his lips, and the first sound wave signal points to the lips of the user moving according to the verification password. The microphone of the terminal obtains a second sound wave signal reflected by the lips of the first sound wave signal. The terminal obtains the lip movement signal in the second sound wave signal and extracts the password signal segment. The password signal segment is subjected to activity detection, and when the result of the activity detection indicates that the lip movement is valid, the lip movement feature is extracted according to the password signal segment. The identity authentication result is determined based on the lip movement feature and the registered lip movement feature corresponding to the verification password. If the identity authentication result is that the identity authentication is passed, the terminal obtains the consumption value of the payment request, and subtracts the consumption value from the value account of the user currently requesting payment, thereby completing the payment.

[0177] The present application also provides an application scenario, which applies the above-mentioned identity authentication method to realize terminal unlocking. Specifically, when the user unlocks the terminal, an unlock request is triggered to the terminal. The terminal generates an identity authentication instruction based on the unlock request, and authenticates the user based on the identity authentication instruction. Specifically, the terminal outputs a first sound wave signal through a speaker. When the user unlocks, the user faces the terminal with his face, and the user silently recites the verification password through his lips. The first sound wave signal points to the lips of the user moving according to the verification password. The microphone of the terminal obtains a second sound wave signal reflected by the lips of the first sound wave signal. The terminal extracts the password signal segment based on the lip movement signal in the second sound wave signal. The password signal segment is detected for activity, and when the result of the activity detection indicates that the lip movement is valid, the lip movement feature is extracted according to the password signal segment. The identity authentication result is determined based on the lip movement feature and the registered lip movement feature corresponding to the verification password. If the identity authentication result is that the identity authentication is passed, the terminal performs an unlocking process, thereby completing the terminal unlocking. It can be understood that the above-mentioned identity authentication method can also be applied to many other scenarios, which will not be repeated here.

[0178] It should be understood that although Figure 3-7 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 3-7 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0179] In one embodiment, Figure 8 As shown, an identity authentication device 800 is provided. The device can adopt a software module or a hardware module, or a combination of the two to become a part of a computer device. The device specifically includes: a signal output module 802, a signal acquisition module 804, a signal extraction module 806, a liveness detection module 808 and an identity authentication module 810, wherein:

[0180] A signal output module 802, configured to output a first sound wave signal, the first sound wave signal being directed to the lips moving according to the verification password;

[0181] A signal acquisition module 804 is used to acquire a second sound wave signal of the first sound wave signal reflected by the lips;

[0182] A signal extraction module 806, configured to extract a command signal segment based on the lip movement signal in the second sound wave signal;

[0183] An activity detection module 808 is used to perform activity detection on the command signal segment, and when the result of the activity detection indicates that the lip movement is valid, extract lip movement features according to the command signal segment;

[0184] The identity authentication module 810 is used to determine the identity authentication result based on the lip movement features and the registered lip movement features corresponding to the verification password.

[0185] In one embodiment, the signal extraction module 806 is further configured to extract a lip movement signal from the second sound wave signal; and extract a password signal segment corresponding to the verification password from the lip movement signal.

[0186] In one embodiment, the signal extraction module 806 is further configured to perform signal demodulation on the second sound wave signal to obtain component signals of the second sound wave signal; and perform interference elimination on the component signals to obtain the lip movement signal in the second sound wave signal.

[0187] In one embodiment, the signal extraction module 806 is also used to perform dynamic interference elimination on the component signal based on a preset interception frequency to obtain the component signal after dynamic interference elimination; extract the static component in the component signal after dynamic interference elimination, perform static interference elimination on the static component, and obtain the lip movement signal in the second sound wave signal.

[0188] In one embodiment, the signal extraction module 806 is also used to perform voice activity detection on the lip movement signal to obtain the endpoints of each word in the lip movement signal; segment the lip movement signal according to the endpoints of each word to obtain the command signal segment corresponding to each word.

[0189] In one embodiment, the endpoints include a starting point and an end point, and the envelope feature corresponding to the lip movement signal is obtained; the envelope difference of the envelope feature is obtained, and the starting point and end point of each word are determined based on the envelope difference and a preset threshold; and the command signal segment corresponding to each word is extracted according to the starting point and end point of the word.

[0190] In one embodiment, the activity detection module 808 is also used to input the command signal segment into a trained lip movement detection model, extract the lip motion vector of each command signal segment through the lip movement detection model; generate a detection label for each command signal segment according to the lip motion vector; the detection label is used to indicate whether the lip movement is valid; and output the activity detection result according to the detection label of each command signal segment.

[0191] In one embodiment, the lip movement detection model includes a first network layer, a second network layer and a feature connection layer, and the activity detection module 808 is also used to extract the first signal feature of the command signal segment through the first network layer; extract the second signal feature of the command signal segment through the second network layer; and use the feature connection layer to perform feature connection on the first signal feature and the second signal feature to obtain the lip motion vector of the command signal segment.

[0192] In one embodiment, the activity detection module 808 is further configured to determine that the result of the activity detection indicates that the lip movement is valid when the detection tag of each password signal segment satisfies the activity detection condition.

[0193] In one embodiment, the activity detection module 808 is further used to splice the command signal segments corresponding to the lip movement signal to obtain a spliced ​​lip movement command signal; and perform feature extraction on the spliced ​​lip movement command signal to obtain a lip movement feature.

[0194] In one embodiment, the signal output module 802 is further used to obtain a preset audio signal; and perform carrier frequency randomization on the preset audio signal to generate a first sound wave signal.

[0195] In one embodiment, Fig. 9 As shown, the device also includes a face recognition module 801, which is used to obtain a face image of a verification object; extract current face features of the face image; perform face recognition on the face image based on the current face features and the target face features corresponding to the verification object; the signal output module 802 is also used to output a first sound wave signal when face recognition is successful.

[0196] For the specific definition of the identity authentication device, please refer to the definition of the identity authentication method above, which will not be repeated here. Each module in the above identity authentication device can be implemented in whole or in part by software, hardware and a combination thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0197] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.10As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as sound wave signals, verification passwords, lip movement features, and registered lip movement features. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an identity authentication method is implemented.

[0198] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.11 As shown. The computer device includes a processor, a memory, a communication interface, a display screen, an input device, a speaker and a microphone connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, an identity authentication method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0199] Those skilled in the art will understand that Fig.10 and Fig.11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0200] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0201] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0202] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0203] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0204] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. An identity verification method, It is characterized in that The method comprises: Outputting a first sound wave signal, the first sound wave signal is directed to the lips moving according to the verification password, and supports the user to silently recite the verification password through the lips during identity verification; Acquire a second sound wave signal that is the result of the first sound wave signal being reflected by the lips. If the user makes a sound when speaking the verification password, the collected signal includes an ultrasonic wave signal and a voice signal reflected by the user's lips. Perform corresponding signal processing on the collected signal to separate the signal and extract only the reflected ultrasonic wave signal as the second sound wave signal. Extracting a password signal segment based on the lip movement signal in the second sound wave signal includes: extracting a lip movement signal corresponding only to the lip movement of the user in the second sound wave signal, and extracting a password signal segment corresponding to the verification password in the lip movement signal; Performing activity detection on the password signal segment, and when the result of the activity detection indicates that the lip movement is valid, extracting lip movement features according to the password signal segment, wherein the lip movement features are biometric features; An identity authentication result is determined based on the lip movement feature and the registered lip movement feature corresponding to the verification password.

2. The method according to claim 1, It is characterized in that The extracted lip movement features are energy band time-frequency features.

3. The method according to claim 2, It is characterized in that The extracting the lip movement signal from the second sound wave signal comprises: Performing signal demodulation on the second sound wave signal to obtain component signals of the second sound wave signal; Interference elimination is performed on the component signal to obtain a lip movement signal in the second sound wave signal.

4. The method according to claim 3, It is characterized in that The performing interference elimination on the component signal to obtain the lip movement signal in the second sound wave signal comprises: Performing dynamic interference elimination on the component signal based on a preset interception frequency to obtain a component signal after dynamic interference elimination; A static component is extracted from the component signal after the dynamic interference is eliminated, and static interference is eliminated from the static component to obtain a lip movement signal in the second sound wave signal.

5. The method according to claim 2, It is characterized in that The step of extracting the password signal segment corresponding to the verification password in the lip movement signal comprises: Performing voice activity detection on the lip movement signal to obtain endpoints of each word in the lip movement signal; The lip movement signal is segmented according to the endpoints of each word to obtain the command signal segments corresponding to each word.

6. The method according to claim 5, It is characterized in that The endpoints include a starting point and an end point, and segmenting the lip movement signal according to the endpoints of each word includes: Acquire envelope features corresponding to the lip motion signal; Obtaining an envelope difference of the envelope feature, and determining a starting point and an end point of each word based on the envelope difference and a preset threshold; The password signal segments corresponding to each word are extracted according to the starting point and the end point of the word.

7. The method according to claim 1, It is characterized in that The performing activity detection on the password signal segment comprises: Inputting the command signal segment into a trained lip movement detection model, and extracting a lip movement vector of each command signal segment through the lip movement detection model; Generating a detection tag for each command signal segment according to the lip motion vector; the detection tag is used to indicate whether the lip motion is valid; The result of the activity detection is output according to the detection tag of each password signal segment.

8. The method according to claim 7, It is characterized in that The lip movement detection model includes a first network layer, a second network layer and a feature connection layer, and extracting the lip movement vector of each command signal segment through the lip movement detection model includes: Extracting a first signal feature of the password signal segment through the first network layer; extracting a second signal feature of the password signal segment through the second network layer; The first signal feature and the second signal feature are feature-connected using the feature connection layer to obtain a lip motion vector of the command signal segment.

9. The method according to claim 7, It is characterized in that The method further comprises: When the detection tags of each of the password signal segments satisfy the activity detection condition, it is determined that the result of the activity detection indicates that the lip movement is valid.

10. The method according to claim 1, It is characterized in that The extracting of lip movement features according to the command signal segment comprises: splicing the command signal segments corresponding to the lip movement signal to obtain a spliced ​​lip movement command signal; Feature extraction is performed on the spliced ​​lip movement command signal to obtain lip movement features.

11. The method according to any one of claims 1 to 10, It is characterized in that Before outputting the first sound wave signal, the method further includes: Obtaining a preset audio signal; The preset audio signal is subjected to carrier frequency randomization to generate a first sound wave signal.

12. The method according to any one of claims 1 to 10, It is characterized in that The method further comprises: Obtain a face image of the verification object; Extracting current facial features of the facial image; Performing face recognition on the face image based on the current face features and the target face features corresponding to the verification object; When the face recognition is successful, a first sound wave signal is output.

13. An identity verification device, It is characterized in that The device comprises: A signal output module, configured to output a first sound wave signal, wherein the first sound wave signal points to the lips moving according to the verification password, and supports the user to silently recite the verification password through the lips during identity verification; A signal acquisition module, used to acquire a second sound wave signal reflected by the lips of the first sound wave signal. If the user makes a sound when speaking the verification password, the collected signal includes an ultrasonic signal and a voice signal reflected by the lips of the user. The collected signal is processed accordingly to separate the signal and only the reflected ultrasonic signal is extracted as the second sound wave signal. A signal extraction module, configured to extract a password signal segment based on the lip movement signal in the second sound wave signal, comprising: extracting the lip movement signal corresponding only to the user's lip movement in the second sound wave signal, and extracting the password signal segment corresponding to the verification password in the lip movement signal; an activity detection module, configured to perform activity detection on the password signal segment, and when the result of the activity detection indicates that the lip movement is valid, extract a lip movement feature according to the password signal segment, wherein the lip movement feature is a biometric feature; The identity authentication module is used to determine the identity authentication result based on the lip movement feature and the registered lip movement feature corresponding to the verification password.

14. A computer device comprising a memory and a processor, wherein the memory stores a computer program. It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 12 are implemented.

15. A computer-readable storage medium storing a computer program, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images